Introduction to Shell Scripting - The Basics

By Ben Okopnik

Originally published in Issue 52 of Linux Gazette, April 2000

Here's a hint. When you think your code to exec a shell function is
just not working, never, repeat NEVER send it "/etc/reboot" just to see
what happens.
 -- Elliott Evans

Introduction

Shell scripting is a fascinating combination of art and science that gives you access to the incredible flexibility and power of Linux with very simple tools. Back in the early days of PCs, I was considered quite an expert with DOS's "batch files", something I now realize was a weak and gutless imitation of Unix's shell scripts. I'm not usually much given to Microsoft-bashing - I believe that they have done some good things in their time, although their attempts to create an operating system have been pretty sad - but their BFL ("Batch File Language") was a joke by comparison. It wasn't even particularly funny.

Since scripting is an inextricable part of understanding shell usage in general, quite a bit of the material in here will deal with shell quirks, methods, and specifics. Be patient; it's all a part of the knowledge that is necessary for writing good scripts.

Philosophy of Scripting

Linux - Unix in general - is not a warm and fuzzy, non-knowledgeable-user oriented system. Rather than specifying exact motions and operations that you must perform (and thus imiting you only to the operations described), it provides you with a myriad of small tools which can be connected in a literally infinite number of combinations to achieve almost any result (I find Perl's motto of "TMTOWTDI" - There's More Than One Way To Do It - highly apropos for all of Unix). That sort of power and flexibility, of course, carries a price - increased complexity and a requirement for higher competence in the user. Just as there is an enormous difference between operating, say, a bicycle versus a super-sonic jet fighter, so is there an enormous difference between blindly following the rigid dictates of a standardized GUI and creating your own program, or shell script, that performs exactly the functions you need in exactly the way you need them.

Shell scripting is programming - but it is programming made easy, with little, if any, formal structure. It is an interpreted language, with its own syntax - but it is only the syntax that you use when invoking programs from your command line; something I refer to as "recyclable knowledge". This, in fact, is what makes shell scripts so useful: in the process of writing them, you continually learn more about the specifics of your shell and the operation of your system - and this is knowledge that truly pays for itself in the long run as well as the short.

Requirements

Since I have a strong preference for Bash, and it happens to be the default shell in Linux, that's what these scripts are written for (although I've tried to keep Bash-isms down to a minimum - most if not all of these scripts should run under plain old "sh".) Even if you use something else, that's still fine: as long as you have Bash installed, these scripts will execute correctly. As you will see, scripts invoke the shell that they need; it's part of what a well-written script does.

I'm going to assume that you're going to do all these exercises in your home directory - you don't want these files scattered all over the place where you can't find them later. I'm also going to assume that you know enough to hit the "Enter" key after each line that you type in, and that, before selecting a name for your shell script, you will check that you do not have an executable with that same name in your path (type "which bkup" to check for an executable called "bkup"). You also shouldn't call your script "test"; that's a Unix FAQ ("why doesn't my shell script/program do anything?") There's an executable in /usr/bin called "test" that does nothing - nothing obvious, that is - when invoked...

It goes without saying that you have to know the basics of file operations - copying, moving, etc. - as well as being familiar with the basic assumptions of the file system, i.e., "." is the current directory, ".." is the parent (the one above the current), "~" is your home directory, etc. You didn't know that? You do now!

Whatever editor you use, whether 'vi', 'emacs', 'mcedit' (the DOS-like editor in Midnight Commander), or any other text editor is fine; just don't save this work in some word-processing format - it must be plain text. If you're not sure, or keep getting "line noise" when you try to run your script, you can check the raw contents of the file you've created with "cat script_name" to be sure.

In order to avoid constant repetition of material, I'm going to number the lines as we go through and discuss different parts of a script file. The line numbers will not, of course, be there in the actual script.

Building a Script

Let's go over the basics of creating a script. Those of you who find this obvious and simplistic are invited to follow along anyway; as we progress, the material will become more complex - and a "refresher" never hurts. The projected audience for this article is a Linux newbie, someone who has never created a shell script before - but wishes to become a Script Guru in 834,657 easy steps. :)

In its simplest form, a shell script is nothing more than a shortcut - a list of commands that you would normally type in, one after another, to be executed at your shell prompt - plus a bit of "magic" to notify the shell that it is indeed a script.

The "magic" consists of two simple things:

1. A notation at the beginning of the script that specifies the program that is used to execute it, and
2. A change in the permissions of the file containing the script in order to make it executable.

As a practical example, let's create a script that will "back up" a specified file to a selected directory; we'll go through the steps and the logic that makes it all happen.

First, let's create the script. Start your editor with the filename you want to create:

mcedit bkup

1: #!/bin/bash

The shebang must:

• Be on the first line of the script, and
• There cannot be any whitespace before the '#!'.

There's a subtle but important point to all of this, by the way: when a script runs, it actually starts an additional bash process that runs under the current one; that process executes the script and exits, dropping you back in the original shell that spawned it. This is why a script that, for example, changes directories as it executes will not leave you in that new directory when it exits: the original shell has not been told to change directories, and you're right where you were when you started - even though the change is effective while the script runs.

To continue with our script:

2: # "bkup" - copies specified files to the user's ~/Backup
3: # directory after checking for name conflicts.

4: cp -i $1 ~/Backup

The "-i" syntax of the 'cp' command makes it interactive; that is, if we run "bkup file.txt" and a file called "file.txt" already exists in the ~/Backup directory, 'cp' will ask you if you want to overwrite it - and will abort the operation if you hit anything but the 'y' key.

The "$1" is a "positional parameter" - it denotes the first thing that you type after the script name. In fact, there's an entire list of these variables:

$0 - The name of the script being executed - in this case, "bkup".
$1 - The first parameter - in this case, "file.txt"; any parameter may
	be referred to by $<number> in this manner.
#@ - The entire list of parameters - "$1 $2 $3..."
$# - The number of parameters.

Making it Smarter

So far, our script doesn't do very much; hardly worth bothering, right? All right; let's make it a bit more useful. What if you wanted to both keep the file in the ~/Backup directory and save the new one - perhaps by adding an extension to show the "version"? Let's try that; we'll just add a line, and modify the last line as follows:

4: a=$(date +'%Y%m%d%H%M%S')
5: cp -i $1 ~/Backup/$1.$a

case do done elif else esac fi for function if in select then until while time

! { } | & * ; ( ) < > space tab

PATH PS1 PWD RANDOM SECONDS (see "man bash" for many others)

The effect of the last two lines of this script is to create a unique filename - something like file.txt.20000117221714 - that should not conflict with anything else in ~/Backup. Note that I've left in the "-i" switch as a "sanity" check: if, for some truly strange reason, two file names do conflict, "cp" will give you a last-ditch chance to abort. Otherwise, it won't make any difference - like dead yeast in beer, it causes no harm even if it does nothing useful.

By the way, the older version of the $(command) construct - the `command` (note that "back-ticks" are being used rather than single quotes) - is more or less deprecated. $()s are easily nested - $(cat $($2$(basename file1 txt))), for example; something that cannot be done with back-ticks, since the second back-tick would "close" the first one and the command would fail, or do something unexpected. You can still use them, though - in single, non-nested substitutions (the most common kind), or as the innermost or outermost pair of the nested set - but if you use the new method exclusively, you'll always avoid that error.

So, let's see what we have so far, with whitespace added for readability and the line numbers removed (hey, an actual script!):

#!/bin/bash

# "bkup" - copies specified files to the user's ~/Backup
# directory after checking for name conflicts.

a=$(date +'%Y%m%d%H%M%S')
cp -i $1 ~/Backup/$1.$a

chmod +x bkup

bash: bkup: command not found

./bkup file.txt

I read the Bash man page each day like a Jehovah's Witness reads the
Bible. No wait, the Bash man page IS the bible. Excuse me...
 -- More on confusing aliases, taken from comp.os.linux.misc