The shell script

The shell script

The shell script is a text file with the execution bit set and contains the commands in the following format.

#!/bin/sh
 ... command lines

The first line specifies the shell interpreter which read and execute this file contents.

Reading shell scripts is the best way to understand how a Unix-like system works. Here, I give some pointers and reminders for shell programming. See "Shell Mistakes" (http://www.greenend.org.uk/rjk/2001/04/shell.html) to learn from mistakes.

Unlike shell interactive mode (see Section 1.5, “The simple shell command” and Section 1.6, “Unix-like text processing”), shell scripts frequently use parameters, conditionals, and loops.

12.1.1. POSIX shell compatibility

Many system scripts may be interpreted by any one of POSIX shells (see Table 1.13, “List of shell programs”). The default shell for the system is "/bin/sh" which is a symlink pointing to the actual program.

• bash(1) for lenny or older
• dash(1) for squeeze or newer

Avoid writing a shell script with bashisms or zshisms to make it portable among all POSIX shells. You can check it using checkbashisms(1).

Table 12.2. List of typical bashisms

Good: POSIX	Avoid: bashism
if [ "$foo" = "$bar" ] ; then …	if [ "$foo" == "$bar" ] ; then …
diff -u file.c.orig file.c	diff -u file.c{.orig,}
mkdir /foobar /foobaz	mkdir /foo{bar,baz}
funcname() { … }	function funcname() { … }
octal format: "\377"	hexadecimal format: "\xff"

The "echo" command must be used with following cares since its implementation differs among shell builtin and external commands.

• Avoid using command option "-e" and "-E".
• Avoid using any command options except "-n".
• Avoid using escape sequences in the string since their handling varies.

	Note
	Although "-n" option is not really POSIX syntax, it is generally accepted.

	Tip
	Use the "printf" command instead of the "echo" command if you need to embed escape sequences in the output string.

12.1.2. Shell parameters

Special shell parameters are frequently used in the shell script.

Table 12.3. List of shell parameters

shell parameter	value
$0	name of the shell or shell script
$1	first(1) shell argument
$9	ninth(9) shell argument
$#	number of positional parameters
"$*"	"$1 $2 $3 $4 … "
"$@"	"$1" "$2" "$3" "$4" …
$?	exit status of the most recent command
$$	PID of this shell script
$!	PID of most recently started background job

Basic parameter expansions to remember are followings.

Table 12.4. List of shell parameter expansions

parameter expression form	value if var is set	value if var is not set
${var:-string}	"$var"	"string"
${var:+string}	"string"	"null"
${var:=string}	"$var"	"string" (and run "var=string")
${var:?string}	"$var"	echo "string" to stderr (and exit with error)

Here, the colon ":" in all of these operators is actually optional.

• with ":" = operator test for exist and not null
• without ":" = operator test for exist only

Table 12.5. List of key shell parameter substitutions

parameter substitution form	result
${var%suffix}	remove smallest suffix pattern
${var%%suffix}	remove largest suffix pattern
${var#prefix}	remove smallest prefix pattern
${var##prefix}	remove largest prefix pattern

12.1.3. Shell conditionals

Each command returns an exit status which can be used for conditional expressions.

• Success: 0 ("True")
• Error: non 0 ("False")

	Note
	"0" in the shell conditional context means "True", while "0" in the C conditional context means "False".

	Note
	"[" is the equivalent of the test command, which evaluates its arguments up to "]" as a conditional expression.

Basic conditional idioms to remember are followings.

• "<command> && <if_success_run_this_command_too> || true"
• "<command> || <if_not_success_run_this_command_too> || true"
• A multi-line script snippet as the following

if [ <conditional_expression> ]; then
 <if_success_run_this_command>
else
 <if_not_success_run_this_command>
fi

Here trailing "|| true" was needed to ensure this shell script does not exit at this line accidentally when shell is invoked with "-e" flag.

Table 12.6. List of file comparison operators in the conditional expression

equation	condition to return logical true
-e <file>	<file> exists
-d <file>	<file> exists and is a directory
-f <file>	<file> exists and is a regular file
-w <file>	<file> exists and is writable
-x <file>	<file> exists and is executable
<file1> -nt <file2>	<file1> is newer than <file2> (modification)
<file1> -ot <file2>	<file1> is older than <file2> (modification)
<file1> -ef <file2>	<file1> and <file2> are on the same device and the same inode number

Table 12.7. List of string comparison operators in the conditional expression

equation	condition to return logical true
-z <str>	the length of <str> is zero
-n <str>	the length of <str> is non-zero
<str1> = <str2>	<str1> and <str2> are equal
<str1> != <str2>	<str1> and <str2> are not equal
<str1> < <str2>	<str1> sorts before <str2> (locale dependent)
<str1> > <str2>	<str1> sorts after <str2> (locale dependent)

Arithmetic integer comparison operators in the conditional expression are "-eq", "-ne", "-lt", "-le", "-gt", and "-ge".

12.1.4. Shell loops

There are several loop idioms to use in POSIX shell.

• "for x in foo1 foo2 … ; do command ; done" loops by assigning items from the list "foo1 foo2 …" to variable "x" and executing "command".
• "while condition ; do command ; done" repeats "command" while "condition" is true.
• "until condition ; do command ; done" repeats "command" while "condition" is not true.
• "break" enables to exit from the loop.
• "continue" enables to resume the next iteration of the loop.

	Tip
	The C-language like numeric iteration can be realized by using seq(1) as the "foo1 foo2 …" generator.

	Tip
	See Section 9.5.9, “Repeating a command looping over files”.

12.1.5. The shell command-line processing sequence

The shell processes a script roughly as the following sequence.

• The shell reads a line.
• The shell groups a part of the line as one token if it is within "…" or '…'.
• The shell splits other part of a line into tokens by the following.
  
  • Whitespaces: <space> <tab> <newline>
  • Metacharacters: < > | ; & ( )
• The shell checks the reserved word for each token to adjust its behavior if not within "…" or '…'.
  
  • reserved word: if then elif else fi for in while unless do done case esac
• The shell expands alias if not within "…" or '…'.
• The shell expands tilde if not within "…" or '…'.
  
  • "~" → current user's home directory
  • "~<user>" → <user>'s home directory
• The shell expands parameter to its value if not within '…'.
  
  • parameter: "$PARAMETER" or "${PARAMETER}"
• The shell expands command substitution if not within '…'.
  
  • "$( command )" → the output of "command"
  • "` command `" → the output of "command"
• The shell expands pathname glob to matching file names if not within "…" or '…'.
  
  • * → any characters
  • ? → one character
  • […] → any one of the characters in "…"
• The shell looks up command from the following and execute it.
  
  • function definition
  • builtin command
  • executable file in "$PATH"
• The shell goes to the next line and repeats this process again from the top of this sequence.

Single quotes within double quotes have no effect.
Executing "set -x" in the shell or invoking the shell with "-x" option make the shell to print all of commands executed. This is quite handy for debugging.