Shell scripts in Linux — theory with a C assignment grader, parameters that arrive split in two and an if that compares nothing (and the script that carried on as if nothing happened)

My IC2 study group had a habit: before submitting, everyone dropped their .c into a shared folder and we checked that everything compiled. By the third day of doing it by hand I’d had enough and thought: “I’ll write a script for this and be done”. I opened the editor, typed four lines and launched it, very proud of myself:

#!/bin/bash
# grade.sh: my first version (counts the submissions)
total = 0
for f in submissions/*.c; do
    total=$((total + 1))
done
echo "Submissions: $total"

This is what the terminal answered:

$ ./grade.sh
bash: ./grade.sh: Permission denied
$ chmod +x grade.sh
$ ./grade.sh
./grade.sh: line 3: total: command not found
Submissions: 5

Two scares in a row. The first, Permission denied: the file had no execute permission. The second is sneakier: total: command not found on line 3 and, even so, the script carried on as if nothing happened and printed Submissions: 5. Bash doesn’t stop when a line fails: it complains and moves on to the next one. That day I learned that a script can “work” while being wrong, and that spaces in bash are not decoration.

Today we’ll cover everything you need to write real scripts: the #!/bin/bash first line, chmod +x, variables, parameters ($1, $#, "$@"), the $? exit status, read, if with test, loops, case and functions. All with the same example: a C assignment grader that goes through a submissions/ folder with five submissions (ana.c, dani.c, laura.c, pablo.c and sergio.c) of the “sum” assignment (read two integers and print their sum). It follows on from Linux redirection and pipes: here you’ll see >, 2>, < and | inside scripts. After this, the practice is waiting for you.

What a shell script is

A shell script is a text file with terminal commands, one after another. What you’d type by hand, but saved so you can repeat it whenever you want. Bash reads it from top to bottom and runs each line as if you had typed it yourself.

And why not write a C program for this? Because the shell is already very good at what gets repeated most on a system: launching programs (gcc, ls, grep), checking whether they went well and moving files around. What in C would be a hundred lines of fork, exec and wait (you saw it in Processes in C) is a single line in bash.

Your first script: #!/bin/bash and chmod +x

Let’s start with the bare minimum. This is hello.sh:

#!/bin/bash
# hello.sh: the grader's first script
echo "IC2 grader ready"
echo "Run by: $USER"

Three parts you’ll see in every script:

• #!/bin/bash (the shebang): always on the first line, with nothing before it. It tells the system which program should read the file.
• Comments with #: bash ignores everything after it. On the first line #! is special; on the others # is a normal comment.
• Commands: echo prints a text. $USER is a variable that already exists and holds your user name.

Now let’s launch it in four different ways. stat -c '%A %n' shows only the permissions and the name (it’s like ls -l, but without size or date):

$ stat -c '%A %n' hello.sh
-rw-r--r-- hello.sh
$ ./hello.sh
bash: ./hello.sh: Permission denied
$ bash hello.sh
IC2 grader ready
Run by: sergio
$ chmod +x hello.sh
$ stat -c '%A %n' hello.sh
-rwxr-xr-x hello.sh
$ ./hello.sh
IC2 grader ready
Run by: sergio
$ hello.sh
bash: hello.sh: command not found
$ echo $PATH
/usr/local/bin:/usr/bin:/bin

Four different things happen, and all four have an explanation. Look at the picture:

• ./hello.sh without the x → Permission denied. The file is created as -rw-r--r-- (you saw this in Linux permissions: a umask of 022 never gives x to a new file).
• bash hello.sh works even without the x. Here you’re not executing the file: you’re executing bash and handing it the file so it can read it. Reading only needs r.
• chmod +x hello.sh adds x for owner, group and others (-rwxr-xr-x). From then on, ./hello.sh works: the system opens the file, sees #!/bin/bash on the first line and hands the file to /bin/bash.
• hello.sh without ./ → command not found. When you type a name with no slash, bash doesn’t look in the current folder: it looks in the $PATH folders (here /usr/local/bin, /usr/bin and /bin). The ./ means “the one in this folder”.

Variables: no spaces, and with quotes

A bash variable holds a piece of text. You create it by writing name=value and use it with a $ in front. Look at this session, which has three traps in it:

$ student=laura
$ echo "Grading $student"
Grading laura
$ echo 'Grading $student'
Grading $student
$ student = laura
bash: student: command not found
$ echo "File: $student.c"
File: laura.c
$ echo "File: $student_v2.c"
File: .c
$ echo "File: ${student}_v2.c"
File: laura_v2.c
$ n=$(ls submissions | wc -l)
$ echo "There are $n submissions"
There are 5 submissions
$ file="final project.c"
$ touch "$file"
$ rm $file
rm: cannot remove 'final': No such file or directory
rm: cannot remove 'project.c': No such file or directory
$ rm "$file"

Let’s go line by line, because half of all script errors live here:

Command	What happens	Why
student=laura	Creates the variable	No spaces around the =
echo "Grading $student"	Grading laura	Inside double quotes, $student is replaced
echo 'Grading $student'	Grading $student	Inside single quotes, nothing is replaced: the text goes as it is
student = laura	student: command not found	With spaces, bash understands “run the command student with the parameters = and laura“
echo "File: $student_v2.c"	File: .c	Bash reads the longest possible name: it looks for a variable student_v2, which doesn’t exist, and replaces it with nothing
echo "File: ${student}_v2.c"	File: laura_v2.c	The braces mark where the name ends
n=$(ls submissions | wc -l)	n is 5	$( ) runs the command and puts its output in its place

The last part of the session is the most important one. file="final project.c" stores a name with a space. touch "$file" creates it correctly. But rm $file, without quotes, tries to delete two files: final and project.c, which don’t exist. Bash replaces the variable and then splits the result at the spaces. With rm "$file" a single parameter arrives and the right file is deleted (that’s why that command prints nothing: on Linux, silence means everything went well).

Parameters: $0, $1… $# and “$@”

When you launch ./script.sh a b c, whatever follows the name are the parameters (or arguments), and the script receives them in special variables. This script shows all of them:

#!/bin/bash
# params.sh: shows what the script receives
echo "My name is: $0"
echo "First parameter: $1"
echo "Second parameter: $2"
echo "Number of parameters: $#"
echo "Looping over \"\$@\":"
for p in "$@"; do
    echo "  [$p]"
done
echo "Looping over \$*:"
for p in $*; do
    echo "  [$p]"
done

We call it with a normal parameter and another one with a space inside, and then with nothing:

$ chmod +x params.sh
$ ./params.sh ana.c "final project.c"
My name is: ./params.sh
First parameter: ana.c
Second parameter: final project.c
Number of parameters: 2
Looping over "$@":
  [ana.c]
  [final project.c]
Looping over $*:
  [ana.c]
  [final]
  [project.c]
$ ./params.sh
My name is: ./params.sh
First parameter: 
Second parameter: 
Number of parameters: 0
Looping over "$@":
Looping over $*:

• $0 is the name the script was called with (./params.sh). Very handy in usage messages.
• $1, $2… are the parameters in order. From the tenth on you have to write ${10}.
• $# is how many parameters there are: 2, even though the second one has a space, because the quotes in the call turn it into a single one.
• "$@" is all the parameters, each one whole. The loop runs twice.
• $* without quotes joins them all and splits them again at the spaces. The loop runs three times and breaks final project.c in two.

In the second call there are no parameters: $1 and $2 are empty strings (it’s not an error, there’s just nothing there) and $# is 0. That’s how a script knows it was called wrongly.

The exit status: $? and exit

This is the idea that changes the way you write scripts the most. Every Linux command ends by returning a number, its exit status:

• 0 means everything went well.
• Any other number (1, 2, 127…) means something failed. Each program chooses which number it uses for each failure.

The $? variable holds the code of the last command. Look at what gcc and ls return:

$ gcc submissions/ana.c -o ana
$ echo $?
0
$ gcc submissions/pablo.c -o pablo 2> /dev/null
$ echo $?
1
$ ls pablo
ls: cannot access 'pablo': No such file or directory
$ echo $?
2
$ gcc submissions/laura.c -o laura && echo "laura compiles"
laura compiles
$ gcc submissions/pablo.c -o pablo 2> /dev/null || echo "pablo does not compile"
pablo does not compile
$ chmod +x verify.sh
$ ./verify.sh
Usage: ./verify.sh file.c
$ echo $?
2
$ ./verify.sh submissions/nobody.c
Does not exist: submissions/nobody.c
$ echo $?
1
$ ./verify.sh submissions/ana.c
OK: submissions/ana.c exists
$ echo $?
0

Let’s check the first half: gcc with ana.c compiles → 0. With pablo.c (it’s missing a ;) it fails → 1. The 2> /dev/null throws away the error messages, because here we only care about the code. ls pablo fails because the executable wasn’t created, and ls uses 2 for “doesn’t exist”. Each program has its own numbers; the only universal thing is 0 = fine.

Then come && and ||, which decide using that code:

• command1 && command2: runs command2 only if command1 returned 0 (laura compiles).
• command1 || command2: runs command2 only if command1 failed (pablo does not compile).

Your scripts return a code too, and you choose it with exit. This is verify.sh, which uses three of them:

#!/bin/bash
# verify.sh: did I get exactly one file that exists?
if [ $# -ne 1 ]; then                 # -ne: "not equal", a NUMBER comparison
    echo "Usage: $0 file.c" >&2       # >&2: errors go to stderr
    exit 2                            # 2 = you called me wrongly
fi
if [ ! -f "$1" ]; then                # ! negates: "if it does NOT exist as a file"
    echo "Does not exist: $1" >&2
    exit 1                            # 1 = something went wrong
fi
echo "OK: $1 exists"
exit 0                                # 0 = all good

The last three calls in the session above test it: no parameters → usage message and 2; a file that doesn’t exist → 1; submissions/ana.c → OK and 0. Whoever calls the script (a person, another script or an if) can tell what happened without reading the messages.

Reading data with read

read waits for a line on standard input and stores it in a variable. This script asks for a student:

#!/bin/bash
# ask.sh: asks for a name and says whether they submitted
echo "Which student should I grade?"
read -r student                        # waits for a line and stores it in student
if [ -f "submissions/$student.c" ]; then
    echo "$student has submitted"
else
    echo "$student has not submitted"
fi

If you launch it in your terminal, it sits there waiting for you to type a name and press Enter. Here we give it the answer in two ways you already know: with <<< (a string as input) and with a pipe:

$ chmod +x ask.sh
$ ./ask.sh <<< "laura"
Which student should I grade?
laura has submitted
$ echo "marta" | ./ask.sh
Which student should I grade?
marta has not submitted

• read -r student reads a whole line and stores it in student. The -r makes backslashes \ be read literally (without it, bash treats them as escapes). Always use it.
• If you give several variables, read -r student grade, the first word goes into student and everything else into grade. You’ll use this in the exercises.
• read returns 0 if it could read and 1 when the input runs out. That’s why while read works for going through a file line by line (we’ll see it with loops).

Conditions with if and test

Here’s the surprise from the title: if compares nothing. if runs a command and checks its exit status. If it’s 0, it goes into the then; if not, into the else:

if command; then
    # command returned 0
else
    # command returned something else
fi

That’s why you can write if gcc submissions/ana.c -o ana; then. And for comparing? For that there’s a command called test, which can also be written [ (yes, [ is the name of a program, and ] is its compulsory last parameter). test prints nothing: it just returns 0 if the condition holds and 1 if it doesn’t. Look at it on its own:

$ [ -f submissions/ana.c ]
$ echo $?
0
$ [ -d submissions/ana.c ]
$ echo $?
1
$ [ 10 -gt 9 ]
$ echo $?
0
$ [[ 10 > 9 ]]
$ echo $?
1
$ [[ 10 < 9 ]]
$ echo $?
0
$ [ "laura" = "laura" ]
$ echo $?
0
$ sum=5+2
$ echo $sum
5+2
$ echo $((5 + 2))
7
$ echo $((7 / 2))
3
$ grade=7
$ grade=$((grade + 2))
$ echo $grade
9

The most important things in this session are two results that look backwards:

• [ 10 -gt 9 ] → 0 (true): -gt compares numbers, and 10 is greater than 9.
• [[ 10 > 9 ]] → 1 (false) and [[ 10 < 9 ]] → 0 (true). > and < compare strings, character by character, like a dictionary: 1 comes before 9, so "10" comes before "9".

And the maths: sum=5+2 stores the text 5+2, without adding anything. To calculate you use $(( )): $((5 + 2)) gives 7 and $((7 / 2)) gives 3, because bash only works with integers (7 divided by 2 is 3 with 1 left over). Finally, grade=$((grade + 2)) with grade=7 leaves 7 + 2 = 9.

These are the test operators you’ll use 95% of the time:

I want to know…	Operator	Example
If it exists and is a file	-f	[ -f submissions/ana.c ]
If it exists and is a folder	-d	[ -d submissions ]
If it exists and can be executed	-x	[ -x submissions/ana ]
If it exists and is not empty	-s	[ -s errors/sergio.txt ]
Numbers: equal / not equal	-eq / -ne	[ $# -ne 1 ]
Numbers: less / greater	-lt / -gt	[ "$grade" -gt 3 ]
Numbers: less or equal / greater or equal	-le / -ge	[ "$grade" -ge 5 ]
Strings: equal / different	= / !=	[ "$mode" = compile ]
Empty / non-empty string	-z / -n	[ -z "$1" ]
Negate	!	[ ! -f "$f" ]

Now everything together in a script with if, elif and else. classify.sh turns a grade into a word:

#!/bin/bash
# classify.sh: turns a grade (0-10) into a word
grade=$1
if [ -z "$grade" ]; then               # -z: the string is empty (no grade given)
    echo "Missing grade" >&2
    exit 2
elif [ "$grade" -ge 9 ]; then          # -ge: greater or equal, comparing NUMBERS
    echo "Excellent"
elif [ "$grade" -ge 7 ]; then
    echo "Very good"
elif [ "$grade" -ge 5 ]; then
    echo "Pass"
else
    echo "Fail"
fi

And this is how it answers with different grades:

$ chmod +x classify.sh
$ ./classify.sh 9
Excellent
$ ./classify.sh 6
Pass
$ ./classify.sh
Missing grade
$ ./classify.sh seven
./classify.sh: line 7: [: seven: integer expression expected
./classify.sh: line 9: [: seven: integer expression expected
./classify.sh: line 11: [: seven: integer expression expected
Fail

With 9 it goes into the first branch (9 ≥ 9). With 6 it fails the first two (6 < 9 and 6 < 7) and goes into the -ge 5 one. With no parameter, -z spots the empty string. But look at what happens with seven: -ge needs numbers, so [ complains on lines 7, 9 and 11 (integer expression expected), every condition fails and the script ends up in the else, with a Fail that isn’t true. The same warning as in the opening story: bash doesn’t stop when something fails.

Loops: for and while

for goes through a list of words. The most useful list in a grader is submissions/*.c, which bash replaces with the names of the matching files, in alphabetical order:

#!/bin/bash
# count.sh: goes through the submissions and numbers each one
n=0
for f in submissions/*.c; do             # bash expands the *, in alphabetical order
    n=$((n + 1))
    echo "$n: $(basename "$f")"
done
echo "Total: $n submissions"

while repeats as long as a command returns 0. With read it’s how you go through a file line by line. This script checks who on the students.txt list (ana, dani, laura, marta, pablo, ruben and sergio) hasn’t submitted:

#!/bin/bash
# pending.sh: who on the list has not submitted?
missing=0
while read -r student; do              # one round per line of students.txt
    if [ ! -f "submissions/$student.c" ]; then
        echo "Missing: $student"
        missing=$((missing + 1))
    fi
done < students.txt                    # the < goes on the done: it feeds the whole while
echo "Not submitted: $missing"

Both in action:

$ chmod +x count.sh pending.sh
$ ./count.sh
1: ana.c
2: dani.c
3: laura.c
4: pablo.c
5: sergio.c
Total: 5 submissions
$ ./pending.sh
Missing: marta
Missing: ruben
Not submitted: 2

In count.sh, n starts at 0 and adds 1 each round. basename "$f" removes the folder from the name (submissions/ana.c → ana.c). This is how the counter goes:

Round	f	n after	Prints
1	submissions/ana.c	1	1: ana.c
2	submissions/dani.c	2	2: dani.c
3	submissions/laura.c	3	3: laura.c
4	submissions/pablo.c	4	4: pablo.c
5	submissions/sergio.c	5	5: sergio.c

In pending.sh, the < students.txt goes after the done: that way the whole file feeds the loop and each read takes the next line. When the lines run out, read returns 1 and the while ends. Out of seven students, marta and ruben are missing: 7 − 5 = 2, which is exactly Not submitted: 2.

case: choosing by pattern

When a variable can hold many different values, a chain of elif gets unreadable. case compares a value with patterns (the same wildcards as in the terminal: *, ?, [0-9]):

#!/bin/bash
# option.sh: decides what to do based on the option
case "$1" in
    -c|--compile)                    # | separates patterns that do the same
        echo "Compile only" ;;
    -p|--test)
        echo "Compile and test" ;;
    -h|--help|"")                    # "" = nothing was passed
        echo "Usage: $0 -c | -p | -h" ;;
    *)                               # * = anything else (goes last)
        echo "Unknown option: $1" >&2
        exit 2 ;;
esac

We try it with a short option, a long one, nothing at all and one that doesn’t exist:

$ chmod +x option.sh
$ ./option.sh -c
Compile only
$ ./option.sh --test
Compile and test
$ ./option.sh
Usage: ./option.sh -c | -p | -h
$ ./option.sh -x
Unknown option: -x
$ echo $?
2

• Each branch ends with ;;. The first pattern that matches wins and the rest aren’t checked.
• | separates several patterns for the same branch: -c|--compile.
• "" matches the empty string (nothing was passed), so with no parameters you get the help.
• * matches everything, which is why it goes last: it’s the “in any other case”. With -x it prints the error and ends with exit 2, which we see with echo $?.

Functions

A function is a named chunk of script that you call as if it were a command. It receives its own parameters ($1, $2 inside the function are the call’s, not the script’s) and returns a code with return:

#!/bin/bash
# functions.sh: compile() does the work and returns a code
compile() {
    local source=$1                  # local: only exists inside the function
    local binary=${source%.c}        # removes the .c at the end: submissions/ana
    gcc -Wall "$source" -o "$binary" 2> /dev/null
    return $?                        # gcc's code becomes the function's code
}

echo "The script received: $1"
for f in submissions/ana.c submissions/pablo.c; do
    if compile "$f"; then            # if checks the code compile returns
        echo "OK     $f"
    else
        echo "ERROR  $f"
    fi
done
echo "source outside the function: '$source'"

We launch it passing hello as a parameter, to see that the function doesn’t mix it up with its own:

$ chmod +x functions.sh
$ ./functions.sh hello
The script received: hello
OK     submissions/ana.c
ERROR  submissions/pablo.c
source outside the function: ''

Notice four details:

• It’s defined before it’s used. Bash reads the script from top to bottom; if you call compile before defining it, command not found.
• $1 inside the function is the file you pass to it (submissions/ana.c), even though the script received hello as $1.
• local makes source and binary exist only inside the function. That’s why the last line prints '': outside, source doesn’t exist. Without local, the variable would “stick around” and could overwrite another one with the same name.
• return returns a code (0-255), not a text. if compile "$f" checks that code just like it would check gcc‘s. For a function to “return” a text, it prints it with echo and the caller collects it with $(compile …).

${source%.c} is an expansion that removes .c from the end: submissions/ana.c → submissions/ana. It’s the quick way to get the executable’s name.

Common mistakes

Try it in your terminal

This topic has no Python Tutor (it isn’t Python, or C), but your terminal is even better: you can write a script in ten seconds with cat and a here-document and try it out. Open a terminal in a test folder and type this (the output is the one from our run):

$ cat > greet.sh << 'EOF'
> #!/bin/bash
> echo "Hello, $1"
> echo "You passed me $# parameters"
> EOF
$ ./greet.sh Laura
bash: ./greet.sh: Permission denied
$ chmod +x greet.sh
$ ./greet.sh Laura
Hello, Laura
You passed me 1 parameters
$ ./greet.sh Ana Ruiz
Hello, Ana
You passed me 2 parameters
$ ./greet.sh "Ana Ruiz"
Hello, Ana Ruiz
You passed me 1 parameters
$ echo $?
0

What to watch for:

1. The here-document (<< 'EOF') creates the file without opening any editor. The quotes around 'EOF' make $1 and $# be saved literally in the file, without being replaced when it’s created.
2. The first ./greet.sh Laura gives Permission denied: a freshly created file never has x. After chmod +x, it works.
3. ./greet.sh Ana Ruiz is two parameters: $1 is only Ana and $# is 2. With quotes, "Ana Ruiz" is one.
4. echo $? gives 0: the script has no exit, so it returns the code of its last command, the echo, which went fine.
5. Change the script so it ends with exit 3 and check echo $? again. That’s how you see that you decide the code.

Summary

Concept	How you write it	What not to forget
First line	#!/bin/bash	On line 1, with nothing before it
Run	chmod +x s.sh and ./s.sh	Without ./ it’s searched in $PATH
Variable	x=5, "$x", ${x}_v2	No spaces around =; use it in double quotes
Output of a command	n=$(ls | wc -l)	$( ) runs and substitutes
Parameters	$0, $1, $#, "$@"	"$@" respects spaces; $* doesn’t
Exit status	$?, exit 2, return 1	0 = fine; check $? right after
Read	read -r x	Returns 1 when input runs out
Condition	if command; then … fi, [ "$n" -gt 3 ]	if checks a code; [ ] needs spaces
Maths	$((n + 1))	Integers only: 7 / 2 = 3
Loops	for f in dir/*.c, while read -r l; do … done < f	The < goes on the done
Choose	case "$1" in -c) … ;; *) … ;; esac	* goes last
Functions	f() { local x=$1; …; return 0; }	Define before use; return ≠ exit

Three ideas to take away

1. In bash, spaces mean something. They separate a command from its parameters: that’s why x = 5 fails, [ ] needs them and variables go in quotes.
2. Everything revolves around the exit status. if, while, && and || don’t look at text: they check whether the command returned 0. And your scripts should return codes that mean something.
3. Bash doesn’t stop when something fails. It complains and carries on. That’s why a good script checks its parameters at the start and ends with exit as soon as something doesn’t add up.

With this you can already read and write real scripts. In the shell scripts practice we build the full grader: compile every submission, test their outputs and give a grade, with options and exit codes. And after that, in the home stretch of IC2’s Linux, it’s time for vi and network configuration.

Next step: practiceShell scripts in Linux — practice with a grader that compiles, tests and marks C submissions (and the assignment that “compiled” without compiling)

Did this article help you? Here is how you can support SergioLearns →