Shell scripts in Linux — exercises with parameters that contain spaces, grades that need validating and a list of students to mark (and the file called “3” that appeared in the folder)

One Monday I opened my IC2 group’s shared folder and there was a new, empty file called 3. Nobody had created it. Or so we thought. I deleted it, ran my summary script again and… there it was once more. The culprit was a line that looked harmless: if [ $correct > 3 ]; then. Inside the brackets, > doesn’t compare: it redirects. Bash was creating the file 3 and, on top of that, the condition always came out true.

That’s what scripts are like: what looks like a comparison can be a redirection, what looks like one parameter can be two, and a $? checked one line too late says the opposite of what happened. In these exercises you’ll write three scripts for the assignment grader and hunt down three bugs of the kind that sneak into real scripts.

How to use this article

1. Read the statement and the expected output for each level. That output is real: it was obtained by running the solution once, as the user sergio, with LANG=C.
2. Write your script in a test folder with the same structure (submissions/ with ana.c, dani.c, laura.c, pablo.c and sergio.c, the “sum” assignment from the practice). Only use the hint if you get stuck.
3. Do the three “What’s wrong here?” snippets predicting the result before reading the explanation.
4. Compare with the solutions and keep the cheat sheet at the end.

Level 1 (basic): information about several .c files

Write info.sh, which takes one or more .c files as parameters and, for each one, prints how many lines it has and whether it contains int main. If a file doesn’t exist, it says so and moves on to the next. With no parameters, a usage message to stderr and code 2. The script ends with code 0 if they all exist and have main, and 1 if not. It has to work with names containing spaces, like final project.c (here, a one-line file with an #include).

Expected output:

$ printf '#include <stdio.h>\n' > "final project.c"
$ chmod +x info.sh
$ ./info.sh
Usage: ./info.sh file.c [file.c ...]
$ echo $?
2
$ ./info.sh submissions/ana.c "final project.c" submissions/nobody.c
submissions/ana.c: 8 lines, has main
final project.c: 1 lines, NO main
submissions/nobody.c: does not exist
$ echo $?
1
$ ./info.sh submissions/*.c
submissions/ana.c: 8 lines, has main
submissions/dani.c: 8 lines, has main
submissions/laura.c: 8 lines, has main
submissions/pablo.c: 8 lines, has main
submissions/sergio.c: 9 lines, has main
$ echo $?
0

Level 2 (intermediate): grades and average

Write grades.sh, which takes a file with student grade lines. For each line, if the grade is a number from 0 to 10, it prints the label (Excellent ≥ 9, Very good ≥ 7, Pass ≥ 5, Fail); if not, it warns that it isn’t valid and skips it. At the end, the number of valid grades, passes and invalid ones, and the average with one decimal place. Code 2 if the parameter is missing, 1 if the file doesn’t exist or there are invalid grades, and 0 if everything is correct. The grades.txt file is:

ana 10
dani 5
laura 10
marta NP
pablo 0
sergio 10

Expected output:

$ chmod +x grades.sh
$ ./grades.sh grades.txt
ana: 10 Excellent
dani: 5 Pass
laura: 10 Excellent
marta: invalid grade (NP)
pablo: 0 Fail
sergio: 10 Excellent
Valid grades: 5. Passed: 4. Invalid: 1
Average: 7.0
$ echo $?
1
$ ./grades.sh grades.csv
Does not exist: grades.csv
$ echo $?
1

Level 3 (challenge): mark the whole list

Write check_list.sh list folder: it goes through students.txt (ana, dani, laura, marta, pablo, ruben and sergio) and, for each student, prints their mark: NOT SUBMITTED if there’s no .c, 0 if it doesn’t compile, 5 if it compiles but its output with input.txt isn’t expected.txt, and 10 if it’s correct. Name aligned in 8 columns. At the end, how many are on the list, how many submitted and how many are right. Code 2 if there aren’t two parameters, 1 if someone hasn’t submitted, 0 if everybody has.

Expected output:

$ chmod +x check_list.sh
$ ./check_list.sh students.txt
Usage: ./check_list.sh list folder
$ echo $?
2
$ ./check_list.sh students.txt submissions
ana      10
dani     5
laura    10
marta    NOT SUBMITTED
pablo    0
ruben    NOT SUBMITTED
sergio   10
On the list: 7. Submitted: 5. Correct: 3
$ echo $?
1

What’s wrong here?

Three short scripts with one bug each. Before reading the explanation, predict what each one prints.

Snippet 1: the usage warning that never warns

#!/bin/bash
# one.sh: warns if I don't get exactly one folder
if [$# -ne 1]; then
    echo "Usage: $0 folder"
    exit 2
fi
echo "Grading folder $1"

We launch it with a folder and with nothing. What do you think happens in each case? This is the real output:

$ chmod +x one.sh
$ ./one.sh submissions
./one.sh: line 3: [1: command not found
Grading folder submissions
$ ./one.sh
./one.sh: line 3: [0: command not found
Grading folder 

Bash has read [$# as a single word. After substituting $#, the command it tries to run is called [1 (or [0), which doesn’t exist: command not found. That error returns 127, which isn’t 0, so the if goes to the else… which doesn’t exist, and the script carries on. That’s why, with no parameters, it prints Grading folder with a gap: it hasn’t warned about anything.

The fix is just that, two spaces:

#!/bin/bash
# one.sh: warns if I don't get exactly one folder
if [ $# -ne 1 ]; then                 # space after [ and before ]
    echo "Usage: $0 folder"
    exit 2
fi
echo "Grading folder $1"

And now it does:

$ chmod +x one.sh
$ ./one.sh submissions
Grading folder submissions
$ ./one.sh
Usage: ./one.sh folder

Snippet 2: more than 3 with only 2

#!/bin/bash
# compare.sh: did more than 3 assignments pass?
correct=2
if [ $correct > 3 ]; then
    echo "More than 3 correct"
else
    echo "3 or fewer"
fi

With correct=2, it should print “3 or fewer”, right? This is the real output, and what’s in the folder afterwards:

$ chmod +x compare.sh
$ ./compare.sh
More than 3 correct
$ ls
3
compare.sh
$ cat 3

It prints More than 3 correct and a file called 3 appears, empty (that’s why cat 3 prints nothing). It’s the story from the start. Inside [ ], > isn’t “greater than”: it’s the usual redirection. Bash substitutes the variable, sets aside the > 3, creates the file 3 for [‘s output and runs [ 2 ]. With a single parameter, test checks whether that string is non-empty, and 2 isn’t empty: true, code 0, then.

The fix uses -gt and, while we’re at it, quotes:

#!/bin/bash
# compare.sh: did more than 3 assignments pass?
correct=2
if [ "$correct" -gt 3 ]; then       # -gt compares numbers; > inside [ ] redirects
    echo "More than 3 correct"
else
    echo "3 or fewer"
fi

The output is now correct, and no 3 appears in the folder:

$ chmod +x compare.sh
$ ./compare.sh
3 or fewer
$ ls
compare.sh

Snippet 3: pablo compiles (but doesn’t)

#!/bin/bash
# builds.sh: says whether a submission compiles
gcc -Wall "$1" -o prog 2> /dev/null
echo "Compilestion finished"
if [ $? -eq 0 ]; then
    echo "Compiles"
else
    echo "Does not compile"
fi

We pass it submissions/pablo.c, which doesn’t compile. Real output:

$ chmod +x builds.sh
$ ./builds.sh submissions/pablo.c
Compilestion finished
Compiles

Compiles. How can that be? Because $? is the code of the last command, and between gcc and the if there’s an echo. The if is checking the echo‘s code, which always went fine (0). gcc‘s 1 was lost as soon as the next command ran.

Two possible fixes: if gcc …; then directly, or saving the code in a variable right after. Here we use the second, because we want to print the message before deciding:

#!/bin/bash
# builds.sh: says whether a submission compiles
gcc -Wall "$1" -o prog 2> /dev/null
rc=$?                                 # save the code RIGHT after gcc
echo "Compilestion finished"
if [ "$rc" -eq 0 ]; then
    echo "Compiles"
else
    echo "Does not compile"
fi

With pablo and with ana:

$ chmod +x builds.sh
$ ./builds.sh submissions/pablo.c
Compilestion finished
Does not compile
$ ./builds.sh submissions/ana.c
Compilestion finished
Compiles

Solutions

Level 1 solution

#!/bin/bash
# info.sh: for each .c file it receives, counts its lines and checks for main
# Usage: ./info.sh file.c [file.c ...]

if [ $# -eq 0 ]; then                 # no parameters, nothing to check
    echo "Usage: $0 file.c [file.c ...]" >&2
    exit 2
fi
problems=0
for f in "$@"; do                     # "$@": each parameter whole, even with spaces
    if [ ! -f "$f" ]; then
        echo "$f: does not exist"
        problems=$((problems + 1))
        continue
    fi
    lines=$(wc -l < "$f")             # with < wc prints only the number, no name
    if grep -q "int main" "$f"; then  # -q: quiet; if only checks grep's code
        echo "$f: $lines lines, has main"
    else
        echo "$f: $lines lines, NO main"
        problems=$((problems + 1))
    fi
done
[ "$problems" -eq 0 ]                 # 0 if all exist and have main

Why it’s done this way:

• for f in "$@": it’s the only way for final project.c to arrive whole. With $* or unquoted $@, the loop would try to check final and project.c separately, and two of the results would say does not exist.
• [ $# -eq 0 ] at the start: with no files there’s nothing to do, so usage message to stderr and exit 2 before entering the loop.
• continue when the file doesn’t exist: it warns, counts the problem and moves to the next one. There’s no point counting the lines of a file that doesn’t exist.
• wc -l < "$f": with <, wc reads from its input and doesn’t know the file’s name, so it prints only the number. With wc -l "$f" you’d get 8 submissions/ana.c and would have to trim it.
• if grep -q "int main" "$f": grep returns 0 if it finds the text and 1 if not. The -q stops it from printing the matching line. It’s the if from the theory: it doesn’t compare anything, it checks a code.
• The final code comes from [ "$problems" -eq 0 ]. In the second call there are 2 problems (final project.c without main and nobody.c, which doesn’t exist), so it returns 1. In the third, all five submissions have main: 0. The line counts add up: four of the programs have 8 and sergio’s has 9, because of its extra int x;.

Level 2 solution

#!/bin/bash
# grades.sh: reads "student grade" from a file, labels it and computes the average
# Usage: ./grades.sh file

[ $# -eq 1 ] || { echo "Usage: $0 file" >&2; exit 2; }
[ -f "$1" ] || { echo "Does not exist: $1" >&2; exit 1; }

sum=0
n=0
passed=0
invalid=0
while read -r student grade; do        # read splits the line: 1st word and the rest
    case "$grade" in
        [0-9]|10) ;;                   # one digit or 10: a valid grade, carry on
        *)
            echo "$student: invalid grade ($grade)"
            invalid=$((invalid + 1))
            continue ;;
    esac
    if [ "$grade" -ge 9 ]; then        # -ge and not >: we compare numbers
        word="Excellent"
    elif [ "$grade" -ge 7 ]; then
        word="Very good"
    elif [ "$grade" -ge 5 ]; then
        word="Pass"
    else
        word="Fail"
    fi
    echo "$student: $grade $word"
    sum=$((sum + grade))
    n=$((n + 1))
    if [ "$grade" -ge 5 ]; then
        passed=$((passed + 1))
    fi
done < "$1"
echo "Valid grades: $n. Passed: $passed. Invalid: $invalid"
echo "Average: $((sum / n)).$(( (sum * 10 / n) % 10 ))"   # bash only does integer maths
[ "$invalid" -eq 0 ]

Why it’s done this way:

• [ $# -eq 1 ] || { …; exit 2; }: the braces group several commands so the || runs both. It’s a one-line if, very common at the start of scripts.
• read -r student grade: with two variables, the first word goes into student and the rest into grade. If a line had three words, grade would get the last two and the case would reject it.
• The case comes before any -ge: if NP reached [ "$grade" -ge 9 ], you’d get integer expression expected (you saw it with seven in the theory). Validate first, compare afterwards. [0-9]|10) matches a single digit or 10; 11, -3 or NP go to *).
• -ge and not >: we’re comparing numbers. With > inside [ ] we’d be creating files named after grades.
• The average: valid grades 10 + 5 + 10 + 0 + 10 = 35, and 35 / 5 = 7. The integer part is $((35 / 5)) = 7 and the decimal, (35 × 10 / 5) % 10 = 70 % 10 = 0. Hence 7.0. If the sum were 36: 36 / 5 = 7 and 360 / 5 = 72, 72 % 10 = 2, giving 7.2, which is 36 / 5 = 7.2.
• The passes: ana, dani, laura and sergio have 5 or more; pablo doesn’t. 4 passes, and marta doesn’t count because her grade isn’t valid.
• The final exit is the comparison [ "$invalid" -eq 0 ]: there’s one odd grade, so it returns 1. With grades.csv, which doesn’t exist, the second check stops before reading anything and also returns 1.

Level 3 solution

#!/bin/bash
# check_list.sh: grades everyone on the list, submitted or not
# Usage: ./check_list.sh list folder

if [ $# -ne 2 ]; then
    echo "Usage: $0 list folder" >&2
    exit 2
fi
list=$1
folder=$2

grade_of() {                             # grade_of STUDENT: prints their grade (or the reason)
    local source="$folder/$1.c"
    local binary="$folder/$1"
    if [ ! -f "$source" ]; then
        echo "NOT SUBMITTED"
        return 1
    fi
    rm -f "$binary"
    if ! gcc -Wall "$source" -o "$binary" 2> /dev/null; then
        echo "0"
        return 0
    fi
    if [ "$("$binary" < input.txt)" = "$(cat expected.txt)" ]; then
        echo "10"
    else
        echo "5"
    fi
}

on_list=0
submitted=0
correct=0
while read -r student; do
    on_list=$((on_list + 1))
    grade=$(grade_of "$student")         # $( ) captures what the function prints
    if [ $? -eq 0 ]; then                # right after: the $? is grade_of's
        submitted=$((submitted + 1))
    fi
    if [ "$grade" = "10" ]; then
        correct=$((correct + 1))
    fi
    printf "%-8s %s\n" "$student" "$grade"   # %-8s: name left-aligned in 8 columns
done < "$list"
echo "On the list: $on_list. Submitted: $submitted. Correct: $correct"
[ "$submitted" -eq "$on_list" ]

Why it’s done this way:

• The function prints one thing and returns another. echo sends the mark to stdout (collected by $( )) and return 1 flags the “not submitted” case. That way, with a single call we know the mark and whether it counts as submitted.
• if [ $? -eq 0 ] goes right after grade=$(grade_of …): an assignment with $( ) leaves in $? the code of whatever ran inside. If we put anything in between, we’d lose the code (snippet 3).
• return 0 when it doesn’t compile: the student did submit, even if they get a 0. Only NOT SUBMITTED returns 1.
• rm -f before compiling, for the same reason as in the practice: so that an old executable doesn’t pass for anybody.
• printf "%-8s %s\n": %-8s prints the name left-aligned and pads it with spaces up to 8 columns. echo can’t align.
• The counts: 7 students on the list; 5 submit (everyone except marta and ruben, 7 − 2 = 5); correct are the 10s: ana, laura and sergio, 3. Since 5 ≠ 7, the last comparison returns 1.

Try it in your terminal

The trap from snippet 2, live and without a script, so you can see it in three commands. Type it in an empty folder:

$ correct=2
$ [ $correct > 3 ] && echo "greater than 3"
greater than 3
$ ls
3
$ [ "$correct" -gt 3 ] && echo "greater than 3" || echo "not greater than 3"
not greater than 3
$ rm 3
$ [[ $correct > 10 ]] && echo "string: 2 comes after 10"
string: 2 comes after 10

What to watch for:

1. [ $correct > 3 ] && echo "greater than 3" prints greater than 3 even though 2 isn’t greater than 3.
2. The next ls shows the freshly created file 3: proof that the > redirected.
3. With -gt, the same comparison comes out false and the || prints not greater than 3.
4. The last line is the [[ ]] trap: there > compares strings, and "2" comes after "10" in dictionary order (2 comes after 1), so “2 > 10” comes out true.

Cheat sheet: shell scripts in Linux

The essentials in 30 seconds

• First line #!/bin/bash, chmod +x once and ./script.sh.
• Variables with no spaces around = and always in double quotes when used: "$x".
• Parameters: $1… $#, and "$@" to go through them.
• if checks the exit status (0 = fine). [ ] is a command: spaces inside.
• Numbers with -eq -ne -lt -le -gt -ge and $(( )); strings with = and !=.
• Errors to stderr (>&2) and exit with a code that means something.

I want… / Code / Result

I want…	Code	Result
Save a command’s output	n=$(ls submissions | wc -l)	n is 5
Use a variable stuck to text	"${student}_v2.c"	laura_v2.c
A parameter with a default value	folder=${1:-submissions}	submissions if there’s no $1
Remove the extension	basename submissions/ana.c .c or ${f%.c}	ana / submissions/ana
Check the number of parameters	[ $# -eq 1 ] || usage	usage if not exactly 1
Know if a file exists	[ -f "$f" ]	code 0 if it exists
Know if a file has anything in it	[ -s "$f" ]	code 0 if not empty
Search for a text without printing it	grep -q "int main" "$f"	code 0 if found
Compare two files	cmp -s a b	code 0 if identical
Do a calculation	$((sum / n))	integer: 7 / 2 = 3
Read a file line by line	while read -r a b; do … done < f	a = 1st word, b = rest
Drop the option already read	shift	$2 becomes $1
Align columns	printf "%-8s %s\n" "$a" "$b"	name in 8 columns
See what the script runs	bash -x script.sh	each command with +
Check the syntax	bash -n script.sh	code 0 if it’s fine

Templates from the exercises

Exercise	Skeleton	Result with our data
Level 1	for f in "$@"; do [ -f "$f" ] || continue; …; done	5 submissions with main, code 0
Level 2	while read -r a n; do case "$n" in [0-9]|10) ;; *) continue ;; esac; …; done < "$1"	Average 7.0, 4 passes, code 1
Level 3	grade=$(grade_of "$a"); if [ $? -eq 0 ]; then …; fi	7 on the list, 5 submitted, 3 correct, code 1

Symptom / Likely cause

Symptom	Likely cause
Permission denied when launching ./script.sh	chmod +x is missing
script.sh: command not found	The ./ in front is missing
x: command not found in an assignment	Spaces around the =
[1: command not found	Missing space after [
A file named after a number appears	> or < inside [ ]
integer expression expected	-eq, -gt… with something that isn’t a number
A name with a space gets split in two	Unquoted variable or $* instead of "$@"
if decides the wrong way round	$? checked too late, or an exit 1 when everything went fine
$? is 0 after a pipeline that failed	It’s the code of the pipeline’s last command (PIPESTATUS)
The script just stops	A program or a read is waiting for keyboard input: a < is missing

Before submitting, ask yourself

1. Does the script check its parameters at the start and end with exit 2 and a message to stderr if it’s called wrongly?
2. Are all the variables in double quotes, and do you go through the parameters with "$@"?
3. Do you compare numbers with -eq, -gt… and strings with =?
4. Does the script return 0 only when everything went well? Check it with echo $? after each case.

With this you’ve finished IC2’s shell scripts. If you want to review, go back to the theory or the practice. Next up in IC2’s Linux: vi and network configuration.

Keep goingAll Linux articles

Did this article help you? Here is how you can support SergioLearns →