Redirection and pipes in Linux — practice with a report that grows with >>, a ranking saved with tee and warnings split from the good lines (and the report that ate itself)

The first time I tried to build a report from the lab log I did it “by hand”: one command per figure, each with its own > into report.txt. Entries, successful logins, failures… Ten commands. I ran cat report.txt to admire my work and there was only one thing in it: the list of users, which was the last thing I had written. Every > had emptied the file before writing, so each figure had eaten the previous one. The lesson, which you’ll see applied in all three programs: > only once, at the start; >> for everything else. And if something has to go to two places, don’t repeat it: tee.

This is the practice for the redirection and pipes theory. There are three terminal “programs”, medium level, that work on the same access.log from lab 1.3 (16 login and logout entries from a Monday). The data looks like real data on purpose: there are mistyped passwords, people who log in twice, a computer that raises warnings… The first builds a report with > and >>; the second produces rankings with pipes and saves a copy with tee; the third separates the good lines from the warnings with 2>, 2>&1 and /dev/null.

Before you start

Make sure you’re clear on:

• What stdout (1) and stderr (2) are, and that by default both go to the screen.
• The difference between > (empties and writes) and >> (appends at the end).
• That | connects the stdout of one command to the stdin of the next, and that uniq only merges adjacent duplicates.
• How to cut a column with cut -d' ' -fN.

If any of these sounds shaky, review the theory before carrying on.

2026-10-19 08:02 pc01 sergio LOGIN OK
2026-10-19 08:05 pc02 laura LOGIN FAIL
2026-10-19 08:06 pc02 laura LOGIN OK
2026-10-19 08:11 pc03 pablo LOGIN OK
2026-10-19 08:30 pc04 ana LOGIN FAIL
2026-10-19 08:31 pc04 ana LOGIN FAIL
2026-10-19 08:33 pc04 ana LOGIN OK
2026-10-19 09:58 pc01 sergio LOGOUT OK
2026-10-19 10:02 pc01 dani LOGIN FAIL
2026-10-19 10:03 pc01 dani LOGIN OK
2026-10-19 10:15 pc02 laura LOGOUT OK
2026-10-19 10:20 pc02 laura LOGIN FAIL
2026-10-19 10:21 pc02 laura LOGIN OK
2026-10-19 11:40 pc03 pablo LOGOUT OK
2026-10-19 11:45 pc03 sergio LOGIN OK
2026-10-19 12:00 pc04 ana LOGOUT OK

The output you’ll see is real: it was run once as the user sergio with the terminal in English (LANG=C). If yours is in another language, only the error messages will change.

Program 1: Monday’s report with > and >>

We want a file report.txt with a summary of the day: how many entries there are, how many logins went well, how many attempts failed, how many logouts, and the list of distinct users. All with commands we already know, but each one writes its piece into the same file. What makes it tricky: the numbers have to appear next to their label (Records: 16, not Records: with the 16 on another line) and nothing must get overwritten.

The idea

A report is a file that grows. The plan:

1. The first line, the title, with >: that way, every time you run it you start with a clean report and older ones don’t pile up.
2. Each figure takes two commands: an echo -n "Label: " >> (the -n avoids the newline) and the command that computes the number, also with >>. Since echo didn’t end the line, the number lands right next to it.
3. To stop wc -l printing the file name, < access.log. grep -c already gives the bare number.
4. The list of users with the pipeline cut | sort -u, redirected at the end with >>.

The code

# Program 1: Monday's report, line by line with > and >>
echo "== Lab 1.3: Monday 19 October ==" > report.txt  # > only the first time: a clean report
echo -n "Records: " >> report.txt         # -n: no newline, so the number goes right after it
wc -l < access.log >> report.txt          # < so that wc does not print the file name
echo -n "Successful logins: " >> report.txt
grep -c "LOGIN OK" access.log >> report.txt  # quotes: the pattern has a space
echo -n "Failed attempts: " >> report.txt
grep -c "LOGIN FAIL" access.log >> report.txt
echo -n "Logouts: " >> report.txt
grep -c LOGOUT access.log >> report.txt
echo "Distinct users:" >> report.txt
cut -d' ' -f4 access.log | sort -u >> report.txt  # the redirection goes at the end of the pipeline
cat report.txt
wc -l report.txt

Output

$ echo "== Lab 1.3: Monday 19 October ==" > report.txt
$ echo -n "Records: " >> report.txt
$ wc -l < access.log >> report.txt
$ echo -n "Successful logins: " >> report.txt
$ grep -c "LOGIN OK" access.log >> report.txt
$ echo -n "Failed attempts: " >> report.txt
$ grep -c "LOGIN FAIL" access.log >> report.txt
$ echo -n "Logouts: " >> report.txt
$ grep -c LOGOUT access.log >> report.txt
$ echo "Distinct users:" >> report.txt
$ cut -d' ' -f4 access.log | sort -u >> report.txt
$ cat report.txt
== Lab 1.3: Monday 19 October ==
Records: 16
Successful logins: 7
Failed attempts: 5
Logouts: 4
Distinct users:
ana
dani
laura
pablo
sergio
$ wc -l report.txt
11 report.txt

Let’s check the numbers by hand. The log has 16 lines, and each one is a successful login (LOGIN OK), a failure (LOGIN FAIL) or a logout (LOGOUT): 7 + 5 + 4 = 16. There are five distinct users, and they come out in alphabetical order because sort -u sorts and removes duplicates at the same time. And wc -l report.txt says 11: 1 title + 4 lines of figures + 1 label + 5 users = 11.

The diagram shows which command wrote each line of the file:

Look at lines 2 to 5: each one is written by two commands. The echo -n leaves the label without a newline and the grep -c (or wc -l) finishes the line with its number and its newline.

Let’s zoom into the most curious line, the one with the entries:

wc -l < access.log >> report.txt           # < so that wc does not print the file name

There are two redirections in the same command here: < connects access.log to the input of wc and >> connects its output to the report. Bash sets up both before starting wc, which only sees “data coming in” and “somewhere to write”. Without the < (with wc -l access.log), the line would read Records: 16 access.log.

Step	Command	Content of informe.txt afterwards
1	echo "== Lab 1.3: Monday 19 October ==" > report.txt	1 line (the title)
2	echo -n "Records: " >>	title + Records: (half a line)
3	wc -l < access.log >>	title + Records: 16
4-9	three echo -n + grep -c pairs	5 lines
10	echo "Distinct users:" >>	6 lines
11	cut … | sort -u >>	11 lines

Questions you might ask

Why does the >> go at the end of the pipeline and not after cut? Because a redirection applies to the command it’s written in. cut … >> report.txt | sort -u would send the output of cut to the file (unsorted and with duplicates) and sort would get nothing. In a pipeline, what goes to the file is what comes out of the last command.

Why is "LOGIN OK" in quotes and LOGOUT isn’t? Because LOGIN OK has a space. Without quotes, grep -c LOGIN OK access.log would understand that you’re looking for LOGIN in two files called OK and access.log, and it would complain that OK doesn’t exist.

And if I searched only for LOGIN? It would count 12: the 7 successful logins and the 5 failures, because both contain LOGIN. You have to search for the full pattern.

Why not grep -c OK for the successful ones? Because the LOGOUT OK lines also end in OK: you’d get 11 (7 + 4). Choosing the right pattern is half the work with grep.

What happens if I run it twice? You get the same report, thanks to the first command using >. If that one were >> too, the second run would add another full report under the first.

Try it yourself

• Add a line Computers used: with the number of distinct computers (hint: cut -d' ' -f3 access.log | sort -u | wc -l; remember the -n on echo).
• Change the program so that Tuesday’s report is appended below Monday’s in a week.txt, without losing Monday’s.

Program 2: user ranking, failures and peak hour

Now we want statistics: who uses the lab the most (and save the full ranking in ranking.txt even though we only see the top 3 on screen), who gets their password wrong most often, and at what hour there’s most activity. What makes it tricky: there’s no command that “counts per user”; you have to build it out of pieces.

The idea

The pattern we’ll repeat three times is extract → sort → count → sort by the number:

1. cut extracts the column we care about (user or time).
2. sort groups equal values.
3. uniq -c counts each group.
4. sort -nr orders by that number, highest first.

And to save the whole ranking while only showing the beginning, tee goes before head: that way the file gets everything and head only trims what goes to the screen.

The code

# Program 2: user ranking, failures per person and peak hour
cut -d' ' -f4 access.log | sort | uniq -c | sort -nr | tee ranking.txt | head -3
wc -l ranking.txt                                             # tee saved the WHOLE ranking, not just the top 3
grep "LOGIN FAIL" access.log | cut -d' ' -f4 | sort | uniq -c | sort -nr
cut -d' ' -f2 access.log | cut -d: -f1 | sort | uniq -c | sort -nr | head -1

Output

$ cut -d' ' -f4 access.log | sort | uniq -c | sort -nr | tee ranking.txt | head -3
      5 laura
      4 ana
      3 sergio
$ wc -l ranking.txt
5 ranking.txt
$ grep "LOGIN FAIL" access.log | cut -d' ' -f4 | sort | uniq -c | sort -nr
      2 laura
      2 ana
      1 dani
$ cut -d' ' -f2 access.log | cut -d: -f1 | sort | uniq -c | sort -nr | head -1
      7 08

Hand check. In the ranking, laura has 5 entries, ana 4, sergio 3, and pablo and dani 2 each: 5 + 4 + 3 + 2 + 2 = 16. Only three appear on screen because head -3 trims them, but wc -l ranking.txt says 5: the file has the full ranking. For failures, 2 + 2 + 1 = 5, the same 5 failed attempts as in program 1. And the peak hour is 8 o’clock: there are 7 entries between 08:02 and 08:33.

The diagram shows why the file and the screen don’t hold the same thing:

Let’s zoom into the last pipeline, the peak-hour one, which has two cut commands in a row:

cut -d' ' -f2 access.log | cut -d: -f1 | sort | uniq -c | sort -nr | head -1

The first cut -d' ' -f2 keeps the time (08:02). The second changes the delimiter, -d:, and keeps what comes before the colon (08). That way we group by hour, not by minute. Here’s the trace, stage by stage:

Stage	What comes out (summarised)	Lines
cut -d' ' -f2	08:02, 08:05, 08:06, … 12:00	16
cut -d: -f1	08, 08, 08, … 12	16
sort | uniq -c	7 08, 1 09, 5 10, 2 11, 1 12	5
sort -nr	7 08, 5 10, 2 11, …	5
head -1	7 08	1

7 + 1 + 5 + 2 + 1 = 16.

Questions you might ask

Why does laura come before ana in the failures if both have 2? Because when two numbers tie, sort breaks the tie by comparing the rest of the line, and with -r it does that in reverse too: “laura” comes after “ana” in the alphabet, so with -r it comes first. If you want ties in normal alphabetical order, sort -k1,1nr -k2 sorts by the first column as a descending number and, on a tie, by the second.

Why sort -n and not just sort? Because uniq -c leaves the numbers padded with spaces and, although they all have one digit here, the day there’s a 10 a text sort would put it before the 9. With numbers, always -n.

What if I put tee after head? Then ranking.txt would only have the first three lines. tee saves what reaches it, and after head only three lines arrive.

Why is there a sort before uniq -c for the failures if they’re “nearly” together already? Because they aren’t: laura failed at 08:05 and at 10:20, with other failures in between. Without sort, laura would appear twice with a 1 (you’ll see it in the exercises).

Are head -1 and head -n 1 the same? Yes. -n 1 is the “official” form and -1 the age-old shortcut.

Try it yourself

• Build the ranking of computers (column 3) and save it in pcs.txt, showing only the most used one.
• Work out how many entries there were in the morning (before 10): hint, cut the time and grep -c the ones that start with 08 or 09 (grep -c '^0[89]').

Program 3: separating the good lines from the warnings

The lab technician has a small script, check.sh, that checks the four computers. It writes one line per computer: if all is well, on stdout; if there’s a problem, a WARNING on stderr. We want to save the good lines in one file and the warnings in another, count the warnings and, finally, have everything together and in order. What makes it tricky: on screen the five lines look the same.

The idea

First we need to look inside the script, to know what comes out on each stream:

#!/bin/bash
# check.sh: checks the four computers in the lab (simulated with echo)
echo "pc01: correct"
echo "pc02: correct"
echo "WARNING pc03: disk almost full" >&2   # >&2: this message goes out on stderr
echo "pc04: correct"
echo "WARNING pc04: no network" >&2

The trick is >&2: it’s the redirection “send this output to stream 2”, that is, echo writes to stderr instead of stdout. That’s how programs (and your scripts) write their errors. With that clear, the plan is:

1. Run it as is to see the mix.
2. Split it with > (stream 1) and 2> (stream 2), each to its own file.
3. To count the warnings with grep -c, they have to enter the pipe: 2>&1 |.
4. To keep only the good lines, warnings to /dev/null.
5. To have everything together in one file, > all.txt 2>&1, in that order.

The code

# Program 3: separating the correct lines from the warnings of check.sh
bash check.sh                                # stdout and stderr get mixed on screen
bash check.sh > correct.txt                  # the warnings (stderr) still show up
bash check.sh > correct.txt 2> warnings.txt  # each stream to its own file: clean screen
cat correct.txt
cat warnings.txt
bash check.sh 2>&1 | grep -c WARNING         # to filter warnings, they must enter the pipe
bash check.sh 2> /dev/null | wc -l           # only the correct ones: warnings to the bin
bash check.sh > all.txt 2>&1                 # everything together, in order, in one file
cat all.txt

Output

$ bash check.sh
pc01: correct
pc02: correct
WARNING pc03: disk almost full
pc04: correct
WARNING pc04: no network
$ bash check.sh > correct.txt
WARNING pc03: disk almost full
WARNING pc04: no network
$ bash check.sh > correct.txt 2> warnings.txt
$ cat correct.txt
pc01: correct
pc02: correct
pc04: correct
$ cat warnings.txt
WARNING pc03: disk almost full
WARNING pc04: no network
$ bash check.sh 2>&1 | grep -c WARNING
2
$ bash check.sh 2> /dev/null | wc -l
3
$ bash check.sh > all.txt 2>&1
$ cat all.txt
pc01: correct
pc02: correct
WARNING pc03: disk almost full
pc04: correct
WARNING pc04: no network

Let’s go over it. In the first run, all five lines come out mixed. In the second, > correct.txt takes the three good lines and the two warnings stay on screen, proof that they were on stderr. In the third, the screen stays clean: each stream has its own file. The cat commands confirm it: 3 lines in correct.txt and 2 in warnings.txt (3 + 2 = 5, the five the script writes).

grep -c WARNING gives 2 because the warnings entered the pipe thanks to 2>&1. wc -l with 2> /dev/null gives 3: only the good lines. And all.txt has all five lines in the same order the script wrote them.

Zoom into the command that’s most often confused:

bash check.sh 2>&1 | grep -c WARNING           # to filter warnings, they must enter the pipe

There’s no file here, but order still matters: bash first connects the pipe to stream 1 and then processes 2>&1, which makes 2 point “wherever 1 points”, that is, to the pipe. That’s why both streams reach grep.

Command	1 goes to	2 goes to	On screen
bash check.sh	screen	screen	5 lines
… > correct.txt	correct.txt	screen	2 warnings
… > correct.txt 2> warnings.txt	correct.txt	warnings.txt	nothing
… 2>&1 | grep -c WARNING	pipe	pipe	2
… 2> /dev/null | wc -l	pipe	/dev/null	3
… > all.txt 2>&1	all.txt	all.txt	nothing

Questions you might ask

Why bash check.sh and not ./check.sh? Because the file doesn’t have execute permission (644, as we saw in Linux permissions). With bash check.sh it’s bash that reads and runs it, and only read permission is needed. With chmod u+x check.sh you could use ./check.sh.

Why do the lines in all.txt come out in order and not “the good ones first, then the warnings”? Because both streams point to the same open file and each echo writes at that moment. If you redirected to two files and then joined them with cat, you’d lose the order.

And if I write 2> all.txt > all.txt? Bash opens the file twice, separately, and each stream writes at its own position: the lines overwrite each other. For “both to the same file”, always > all.txt 2>&1 (or &> all.txt).

Does /dev/null delete the script’s warnings? No: the script still writes them, they just go somewhere that throws them away. If the problem is real (a nearly full disk), it’s still there.

What exactly does >&2 do? It’s 1>&2: “stream 1 of this echo points to wherever 2 points”. It’s the same as 2>&1, but the other way round.

Try it yourself

• Save only the warnings in warnings.txt without anything appearing on screen and without creating correct.txt (hint: 1 to /dev/null).
• Append the warnings from each run to a history file warnings-week.txt instead of overwriting it.

Common mistakes in this practice

• > on every line of a report. Each one empties the file: only the last one survives.
• The redirection in the middle of the pipeline. cut … > f | sort sends cut‘s output to f and leaves sort without data.
• grep patterns that are too short. OK also matches the LOGOUT OK lines; search for "LOGIN OK" in quotes.
• uniq -c without sort before it. Counts come out split when equal lines aren’t adjacent.
• tee after head. The file only keeps the trimmed part.
• Counting errors without 2>&1. stderr doesn’t enter the pipe.

Try it in your terminal

You don’t need check.sh to see the two streams: { …; } groups several commands and bash treats the group as a single program, so you can redirect its stdout and its stderr. Type this and compare with the transcript (note the space after { and the ; before }; both are required):

$ { echo "pc01: correct"; echo "WARNING pc03" >&2; } > good.txt 2> bad.txt
$ cat good.txt
pc01: correct
$ cat bad.txt
WARNING pc03
$ { echo "pc01: correct"; echo "WARNING pc03" >&2; } | wc -l
WARNING pc03
1
$ { echo "pc01: correct"; echo "WARNING pc03" >&2; } 2>&1 | wc -l
2

What to watch for:

1. The first command writes nothing on screen: the normal echo goes to good.txt and the one with >&2 goes to bad.txt.
2. The cat commands confirm that each line ended up in its file.
3. In the second-to-last, | wc -l counts 1, and WARNING pc03 appears on screen: it didn’t enter the pipe.
4. In the last one, with 2>&1 before the bar, wc -l counts 2.
5. Replace > good.txt 2> bad.txt with &> all.txt and look at what’s inside.

Summary

• A report that grows: > only once, at the start; >> on every other command.
• echo -n plus a command that gives a number (wc -l < f, grep -c) write “label: number” on the same line.
• The counting pattern: cut | sort | uniq -c | sort -nr, and head to keep the first ones.
• tee saves what reaches it and lets the pipeline carry on: put it before the trimming.
• > f 2> g separates the streams; 2>&1 | puts errors in the pipe; 2> /dev/null discards them; > f 2>&1 saves everything in order.
• Scripts write their warnings with >&2 so they can be separated.

Checklist: have you practised it?

• Have you run the three programs and checked the output?
• Can you explain why each line of the report in program 1 was written by the command that wrote it?
• Can you say, without looking, what ranking.txt holds and what appears on screen in program 2?
• Do you know where stream 1 and stream 2 go in each command of program 3?
• Have you tried the “Try it yourself” tasks without looking at the theory?

If you answered yes to almost everything, you’re ready for the redirection and pipes exercises, where I no longer give you the commands ready-made. See you there!

Next step: exercisesRedirection and pipes in Linux — exercises with failed logins, rankings that split apart and a 2>&1 in the wrong place (and a grep that counted zero errors with a screen full of them)

Did this article help you? Here is how you can support SergioLearns →