Redirection and pipes in Linux — theory with the lab log, the mysterious order of 2>&1 and filters that chain together (and a sort that left me without the group list)

It was the week the IC2 assignment was due and I had a file, users.txt, with the names of my group, unsorted. I thought: “I’ll sort it and save it in the same place, one line and done”. I typed this, hit Enter and went for a coffee:

# The hook: I wanted to sort the group list "in the same file"
cat users.txt               # five names, unsorted
sort users.txt > users.txt  # looks logical: sort and save on top
cat users.txt               # ...and nothing comes out
wc -c users.txt             # 0 bytes: the file has been emptied

When I came back, this is what the terminal showed:

$ cat users.txt
pablo
laura
sergio
ana
dani
$ sort users.txt > users.txt
$ cat users.txt
$ wc -c users.txt
0 users.txt

No error, no warning. The file was simply empty: 0 bytes. The group list was gone and I had to type it again from memory. What happened? The > doesn’t wait for sort to finish: bash empties users.txt before it starts sort, so sort reads a file that is already empty and has nothing to sort. By the end of this article you’ll understand exactly why, and how to do it properly.

Today it’s time for the advanced shell: how to send the output of a command to a file (>, >>), how to read from a file (<), what to do with error messages (2>, 2>&1, /dev/null) and how to chain commands with pipes (|) so that the output of one becomes the input of the next. With that, the usual filters (grep, sort, uniq, wc, head, tail, cut and tee) become a Swiss army knife. If you’re coming from Linux permissions, you already know how to read and change who can touch a file; now we’ll see how to fill them. Afterwards you have the practice with three complete programs.

The lab log we’ll use throughout

Every example in this topic uses the same file: access.log, the log of logins and logouts in lab 1.3 on an ordinary Monday. It has 16 lines like these:

# stdout and stderr: two different outputs that go to the same screen by default
head -3 access.log                   # stdout: the first three lines of the log
ls access.log notfound               # one part goes well (stdout) and another fails (stderr)
ls access.log notfound > output.txt  # > only redirects stdout: the error stays on screen
cat output.txt                       # the file only has what went well

For now, look only at the first command, head -3 access.log (the first three lines). Each line has six columns separated by a single space:

Column	What it is	Example
1	date	2026-10-19
2	time	08:05
3	computer	pc02
4	user	laura
5	event	LOGIN or LOGOUT
6	result	OK or FAIL (wrong password)

This matters because tools like cut work by columns: “give me column 4” will mean “give me the user”. Keep the table handy; we’ll use it all the time.

Three streams: stdin, stdout and stderr

Before redirecting anything, you need to know what gets redirected. Every Linux program is born with three “pipes” already connected, numbered:

Number	Name	What it’s for	Where it goes by default
0	stdin (standard input)	where the program reads from	the keyboard
1	stdout (standard output)	normal results	the screen
2	stderr (standard error)	error messages and warnings	the screen

Those numbers are called file descriptors. Ring a bell? They’re the same ones we used with read and write in system calls in C: write(1, …) wrote to the screen and so did write(2, …), but through the error channel.

The catch is that stdout and stderr end up on the same screen, so you can’t tell them apart at a glance. Look at the full output of the session above:

$ head -3 access.log
2026-10-19 08:02 pc01 sergio LOGIN OK
2026-10-19 08:05 pc02 laura LOGIN FAIL
2026-10-19 08:06 pc02 laura LOGIN OK
$ ls access.log notfound
ls: cannot access 'notfound': No such file or directory
access.log
$ ls access.log notfound > output.txt
ls: cannot access 'notfound': No such file or directory
$ cat output.txt
access.log

Step by step:

• ls access.log notfound asks for two things. The file access.log exists, so ls writes its name on stdout. notfound doesn’t exist, so ls writes a complaint on stderr. Both lines appear on screen, one under the other, as if they were the same thing.
• ls access.log notfound > output.txt sends stdout to the file. But the error still appears on screen: > only touches stream 1.
• cat output.txt confirms it: the file only contains access.log, the part that went well.

Look at the diagram: the program doesn’t know or care where its streams go. The one who connects them is bash, before starting it.

> and >>: writing output to a file

> and >> send stdout to a file. The difference is what they do if the file already exists:

# > creates or empties; >> appends at the end
echo "Report for lab 1.3" > report.txt  # > creates the file (or empties it if it existed)
echo "Monday 19 October" >> report.txt  # >> appends without touching what was there
cat report.txt
echo "Tuesday 20 October" > report.txt  # > again: what was there disappears
cat report.txt
set -o noclobber                        # safety net: > no longer overwrites existing files
echo "Trying to overwrite it" > report.txt
echo "This one gets in" >| report.txt   # >| forces the write even with noclobber on
cat report.txt
set +o noclobber                        # turn it off so it does not confuse the rest of the session

And this is what came out:

$ echo "Report for lab 1.3" > report.txt
$ echo "Monday 19 October" >> report.txt
$ cat report.txt
Report for lab 1.3
Monday 19 October
$ echo "Tuesday 20 October" > report.txt
$ cat report.txt
Tuesday 20 October
$ set -o noclobber
$ echo "Trying to overwrite it" > report.txt
bash: report.txt: cannot overwrite existing file
$ echo "This one gets in" >| report.txt
$ cat report.txt
This one gets in
$ set +o noclobber

Line by line:

1. The first echo … > report.txt creates the file with one line.
2. echo … >> report.txt appends the second line at the end. The cat shows both.
3. The third echo uses > again, and > empties the file before writing: the cat now only shows “Tuesday 20 October”. Monday’s content is gone for good.
4. With set -o noclobber we switch on a safety net: from then on > refuses to overwrite an existing file (cannot overwrite existing file). If you really want to overwrite it, >| forces it.

Operator	If the file doesn’t exist	If the file already exists
>	creates it	empties it and writes from the start
>>	creates it	writes at the end, leaving what was there
>|	creates it	empties it even when noclobber is on

Now we can explain the hook. When you type sort users.txt > users.txt, bash does things in this order:

1. It reads the line and sees > users.txt.
2. It opens users.txt for writing and empties it (that’s what > does).
3. Only then does it start sort, which opens users.txt for reading… and finds it empty.
4. sort sorts zero lines and writes zero lines. Result: 0 bytes.

<: reading input from a file

< is the mirror image of >: it connects a file to stream 0, standard input.

# < connects a file to standard input (stdin, descriptor 0)
wc -l access.log        # wc opens the file itself, so it prints its name
wc -l < access.log      # with < bash opens it: wc only gets data, no name
sort < users.txt        # sort reads stdin if you give it no file
tr a-z A-Z < users.txt  # tr ONLY reads stdin: it takes no file names

The output:

$ wc -l access.log
16 access.log
$ wc -l < access.log
16
$ sort < users.txt
ana
dani
laura
pablo
sergio
$ tr a-z A-Z < users.txt
PABLO
LAURA
SERGIO
ANA
DANI

Why does wc -l access.log print 16 access.log but wc -l < access.log only 16? Because they’re two different situations:

• In the first, wc opens the file itself: it knows its name, so it puts it in the output.
• In the second, bash opens it and connects it to stdin. wc only sees a stream of data arriving on its input; it has no idea it came from a file, so it can’t print any name.

This looks trivial, but it’s very handy when you want just the number to put in a report (we’ll do it in the practice). 16 lines, by the way, are the 16 entries in the log.

sort < users.txt sorts just like sort users.txt: almost every filter accepts both forms. But tr (which swaps some characters for others; here a-z for A-Z, lower case to upper case) only reads stdin: if you give it a file name, it complains. That’s why you need <.

2>, 2>&1 and /dev/null: what to do with errors

If > is the same as 1> (the 1 can be left out), to redirect stderr you just write its number: 2>. And to send both streams to the same place there’s 2>&1, which is exactly the part that confuses people most. Here’s the full session:

# 2>, 2>&1, &> and /dev/null
ls access.log notfound 2> errors.txt           # 2> redirects only stderr (descriptor 2)
cat errors.txt
ls access.log notfound > all.txt 2>&1          # 1 to the file and THEN 2 to wherever 1 points
cat all.txt
ls access.log notfound 2>&1 > only-one.txt     # reversed: 2 copies the screen BEFORE 1 moves
cat only-one.txt
ls access.log notfound &> all2.txt             # bash shortcut for > all2.txt 2>&1
cat all2.txt
ls notfound 2> /dev/null                       # /dev/null swallows whatever you send it
echo $?                                        # the error still happened: exit code 2
wc -c /dev/null                                # and it is always empty

And its real output:

$ ls access.log notfound 2> errors.txt
access.log
$ cat errors.txt
ls: cannot access 'notfound': No such file or directory
$ ls access.log notfound > all.txt 2>&1
$ cat all.txt
ls: cannot access 'notfound': No such file or directory
access.log
$ ls access.log notfound 2>&1 > only-one.txt
ls: cannot access 'notfound': No such file or directory
$ cat only-one.txt
access.log
$ ls access.log notfound &> all2.txt
$ cat all2.txt
ls: cannot access 'notfound': No such file or directory
access.log
$ ls notfound 2> /dev/null
$ echo $?
2
$ wc -c /dev/null
0 /dev/null

Let’s take it in four blocks.

2> errors.txt. Only stderr is redirected. access.log (stdout) stays on screen and the error ends up inside errors.txt. It’s the exact opposite of what happened with >.

> all.txt 2>&1. Both streams go to all.txt; nothing appears on screen. Notice that inside the file the error comes first: ls complains about the files it can’t find before listing the ones it can.

2>&1 > only-one.txt. It’s the same thing in a different order, and the error appears on screen! This is the hardest part, so let’s go slowly. The key is to read 2>&1 as “make 2 point to wherever 1 points right now”, not as “2 follows 1 forever”. And bash processes redirections from left to right:

As a trace table, step by step:

Command	Step	Stream 1 points to	Stream 2 points to
> all.txt 2>&1	at the start	screen	screen
	> all.txt	all.txt	screen
	2>&1 (copies current 1)	all.txt	all.txt
2>&1 > only-one.txt	at the start	screen	screen
	2>&1 (copies current 1)	screen	screen (nothing changes)
	> only-one.txt	only-one.txt	screen

In the second case, when 2>&1 runs, stream 1 still points to the screen, so 2 “copies” the screen. Then 1 moves to the file, but 2 had already stored its destination and doesn’t notice. That’s why only-one.txt only contains access.log.

&> all2.txt. It’s a bash shortcut that means exactly > all2.txt 2>&1. Shorter, and you can’t get the order wrong. Careful: it’s bash-specific; a strict sh doesn’t have it, so in scripts that must run anywhere you’ll see the long form.

/dev/null. It’s a special file that swallows everything you write to it and, if you read it, it’s always empty (wc -c /dev/null gives 0). ls notfound 2> /dev/null shows nothing… but the error still happened: echo $? shows the exit code of the last command, and it’s 2 (for ls, 0 = all good, 2 = serious trouble, like a file that doesn’t exist). /dev/null silences the message; it doesn’t fix the failure.

Pipes: the output of one command is the input of the next

Up to now we connected programs to files. A pipe (|, the vertical bar) connects the stdout of one command to the stdin of the next, with no file in between. It’s the most powerful idea in the terminal: lots of small tools that each do one thing, chained together.

Let’s answer a question: who logs into the lab the most? We’ll build it step by step, adding one command each time:

# A pipeline built step by step: who logs into the lab the most
cut -d' ' -f4 access.log                                # step 1: only column 4 (user)
cut -d' ' -f4 access.log | sort                         # step 2: sorted, equal names end up together
cut -d' ' -f4 access.log | sort | uniq -c               # step 3: uniq -c counts each group of equal lines
cut -d' ' -f4 access.log | sort | uniq -c | sort -nr    # step 4: from most to fewest logins
cut -d' ' -f4 access.log | uniq -c                      # without sort: uniq only merges ADJACENT duplicates

The output (long, but worth reading in full):

$ cut -d' ' -f4 access.log
sergio
laura
laura
pablo
ana
ana
ana
sergio
dani
dani
laura
laura
laura
pablo
sergio
ana
$ cut -d' ' -f4 access.log | sort
ana
ana
ana
ana
dani
dani
laura
laura
laura
laura
laura
pablo
pablo
sergio
sergio
sergio
$ cut -d' ' -f4 access.log | sort | uniq -c
      4 ana
      2 dani
      5 laura
      2 pablo
      3 sergio
$ cut -d' ' -f4 access.log | sort | uniq -c | sort -nr
      5 laura
      4 ana
      3 sergio
      2 pablo
      2 dani
$ cut -d' ' -f4 access.log | uniq -c
      1 sergio
      2 laura
      1 pablo
      3 ana
      1 sergio
      2 dani
      3 laura
      1 pablo
      1 sergio
      1 ana

• Step 1: cut -d' ' -f4 splits each line at the spaces (-d' ', the delimiter) and keeps field 4, the user. 16 lines, one per entry.
• Step 2: sort puts them in alphabetical order. What matters isn’t the order itself but that equal names end up together: the four ana in a row, the two dani…
• Step 3: uniq -c walks through the lines and, every time a line repeats the previous one, adds 1. With -c it puts in front how many there were in each group: 4 + 2 + 5 + 2 + 3 = 16, the same 16 lines as before.
• Step 4: sort -nr sorts those counts as numbers (-n) and in reverse (-r), from most to fewest. Laura wins with 5 entries.

And the last experiment, without sort? uniq -c gives 10 groups and sergio appears three times with a 1. It isn’t a bug in uniq: uniq only compares each line with the previous one. If two equal lines aren’t next to each other, for uniq they’re different groups. That’s why you’ll almost always see it after a sort: sort | uniq -c.

What travels through the pipe (and what doesn’t)

A pipe connects stdout. And stderr? It still goes to the screen:

# Only stdout travels through the pipe
ls access.log notfound | wc -l               # the error does not get in: it shows on screen and wc counts 1 line
ls access.log notfound 2>&1 | wc -l          # 2>&1 before |: the error travels too, 2 lines
ls access.log notfound 2> /dev/null | wc -l  # the error to the bin; wc still counts 1

The output:

$ ls access.log notfound | wc -l
ls: cannot access 'notfound': No such file or directory
1
$ ls access.log notfound 2>&1 | wc -l
2
$ ls access.log notfound 2> /dev/null | wc -l
1

• In the first command, the error appears on screen and wc -l counts 1 line: the only one it received was access.log. The error message never entered the pipe.
• With 2>&1 before |, stderr joins stdout and both travel: wc -l counts 2. (The order causes no trouble here: bash connects the pipe to stream 1 before processing 2>&1, so 2 copies “the pipe”.)
• With 2> /dev/null, the error goes in the bin and wc still counts 1, but the screen stays clean.

The filters you’ll use most

A filter is a command that reads text (from a file or from stdin), transforms it and writes it to stdout. That’s why they fit so well in pipes.

grep, wc, head, tail and cut

# grep, wc, head, tail and cut on the lab log
grep FAIL access.log                    # the lines that contain FAIL
grep -c FAIL access.log                 # -c: only how many there are
grep -v LOGIN access.log                # -v inverts: the ones that do NOT contain LOGIN
grep -ic fail access.log                # -i ignores upper and lower case
wc access.log                           # lines, words and bytes
tail -n 3 access.log                    # the last three lines
cut -d' ' -f2,4 access.log | head -4    # columns 2 and 4 (time and user) of the first 4
cut -c12-16 access.log | head -3        # by position: characters 12 to 16 are the time

The output:

$ grep FAIL access.log
2026-10-19 08:05 pc02 laura LOGIN FAIL
2026-10-19 08:30 pc04 ana LOGIN FAIL
2026-10-19 08:31 pc04 ana LOGIN FAIL
2026-10-19 10:02 pc01 dani LOGIN FAIL
2026-10-19 10:20 pc02 laura LOGIN FAIL
$ grep -c FAIL access.log
5
$ grep -v LOGIN access.log
2026-10-19 09:58 pc01 sergio LOGOUT OK
2026-10-19 10:15 pc02 laura LOGOUT OK
2026-10-19 11:40 pc03 pablo LOGOUT OK
2026-10-19 12:00 pc04 ana LOGOUT OK
$ grep -ic fail access.log
5
$ wc access.log
 16  96 599 access.log
$ tail -n 3 access.log
2026-10-19 11:40 pc03 pablo LOGOUT OK
2026-10-19 11:45 pc03 sergio LOGIN OK
2026-10-19 12:00 pc04 ana LOGOUT OK
$ cut -d' ' -f2,4 access.log | head -4
08:02 sergio
08:05 laura
08:06 laura
08:11 pablo
$ cut -c12-16 access.log | head -3
08:02
08:05
08:06

• grep FAIL lets through the lines that contain FAIL: the five failed attempts. With -c it only says how many (5), and -v inverts the filter: the 4 lines that don’t contain LOGIN are the LOGOUT ones. With -i it ignores case: fail also finds FAIL.
• wc with no options gives lines, words and bytes: 16 96 599. It adds up: 16 lines × 6 words = 96. And the bytes: each line is between 35 and 39 characters plus the newline, and together they add up to 599 (the first one, 2026-10-19 08:02 pc01 sergio LOGIN OK, has 37 characters + 1 newline = 38).
• tail -n 3 gives the last 3 lines (and head -n 3, or head -3, the first 3).
• cut -d' ' -f2,4 extracts several fields: the time and the user. cut -c12-16 cuts by character position: in every line, characters 12 to 16 are the time (the date takes 1 to 10 and 11 is a space).

I want…	Command
lines that contain some text	grep text f
how many lines contain it	grep -c text f
the ones that DON’T contain it	grep -v text f
count lines / words / bytes	wc -l / wc -w / wc -c
the first / last N	head -n N / tail -n N
from line N to the end	tail -n +N
a column separated by a character	cut -d'X' -fN

sort, uniq and tee

# sort, uniq and tee
sort -r users.txt                                        # reverse alphabetical order
printf '10\n9\n100\n' | sort                             # sort compares text: "100" comes before "9"
printf '10\n9\n100\n' | sort -n                          # -n compares numbers
cut -d' ' -f3 access.log | sort -u                       # -u: no duplicates (like sort | uniq)
grep FAIL access.log | cut -d' ' -f4 | sort | uniq -d    # -d: only the ones that appear more than once
grep FAIL access.log | tee fails.txt | wc -l             # tee saves a copy and lets the data go on
cat fails.txt

The output:

$ sort -r users.txt
sergio
pablo
laura
dani
ana
$ printf '10\n9\n100\n' | sort
10
100
9
$ printf '10\n9\n100\n' | sort -n
9
10
100
$ cut -d' ' -f3 access.log | sort -u
pc01
pc02
pc03
pc04
$ grep FAIL access.log | cut -d' ' -f4 | sort | uniq -d
ana
laura
$ grep FAIL access.log | tee fails.txt | wc -l
5
$ cat fails.txt
2026-10-19 08:05 pc02 laura LOGIN FAIL
2026-10-19 08:30 pc04 ana LOGIN FAIL
2026-10-19 08:31 pc04 ana LOGIN FAIL
2026-10-19 10:02 pc01 dani LOGIN FAIL
2026-10-19 10:20 pc02 laura LOGIN FAIL

• sort -r reverses the order.
• The printf experiment is the classic one: sort with no options compares text, character by character. “100” starts with “1”, which comes before “9”, so it goes first. With -n it compares numbers and you get the order you expected: 9, 10, 100.
• sort -u sorts and removes duplicates in one go: the 4 computers.
• uniq -d shows only the lines that repeat: ana and laura failed more than once (dani only once).
• tee fails.txt is a plumber’s “T”: it writes whatever reaches it to the file and also to stdout, so the pipeline carries on. wc -l counts 5 and, at the same time, fails.txt keeps the 5 lines.

Bonus: here-documents (<<)

Sometimes you want to give a command several lines of text without creating a file first. That’s what a here-document is for: << EOF tells bash “whatever I write next, up to a line that says just EOF, is the standard input of this command”. The word EOF is a convention; it can be anything.

# Here-documents: the input text is written inside the command itself
cat << EOF > note.txt
Lab check done by $USER
Computers: pc01 to pc04
EOF
cat note.txt
cat << 'EOF'
With quotes nothing is replaced: $USER
EOF
wc -l << EOF
one
two
three
EOF

This is what you see in the terminal (the > at the start of the lines are bash’s “keep typing” prompt, not redirections):

$ cat << EOF > note.txt
> Lab check done by $USER
> Computers: pc01 to pc04
> EOF
$ cat note.txt
Lab check done by sergio
Computers: pc01 to pc04
$ cat << 'EOF'
> With quotes nothing is replaced: $USER
> EOF
With quotes nothing is replaced: $USER
$ wc -l << EOF
> one
> two
> three
> EOF
3

• The first one combines << with >: it creates note.txt with two lines, and $USER has been replaced by sergio.
• In the second, with 'EOF' in quotes, bash replaces nothing: $USER comes out as is. Useful when the text contains $ signs you don’t want interpreted.
• The third feeds wc -l three lines: 3.

You’ll use it a lot in scripts, the next topic, to write report headers or configuration files.

When do I use what?

Situation	Use	Why
Save a result, starting from scratch	command > f	empties and writes
Keep accumulating results	command >> f	appends at the end
Have a program read a file without seeing its name	command < f	bash opens it
Separate the errors	command 2> errors.txt	only stream 2
Save everything in one file	command > f 2>&1 (or &> f)	1 to the file, then 2 to where 1 is
Ignore errors you don’t care about	command 2> /dev/null	it swallows them
Process the output with another command	command | filter	no file in between
Make errors enter the pipe too	command 2>&1 | filter	stderr joins stdout
Save and keep processing	… | tee f | …	copies and lets through
Count repetitions	sort | uniq -c | sort -nr	groups, counts and sorts

Common mistakes

Try it in your terminal

Python Tutor doesn’t work for the terminal, but the terminal is even better: it’s right in front of you. Open one (on Linux, in WSL on Windows or in the course virtual machine), go into a test folder and type these commands one by one. This is the transcript of what you should see (error messages will be in your language):

$ printf 'ana\nlaura\nana\n' > test.txt
$ uniq -c test.txt
      1 ana
      1 laura
      1 ana
$ sort test.txt | uniq -c
      2 ana
      1 laura
$ ls test.txt notfound > out.txt 2>&1
$ cat out.txt
ls: cannot access 'notfound': No such file or directory
test.txt
$ ls test.txt notfound 2>&1 > out.txt
ls: cannot access 'notfound': No such file or directory
$ cat out.txt
test.txt

What to watch for:

1. printf 'ana\nlaura\nana\n' writes three lines (\n is the newline) and > saves them in test.txt.
2. uniq -c test.txt counts three groups of 1: the two ana lines aren’t next to each other. With sort before it you get 2 ana and 1 laura.
3. With > out.txt 2>&1 the screen stays silent and out.txt has two lines: the error and test.txt.
4. With 2>&1 > out.txt the error appears on screen and the final cat only shows test.txt. Change one thing (the order) and the result changes.
5. Bonus: repeat the last command with &> out.txt and check that it behaves like the third one.

Summary

Concept	In one sentence
Streams	0 stdin (keyboard), 1 stdout and 2 stderr (screen); bash connects them before starting the command
> / >>	stdout to a file: > empties it first, >> appends at the end
<	a file as stdin; the command doesn’t see its name
2>	only the errors to a file
2>&1	2 points to wherever 1 points at that moment: > f 2>&1, not the other way round
/dev/null	swallows whatever you send it; the error still happens ($?)
|	stdout of one command → stdin of the next; stderr doesn’t travel
Filters	grep, wc, head, tail, cut, sort, uniq, tee
<< EOF	several lines of input written in the command itself

Three ideas to take away

1. Bash sets up the redirections before starting the command. That’s why sort f > f empties the file and why the order of 2>&1 matters.
2. stdout and stderr are two different streams even though they show up on the same screen. > and | only touch 1; errors need their own 2>.
3. Pipelines are built step by step. Add a command, look at the output, add the next one. And if you’re counting, sort | uniq -c.

You now know how to connect commands to each other and to files. The next step is to save them in a file and run them all at once: shell scripts, the next Linux topic. Before that, work through the redirection and pipes practice: three complete programs on the same lab log.

Next step: practiceRedirection and pipes in Linux — practice with a report that grows with >>, a ranking saved with tee and warnings split from the good lines (and the report that ate itself)

Did this article help you? Here is how you can support SergioLearns →