Redirection, Pipes, and Combining Commands

Three streams every command has

Every command that runs is connected to three data streams.

• stdin (standard input): where the command reads its input. By default this is your keyboard.
• stdout (standard output): where the command writes its normal results. By default this is your screen.
• stderr (standard error): where the command writes error messages. By default this is also your screen, so you do not notice it is a separate stream.

Redirection means changing where these streams go: output into a file instead of the screen, or input from a file instead of the keyboard.

Send output to a file: > and >>

echo "Hello" > hello.txt
echo "World" >> hello.txt
ls -l > listing.txt
date >> log.txt

• > sends the output into a file. If the file exists, its old contents are replaced.
• >> also sends output to a file but appends to the end and keeps the old contents.

Mixing these up is a classic way to lose data. Use >> when you want to keep what is already there.

Send errors elsewhere: 2>

Streams have numbers: stdin is 0, stdout is 1 and stderr is 2. Put a number before > to redirect that stream.

ls /nonexistent 2> errors.txt
ls /home /nonexistent > out.txt 2> err.txt
command > all.txt 2>&1
command 2> /dev/null

• 2> errors.txt sends only error messages to a file.
• The second line sends normal output to one file and errors to another.
• > all.txt 2>&1 sends stdout to all.txt, and 2>&1 sends stderr to the same place as stdout.
• /dev/null is a special file that discards everything written to it. Redirecting a stream there hides it.

Read input from a file: <

wc -l < fruits.txt
sort < unsorted.txt

< makes a command read its input from a file instead of the keyboard. Most commands also accept a file name directly, so you will use < less often than >.

Pipes: connect commands

A pipe | takes the output of the command on its left and uses it as the input of the command on its right. No file is needed in between.

ls | wc -l
ls -l | head -n 5
history | tail -n 5

• ls | wc -l lists the current folder and counts the lines, so it tells you how many items are in it.
• ls -l | head -n 5 shows only the first five lines of the long listing.
• history | tail -n 5 shows your last five commands. history prints every command you have run.

Each command does one small job. The pipe lets you join them for bigger tasks. This is the core idea of the Unix tools: small programs that do one thing well and work together.

Sorting in a pipe

sort puts lines in order. Useful options:

• -n sorts numbers by value instead of as text.
• -k5 sorts by the 5th column, where columns are separated by spaces.
• -r reverses the order.

In ls -l output, the 5th column is the file size. To find the three largest log files:

ls -l *.log | sort -k5 -n | tail -n 3

Read it from left to right: list the logs, sort by size from smallest to largest, keep the last three lines, which are the largest.

Running commands in sequence

You can put several commands on one line.

mkdir demo ; cd demo ; touch a.txt
mkdir demo && cd demo
cd /nope || echo "Could not cd"

• ; runs the next command always.
• && runs the next command only if the first one succeeded.
• || runs the next command only if the first one failed.

&& is the one you will use most. It stops you from carrying on after something went wrong. Every command ends with an exit status: 0 means success and anything else means failure. The shell uses it to decide what && and || do.

tee: save and show at the same time

ls -l | tee listing.txt
ls -l | tee -a listing.txt

tee copies its input to the screen and to a file at once. -a appends instead of replacing.