Pipes & Redirection

Every command you run on Linux reads input and produces output. Pipes and redirection let you control where that input comes from and where the output goes. This is one of the most powerful ideas in Linux: instead of one giant program that does everything, you connect small, focused commands together like building blocks. Once this clicks, you will write one-liners that pull a useful answer out of a 2 GB log file in seconds.

The three standard streams

When a program runs, Linux gives it three “streams” (channels for data). Think of them as three pipes attached to every command.

Stream	Name	Number (file descriptor)	What it carries
stdin	standard input	0	Data going into the command (usually your keyboard)
stdout	standard output	1	Normal results going out of the command
stderr	standard error	2	Error messages and warnings

A “file descriptor” is just a number Linux uses to refer to each stream. You will use these numbers (especially 2 for stderr) when redirecting.

The key insight: stdout and stderr are separate. A command can print a normal result and an error at the same time, and Linux keeps them on different channels. That is why a command can fail loudly on screen while its > file ends up empty — the error went to stderr, not stdout.

Redirecting output to a file

Redirection sends a stream somewhere other than the screen. The most common use is saving output to a file.

> writes stdout to a file, replacing the file’s contents.
>> appends stdout to the end of a file (creating it if needed).

# Save the list of files to a file (overwrites every time)
ls -lh /etc > filelist.txt

# Append the current date to a log (keeps old content)
date >> /var/log/myapp/heartbeat.log

When to use which: Use > for snapshots you want fresh each run (a report, an export). Use >> for logs and anything you want to keep growing.

Gotcha: > truncates the target file before the command even runs. Typing sort file.txt > file.txt will empty file.txt first, then sort nothing. Always write to a different filename, then mv it back if needed.

Redirecting stderr

Errors travel on stream 2, so you redirect them with 2>. This lets you separate “what worked” from “what broke”.

# Search the whole disk; send results to one file, errors to another
find / -name "*.conf" > results.txt 2> errors.txt

Here normal matches land in results.txt, while “Permission denied” messages land in errors.txt instead of cluttering your screen.

To throw errors away entirely, redirect them to /dev/null, a special file that discards everything written to it (think of it as a black hole).

find / -name "*.conf" 2> /dev/null

Combining stdout and stderr with 2>&1

Often you want both streams in the same place — for example, capturing a full build log. The syntax 2>&1 means “send stream 2 (stderr) to wherever stream 1 (stdout) is currently going”.

# Capture normal output AND errors into one log file
sudo apt update > apt.log 2>&1

Order matters. > apt.log 2>&1 works because > apt.log points stdout at the file first, then 2>&1 points stderr to the same place. Writing 2>&1 > apt.log would point stderr at the screen (where stdout was), then move only stdout to the file — a classic mistake.

Modern Bash (the default shell on Ubuntu 22.04 and 24.04 LTS) also offers a shorthand: &> sends both streams at once.

sudo apt update &> apt.log

Pipes: connecting commands together

A pipe (|) takes the stdout of one command and feeds it as the stdin of the next. No temporary files needed — the data flows straight through.

# Count how many users have a login shell
cat /etc/passwd | grep "/bin/bash" | wc -l

Output:

Read it left to right: cat prints the file, grep keeps only lines containing /bin/bash, and wc -l counts those lines. Each stage does one job.

Tip: cat file | grep x is a common habit, but grep x file does the same thing in one process. Save pipes for when you genuinely need to chain commands; do not pipe cat into things out of reflex.

Real, useful pipelines

# Top 5 processes by memory usage
ps aux --sort=-%mem | head -n 6

# Find the 3 biggest files in /var/log
du -ah /var/log | sort -rh | head -n 3

# How many times an IP hit your Nginx site today
grep "203.0.113.7" /var/log/nginx/access.log | wc -l

Output (top processes):

USER  PID %CPU %MEM    VSZ   RSS TTY  STAT START   TIME COMMAND
mysql 812  1.2 18.4 2.1G  742M ?    Ssl 09:01   2:14 /usr/sbin/mysqld
www-d 944  0.4  6.1 410M  246M ?    S   09:02   0:38 php-fpm: pool www

Useful extras: tee and stdin redirection

The tee command splits a stream: it writes to a file and passes the data on to the screen (or the next pipe). This is perfect when you want to watch output live and save it.

# Run an upgrade, see it live, and keep a log
sudo apt upgrade -y 2>&1 | tee /var/log/myapp/upgrade.log

You can also redirect a file into a command’s stdin with <. Many commands accept this, though most also take a filename argument directly.

# Feed a file into a command as stdin
wc -l < /etc/services

Best Practices

Write > output to a new filename, never the same file you are reading from.
Use 2> /dev/null to silence permission noise from find, but only when you are sure you do not need those errors.
Capture full logs with &> file.log (or > file.log 2>&1) so errors are not lost.
Reach for tee when you need output both on screen and saved to disk.
Keep each stage of a pipe focused on one task — small commands are easier to debug than one giant one.
Test a pipeline stage by stage: build it from the left, checking the output before adding the next |.
Quote variables and filenames ("$file") so spaces in names do not break your redirection.