Linux Mastery

The Human Knowledge Project

Chapter 11 — Pipes & Redirection

Why This Chapter Matters

One of the defining strengths of Linux is that small programs can work together to solve complex problems.

Rather than relying on large, all-in-one applications, Linux encourages simple tools that each perform one task well.

Pipes and redirection allow these tools to exchange information, save results, filter data, and automate repetitive tasks.

Understanding these concepts is one of the biggest steps toward thinking like an experienced Linux user.

Learning Objectives

Upon completing this chapter, you will be able to:

Introduction

Imagine a workshop filled with specialized tools.

One tool cuts wood.

Another drills holes.

A third sands rough edges.

Rather than building one enormous machine that attempts every task, the craftsman combines simple tools to complete the job.

Linux works the same way.

Commands produce information.

Other commands receive that information.

Pipes connect them together.

Redirection controls where the information goes.

Once you understand this flow of information, Linux becomes dramatically more powerful.

1. Streams of Information

Linux treats information as streams flowing between commands.

Every command normally works with three standard streams.

| Stream | Meaning |

|---------|---------|

| stdin | Standard input |

| stdout | Standard output |

| stderr | Standard error |

Understanding these three streams is fundamental to understanding how Linux commands communicate.

THKI Memory Aid


Keyboard
      ↓
stdin
Program
      ↓
stdout
Errors
      ↓
stderr

Whenever you run a command, think about where information comes from and where it goes.

2. Standard Input (stdin)

Standard input is where a command receives information.

Input commonly comes from:

Example:


cat

The command waits for input from the keyboard.

Type several lines of text.

Press:


Ctrl+D

to indicate the end of input.

The text is then displayed because cat echoes its standard input to standard output.

3. Standard Output (stdout)

Standard output is the normal information produced by a command.

Example:


ls

The directory listing appears on the terminal because the terminal is normally connected to stdout.

Most Linux commands send their normal results to this stream.

Later in this chapter we will learn how to redirect this output into files instead of displaying it on the screen.

4. Standard Error (stderr)

Error messages are sent through a separate stream called standard error.

Example:


ls nonexistentfile

Possible output:


ls: cannot access 'nonexistentfile': No such file or directory

Although this appears on the screen, it is not part of the command's normal output.

It is sent through stderr.

Keeping normal output separate from error messages allows Linux to automate tasks much more effectively.

THKI Insight

Separating normal output from error messages is one of the reasons Linux scripting and automation are so powerful.

5. Pipes

The pipe operator is:


|

A pipe sends the standard output of one command directly into the standard input of another command.

Example:


ls | less

Read this command from left to right.

  1. ls creates a directory listing.
  2. The pipe transfers that listing.
  3. less receives the listing and displays it one screen at a time.

Another example:


cat logfile.txt | grep error

Here:

  1. cat displays the file.
  2. The pipe transfers the contents.
  3. grep searches for lines containing the word:
    4. 
error

Commands become building blocks that can be connected together to solve larger problems.

THKI Insight

One of the greatest strengths of Linux is that independent programs can cooperate simply by passing text from one command to another.

6. Multiple Pipes

One pipe is useful.

Multiple pipes allow Linux commands to perform increasingly sophisticated tasks.

Example:


cat logfile.txt | grep error | wc -l

Read the pipeline from left to right.

  1. cat displays the file.
  2. grep keeps only lines containing:
    3. 
error
  3. wc -l counts those matching lines.

Each command performs one simple task.

Together they solve a more complex problem.

Real-World Example


ps aux | grep firefox

Here:

This pipeline is used constantly by Linux administrators.

7. Redirecting Output with >

Normally, command output appears on the screen.

The greater-than symbol redirects that output into a file.

Example:


ls > files.txt

Instead of displaying the directory listing, Linux writes it into:


files.txt

You can display the saved file using:


cat files.txt

Important

The > operator overwrites an existing file.

Example:


echo Hello > notes.txt

If notes.txt already exists, its previous contents are replaced.

THKI Insight

Before pressing Enter, always ask yourself:

"Will this command overwrite an existing file?"

8. Appending Output with >>

Sometimes you want to preserve the existing contents of a file.

The append operator accomplishes this.

Example:


echo First line > notes.txt

Then:


echo Second line >> notes.txt

The resulting file contains:


First line
Second line

Unlike >, the append operator adds new information to the end of the file without replacing what is already there.

Comparison

| Operator | Action |

|----------|--------|

| > | Overwrite file |

| >> | Append to file |

9. Redirecting Errors

Linux keeps normal output and error messages separate.

Because errors use stderr, they can be redirected independently.

Example:


ls nonexistentfile 2> errors.txt

Instead of displaying the error on the screen, Linux writes it into:


errors.txt

Why 2>?

Linux identifies its standard streams with numbers.

| Number | Stream |

|--------|--------|

| 0 | stdin |

| 1 | stdout |

| 2 | stderr |

Therefore:


2>

means:


Redirect standard error.

10. Redirecting Both Output and Errors

Sometimes both streams should be saved.

Example:


command > output.txt 2> errors.txt

Normal output is written to:


output.txt

Errors are written to:


errors.txt

To combine both streams into one file:


command > all_output.txt 2>&1

Both normal output and error messages are collected together.

11. /dev/null

Linux includes a special device called:


/dev/null

Anything sent there disappears permanently.

Example:


ls nonexistentfile 2> /dev/null

The error message is discarded instead of appearing on the screen.

For this reason, /dev/null is often called:

The Linux Black Hole

It is commonly used in:

when unwanted output should be ignored.

12. tee — Display and Save Output

Normally, redirection sends output to a file instead of displaying it.

The tee command allows you to do both.

Example:


ls | tee files.txt

The output:


files.txt

To append instead of overwrite:


ls | tee -a files.txt

The -a option means:


append

tee is especially useful for:

because it allows you to watch information while saving a copy.

THKI Insight

tee is like placing a "T" in a water pipe:

one stream continues to the screen while the other flows into a file.

13. The Linux Philosophy

Unix and Linux were designed around one simple idea:

Write small programs that each perform one task well.

Rather than creating one enormous application that attempts to do everything, Linux provides many specialized tools that can be combined into powerful workflows.

Commands such as:

become far more powerful when connected together using pipes.

This modular design has been one of the defining characteristics of Unix and Linux for decades.

THKI Insight

Experienced Linux users often solve difficult problems not by finding a new command, but by combining familiar commands in new ways.

14. Reading Pipelines Left to Right

When you first encounter a pipeline, it may appear complicated.

The easiest way to understand it is to read it from left to right.

Example:


cat file.txt | grep error | wc -l

Think of it as three separate steps:

  1. Display the file.

↓

  1. Keep only lines containing:
    2. 
error

↓

  1. Count the remaining lines.

Once you begin reading pipelines one command at a time, even long pipelines become much easier to understand.

THKI Memory Aid


Create
      ↓
Filter
      ↓
Count

Every pipeline is simply a sequence of small steps.

15. Common Real-World Workflows

Linux administrators routinely combine commands to investigate problems.

For example, to monitor a log file in real time while displaying only error messages:


tail -f logfile.txt | grep error

To browse kernel messages:


dmesg | less

To count configuration files beneath /etc:


find /etc -name "*.conf" | wc -l

Each command contributes one piece of the solution.

Together they become a powerful troubleshooting toolkit.

16. Safety Note

Pipes themselves do not modify files.

However, redirection can.

Always verify commands before pressing Enter, especially when using:

Accidentally overwriting an important file is much easier than recovering it.

A few seconds of careful review can prevent hours of repair.

Chapter Summary

| Command or Operator | Purpose |

|---------------------|---------|

| stdin | Standard input |

| stdout | Standard output |

| stderr | Standard error |

| \| | Pipe output between commands |

| > | Redirect output (overwrite) |

| >> | Redirect output (append) |

| 2> | Redirect error messages |

| /dev/null | Discard unwanted output |

| tee | Display and save output simultaneously |

Key Ideas

Linux commands communicate by passing streams of information.

Understanding:

opens the door to automation, scripting, troubleshooting, and efficient system administration.

Pipes and redirection illustrate one of the central principles of Unix and Linux:

Small tools, working together, accomplish remarkable things.

Practice Exercises

  1. Run:
    2. 
cat

Type several lines of text.

End input using:


Ctrl+D

Observe what happens.

  1. Redirect the output of:
    2. 
ls

into a file.

Display the file using cat.

  1. Compare the behavior of:
    2. 
>

and:


>>

Explain the difference.

  1. Create a pipeline using:
    2. 
cat notes.txt | grep hello

Add:


| wc -l

Explain what each command contributes.

  1. Generate an error intentionally:
    2. 
ls nonexistentfile

Redirect the error into:


errors.txt

Display the saved error.

  1. Experiment with:
    2. 
/dev/null

Observe what changes when error messages are discarded.

  1. Use:
    2. 
ls | tee files.txt

Explain why the output appears both on the screen and inside the file.

  1. Create your own pipeline using at least three commands.

Describe the purpose of each command and the overall result.

Looking Ahead

So far we have learned how to navigate the filesystem, inspect files, search for information, and combine commands into powerful workflows.

In the next chapter, we will begin creating and editing text directly using Linux text editors—an essential skill for programming, scripting, system administration, and software development.