Input/Output Redirection in Unix

Most Unix system commands take input (by default) comes from the terminal and the resulting output (stream) is displayed on (or directed to) the monitor. Commands typically get their input from a source referred to as standard input (stdin) and typically display their output to a destination referred to as standard output (stdout) as pictured below:

The way of indicating an end-of-file on the default standard input, a terminal, is usually <Ctrl-d>.

As depicted in the diagram above, input flows (by default) as a stream of bytes from standard input along a channel, is then manipulated (or generated) by the command, and command output is then directed to the standard output. The ls command can then be described as follows; there is really no input (other than the command itself) and the ls command produces output which flows to the destination of stdout (the terminal screen), as below:

The notations of standard input and standard output are actually implemented in Unix as files (as are most things) and referenced by integer file descriptors (or channel descriptors). The file descriptor for standard input is 0 (zero) and the file descriptor for standard output is 1. These are not seen in ordinary use since these are the default values.

File Descriptor	Descriptor Points to -
0	Standard Input (Generally Keyboard)
1	Standard output (Generally Display/Screen)
2	Standard Error Ouput (Generally Display/Screen)

Output Redirection:

The output from a command normally intended for standard output can be easily diverted to a file instead. This capability is known as output redirection:

If the notation > file is appended to any command that normally writes its output to standard output, the output of that command will be written to file instead of your terminal:

The simplest case to demonstrate this is basic output redirection. The output redirection operator is the > (greater than) symbol, and the general syntax looks as follows:

command > output_file_spec

Spaces around the redirection operator are not mandatory, but do add readability to the command. Thus in our ls example from above, we can observe the following use of output redirection:

$ ls > my_files [Enter]
$

Notice there is no output appearing after the command, only the return of the prompt. Why is this, you ask? This is because all output from this command was redirected to the file my_files. Observe in the following diagram, no data goes to the terminal screen, but to the file instead.

Examining the file as follows results in the contents of the my_files being displayed:

$ cat my_files [Enter]

foo

bar

fred

barney

dino

If a command has its output redirected to a file and the file already contains some data, that data will be lost. Consider this example:

$ echo line 1 > users

$ cat users

line 1

You can use >> operator to append the output in an existing file as follows:

$ echo line 2 >> users

$ cat users

line 1

line 2

Input Redirection:

Just as the output of a command can be redirected to a file, so can the input of a command be redirected from a file. As the greater-than character > is used for output redirection, the less-than character < is used to redirect the input of a command.

command < input_file_spec

The commands that normally take their input from standard input can have their input redirected from a file in this manner. For example, to count the number of lines in the file users generated above, you can execute the command as follows:

$ wc -l users

2 users

Here it produces output 2 lines. You can count the number of lines in the file by redirecting the standard input of the wc command from the file users:

$ wc -l < users

Note that there is a difference in the output produced by the two forms of the wc command. In the first case, the name of the file users is listed with the line count; in the second case, it is not.

In the first case, wc knows that it is reading its input from the file users. In the second case, it only knows that it is reading its input from standard input so it does not display file name.

Someone will certainly ask if input redirection and output redirection can be combined, and the answer is most definitely yes. They can be combined as follows:

$ wc < my_files > wc_output [Enter]
$

There is no output sent to the terminal screen since all output was sent to the file wc_output. If we then looked at the contents of wc_output, it would contain the same data as above.

To this point, we have discussed the standard input stream (descriptor 0) and the standard output stream (descriptor 1). There is another output stream called standard error (stderr) which has file descriptor 2. Typically when programs return errors, they return these using the standard error channel. Both stdout and stderr direct output to the terminal by default, so distinguishing between the two may be difficult.

However each of these output channels can be redirected independently. Refer to the diagram below:

Standard Error:

The standard error redirection operator is similar to the stdout redirection operator and is the 2> (two followed by the greater than, with no spaces) symbol, and the general syntax looks as follows:

command 2> output_file_spec

Thus to show an example, we observe the following:

$ ls foo bar 2> error_file [Enter]

foo

$ cat error_file [Enter]

ls: bar: No such file or directory

Note here that only the standard output appears once the standard error stream is redirected into the file named error_file, and when we display the contents of error_file, it contains what was previously displayed on the terminal. To show another example:

$ ls foo bar > foo_file 2> error_file [Enter]

$ cat foo_file [Enter]

foo

$ cat error_file [Enter]

ls: bar: No such file or directory

In this case both stdout and stderr were redirected to file, thus no output was sent to the terminal. The contents of each output file was what was previously displayed on the screen.

Another relevant topic that merits discussion here is the special file named /dev/null (sometimes referred to as the "bit bucket"). This virtual device discards all data written to it, and returns an End of File (EOF) to any process that reads from it. I informally describe this file as a "garbage can/recycle bin" like thing, except there's no bottom to it. This implies that it can never fill up, and nothing sent to it can ever be retrieved. This file is used in place of an output redirection file specification, when the redirected stream is not desired. For example, if you never care about viewing the standard output, only the standard error channel, you can do the following:

$ ls foo bar > /dev/null [Enter]
ls: bar: No such file or directory

In this case, successful command output will be discarded. The /dev/null file is typically used as an empty destination in such cases where there is a large volume of extraneous output, or cases where errors are handled internally so error messages are not warranted.

One final miscellaneous item is the technique of combining the two output streams into a single file. This is typically done with the 2>&1 command, as follows:

$ command > /dev/null 2>&1 [Enter]
$

Here the leftmost redirection operator (>) sends stdout to /dev/null and the 2>&1 indicates that channel 2 should be redirected to the same location as channel 1, thus no output is returned to the terminal.

Discard the output:

Sometimes you will need to execute a command, but you don't want the output displayed to the screen. In such cases you can discard the output by redirecting it to the file /dev/null:

$ command > /dev/null

Here command is the name of the command you want to execute. The file /dev/null is a special file that automatically discards all its input.

To discard both output of a command and its error output, use standard redirection to redirect STDERR to STDOUT:

$ command > /dev/null 2>&1

Here 2 represents STDERR and 1 represents STDOUT. You can display a message on to STDERR by redirecting STDIN into STDERR as follows:

$ echo message 1>&2

Redirection Summary:

Redirection Operator	Resulting Operation
command > file	stdout written to file, overwriting if file exists
command >> file	stdout written to file, appending if file exists
command < file	input read from file
command 2> file	stderr written to file, overwriting if file exsits
command 2>> file	stderr written to file, appending if file exists
command > file 2>&1	stdout written to file, stderr written to same file descriptor