Input/Output redirection made simple in Linux

January 07, 2006
Linux follows the philosophy that every thing is a file.

A keyboard, monitor, mouse, printer ... are all files in Linux. Linux identifies each of the hardware with unique file descriptors that are associated with it.

Now this nomenclature has got its own advantages. The main one being you can use many command line tools to send, receive or manipulate data with these devices.

For example, my mouse has the file descriptor /dev/input/mice associated with it (yours may be different).

So if I want to see the output of the mouse on my screen, I just enter the command :

$ cat /dev/input/mice

... and then move the mouse to get characters on the terminal. Try it for yourselves.

In some cases, running the above command will scramble your terminal display. If that happens, you can get back your terminal by typing reset

All programs in Linux have access to 3 special files. They are -

  1. Standard input - 0
  2. Standard output - 1, and
  3. Error - 2

Where the numbers 0, 1, and 2 denote file descriptors.

In the previous example, the tool cat used standard output which by default is the screen or the console to display the output.

Redirecting output to other files


You can easily redirect input / output to any file other than the screen. This is achieved in Linux using input and output redirection symbols.

These symbols are as follows:

> - Output redirection

< - Input redirection

Using a combination of these symbols and the standard file descriptors you can achieve complex redirection tasks quite easily.

Output Redirection


Suppose, I want to redirect the output of ls to a text file instead of the console. I can use the output redirection symbol and do it as shown below.

$ ls -l myfile.txt > test.txt

The above command will redirect the output to the file - test.txt. If the file 'test.txt' does not exist, then it is automatically created and the output of the command ls -l is written to it.

This is assuming that there is a file called myfile.txt existing in my current directory.

Now lets see what happens when we execute the same command after deleting the file myfile.txt.

$ rm myfile.txt
$ ls -l myfile.txt > test.txt
ls: myfile.txt: No such file or directory -- ERROR

What happens is that ls does not find the file named myfile.txt and displays an error on the console or terminal. Now here is the fun part.

You can also redirect the error generated above to another file instead of displaying on the console by using a combination of error file descriptor and output file redirection symbol as follows:


$ ls -l myfile.txt 2> test.txt

By typing 2>, you are telling the program to redirect any error (2) to the file test.txt.

Linux terminals
Two open terminals can be used to practice output redirection

Examples of Redirection


I can give one practical purpose for this error redirection which I use on a regular basis. When I am searching for a file in the whole hard disk as a normal user, I get a lot of errors such as :

find: /file/path: Permission denied

In such situations I use the error redirection to weed out these error messages as follows:

# find / -iname \* 2> /dev/null

Now all the error messages are redirected to /dev/null device. Thus I can reduce a lot of clutter in my terminal.

/dev/null is a special kind of file in that its size is always zero. Anything you write to that file will just disappear.

The opposite of this file is /dev/zero which acts as an infinite source. You can use /dev/zero to create a file of any size - when creating a swap file for instance.

If you have a line printer /dev/lp0 connected to your Linux machine, you can send any output to the printer using output redirection. For example, printing the contents of a text file (testfile.txt) is simple.

$ cat testfile.txt > /dev/lp0

Input Redirection


You use input redirection using the less-than symbol (<) and it is usually used with a program which accepts user input from the keyboard.

A legendary use of input redirection that I have come across is mailing the contents of a text file to another user.

$ mail ravi < mail_contents.txt

Now with the advances in GUI, and also availability of good email clients, this method is seldom used.

Here is another example of input redirection ...

Suppose you want to find the exact number of lines, number of words and characters respectively in a text file and you want to simultaneously write it to another file.

You can do it using a combination of input and output redirection symbols as follows:


$ wc < my_text_file.txt > output_file.txt

What happens above is the contents of the file my_text_file.txt are passed to the command wc whose output is in turn redirected to the file output_file.txt.

Appending data to a file You can also use the >> symbol instead of output redirection to append data to a file. For example,

$ cat - >> test.txt

... will append what ever you write to the file test.txt.

Hope you liked this short tutorial on input output redirection in Linux.

23 comments:

  • A very good article. Enjoyed it.

    I may also add that you can do something like :

    ls -l xxx.txt 2>1& another_file

    which will redirect both output and any errors to another_file.

    Henry

  • Excellent. The first time I have ever really understood redirection!

  • You mean:

    ls -l xxx.txt 2>&1 another_file

    What you said will redirect stderr to a file named 1 and try to run another_file as a command. I prefer this simpler syntax to redirect stdout and stderr to a file:

    ls -l xxx.txt &> another_file

  • Maybe the first example:

    $ cat /dev/input/mice

    should be changed by:

    # cat /dev/input/mice

    if for example I had a vulnerability in a web application and someone could just read one of those files from www-data user, and with them, the attacker could read all what I do with my mouse or my keyboard, I would be really scared.

    Fortunately, just root can do that

  • Aren't you glad that GNU provides all of these core utilities? Welcome to the GNU userland.

  • @ anonymous 5:57 AM

    $ cat /dev/input/mice

    should be changed by:

    # cat /dev/input/mice


    I think the author has written $ cat... instead of # cat ... for a reason. He is just conveying that it is better to play it safe and run those commands as a normal user and not as root.

    A nice article. Cleared a lot of my doubts on this topic.

  • >I think the author has written $ cat... instead of # cat ... for a reason. He is just conveying that it is better to play it safe and run those commands as a normal user and not as root.

    In a sane environment you can't do that. Normal users only will have permissions to write to this device, not to read from it.

  • If "everything is a file", then where are the semantics of file operations defined? I'm especially interested in the semantics of 'ioctl'.

  • great
    this shows, how powerful gnu/linux/unix is

  • @ anonymous (3:26 PM)

    Semantics is defined at some appropriate driver level.

    For example, do a 'ls -l' on the files in the /dev directory, these are special devices with major/minor number pairs which maps to a device driver in kernel space, hence the semantics is left to that driver.

    The same can be applied to ordinary files and directories, The VFS layer performs the mapping to the appropriate file system driver for those.

    The character far left of an 'ls -l' on a file tells the file type (e.g. '-', 'd', 'b' or 'c' to mention a few)

  • I think the author has written $ cat... instead of # cat ... for a reason. He is just conveying that it is better to play it safe and run those commands as a normal user and not as root.

    Yeah, run everything with the less privileges you can... but reading /dev/input/mice is something that just root can do it! run those commands as a normal user and you will get a "Permission denied".

    In a sane environment you can't do that. Normal users only will have permissions to write to this device, not to read from it.

    In a sane environment you also can't write to the device.

    Those files represent devices.

    Would you imagine that *any user* of the system would have permissions over input devices? I'm thinking in "postfix" user, "www-data" user, and so on.

    You would be in almost the same situation as if you were just in front of the computer.

  • This should have been labeled 'Simple Input/Output Redirection at the command-line.' A more useful article would have been how to redirect stdin, stdout, stderr within a shell script using 'exec cat'. With a little info about tee thown in for good measure.

    /djs

  • wc < my_text_file.txt > output_file.txt

    can also be written as

    wc my_text_file.txt | tee output_file.txt

  • Great article. I've always wanted to know how to redirect error messages. I'm kicking myself now since it is so easy.

  • I appreciate the examples given. They are very clear and easy to understand. It's a great article indeed.

  • Nice article, but let's dig deeper.
    For example, you woud like to change your password.
    You type:

    $ passwd

    That will ask for passwords.
    Now how do you realize a redirection in this case, if you want to input these passwords from a file?

  • Very good,
    Thank you!

  • good ... but I would like to
    redirect the output of command "dsh"
    dsh -a ps -u $USER > file
    it does not work !!!!
    how can i do this?
    Thank you!
    Cabellos JL

  • dsh -a ps -u $USER > file

    probably because 'dsh' is a shell that's accepting a command as an argument. in this case it thinks you're passing it

    ps -u $USER > file

    in order to redirect the output of that command to the file you'd need to do:

    dsh -a "ps -u $USER" > file

  • well, to ensure that the terminal is not scrambled, you could use hexdump

    $ hexdump /dev/input/mice

    prints a viewable dump..od would work as well.

  • Now how do you realize a redirection in this case, if you want to input these passwords from a file?

    just try this one

    useradd eprasad | echo 'password' | passwd --stdin eprasad

  • Brad Fallon

    Simple linux question. What does the -aux mean in ps -aux same question with ps -ef what does -ef mean?

  • Helpful article. I tried and found that only

    $ cat f1 f2 f3 &> file

    works. Here assuming only f1 and f3 exist and f2 does not exists.

    "file" would contain the normal output as well as error messages

    Again

    $ cat f1 f2 f3 1>outfile 2> errfile

    here "outfile" would collect normal output only and "errfile" would collect error messages