Shell Scripting – Standard Input, Output and Error
Last Updated : 25 Jan, 2024
Working on Linux applications we have several ways to get information from outside and to put it inside: command line args, environment variables, files. All of these sources are legal and good. But it has a finite size. Another way to establish communication is standard streams: input stream (stdin) used for getting data from outside of the app, output stream (stdout) to put data outside of the app, and error to put data outside of the app (stderr).
standard Linux application stream at sh
Each stream acts like a pipe: It has the same buffer to write and read the data. This buffer is available for reading from one application and available for writing from another one. On reading, the occupied buffer size will be reduced and it will be increased on writing. If the average rate of reading and writing is equal – then data passed over the stream can be any number of bytes long.
Let’s look at an example of the “grep” application describing how it interacts with it.
script body:
#!/usr/bin/env bash
echo pattern | grep pattern
echo pattern | grep pattern
The script running at the console:
script body in the yellow frame, output next
This time we ran 2 commands: “cat stdio1.sh” – to list script body, “./stdio1.sh” to run the script. “echo pattern” forms the standard output stream that is connected to the standard input stream of grep. So all of the data comes to grep. grep is an app that writes from input stream to output stream if it matches the pattern. The first input does not match – no output expected. The Second-string of the script “echo pattern | grep pattern” has appropriate input – it has been passed to grep output and displayed at the console next.
Error stream operating in the example
Stderr is very similar to stdout: with only one difference – it will contain an error report if it will take a place
script body:
#!/usr/bin/env bash
ls > /tmp/log
cat /tmp/log
ls /not/existed_path > /tmp/log
cat /tmp/log
cat /tmp/log
rm /tmp/log
The script running at the console:
script body in the yellow frame, output next
At this script, we make a listing of files in the current directory (“ls”) and redirect standard output to the file (“/tmp/log”). Then we display this file data at the console from the standard output stream of the cat utility that read the file. Next, we try to list files in the not existing folder and store results into the file as above, and display it two times. But there is no effect. Just one output about the error in the console. That is because this error comes from the standard errors stream of ls. We removed the file to clear some space “/tmp/log“.
Streams redirecting
It is possible to manipulate the standard streams: redirect them or use pipes to process them.
script body:
#!/usr/bin/env bash
ls /not/existed_path 2>/dev/null
ls * /not/existed_path > /tmp/log 2> /tmp/err_log
cat /tmp/log
rm /tmp/log /tmp/err_log
The script running at the console:
script body in the yellow frame, output next
We use the “>” redirect action in this command. /dev/null – is a stream termination file to drop any input with no processing. “2>” clause redirects standard output stream to file. Now it is possible to look at output and errors later by calling “cat /tmp/log” or “cat /tmp/err_log”.
Discard the output
In some cases, it is useful to drop all of the application output (to save storage space or remove the unimportant sources of data to analyze). We should remember that we should mute not just the standard output stream but the standard error stream also. Let’s look at an example of how does it work.
script body:
#!/usr/bin/env bash
ls . unexisted_file > /dev/null
ls . unexisted_file > /dev/null 2>&1
The script running at the console:
script body in the yellow frame, output next
In both lines of script standard output stream redirected to file /dev/null. This file is not a file itself: but a special Linux device with a file interface. It has a write function available from shell (that does nothing). So all of the input is just dropped. But the second string of the script has a difference from the first one: “2>&1” at the end of the command. These symbols tell that the second standard stream ( standard error stream ) needs to be directed at the first standard stream (standard output stream) permanently for this command. That is why we see only one error message displayed in the console – at the second command string of the script standard error stream redirected to terminating file device /dev/null in the end.
Pipelined streams
It also can be useful to modify the stream. is able to run several applications connected by the streams.
script body:
#!/usr/bin/env bash
touch /tmp/a
ls /tmp/* 2> /dev/null | grep -w a
rm /tmp/a
The script running at the console:
script body in the yellow frame, output next
Here we collect the listing and set “ls” standard output stream as grep standard input stream by using “|”. On “2>” standard errors stream is redirected to the termination file. The output stream of grep contains the “/tmp/a” filename from listing that matches by grep pattern.
Here document
Here the document is redirection option to fill input stream of application by information native way:
application << delimiter
some useful text
delimiter
script body:
#!/usr/bin/env bash
grep perfect << EOF
PERFECT STORY
past perfect
is it Perfect?
EOF
The script running at console
script body in the yellow frame, output next
As we can see only the second string from 3 of the strings of text for the grep standard input stream delimited by “EOF” match the grep pattern: that is why it passed to grep standard output stream and be displayed in the console next.
Conclusion
There are several standard utilities to work with streams (cat, cut, grep, sed, tr, …). You can use it or write your own to make a data stream processing pipeline that will match your needs. Under the hood, it will work with stdin, stdout, stderr.
Similar Reads
Introduction to Linux Shell and Shell Scripting
If we are using any major operating system, we are indirectly interacting with the shell. While running Ubuntu, Linux Mint, or any other Linux distribution, we are interacting with the shell by using the terminal. In this article we will discuss Linux shells and shell scripting so before understandi
7 min read
Introduction to Shell Scripting
Basic Shell Commands in Linux
Basic Shell Commands in Linux: Complete List
Anyone using Linux should become an expert in the essential shell commands, as they form the backbone of working with the Linux terminal. These commands enable you to navigate the system, manage files, handle processes, and configure settings effectively. The Linux shell serves as an interface for u
5 min read
Linux Directory Structure
Prerequisite: Linux File Hierarchy Structure In Linux/Unix operating system everything is a file even directories are files, files are files, and devices like mouse, keyboard, printer, etc are also files. Here we are going to see the Directory Structure in Linux. Types of files in the Linux system.
5 min read
Input Output Redirection in Linux
In Linux, whenever an individual runs a command, it can take input, give output, or do both. Redirection helps us redirect these input and output functionalities to the files or folders we want, and we can use special commands or characters to do so. For example, if we run the "date" command, it giv
4 min read
Functions and Script Organization
Command-Line Arguments and Options
Error Handling and Debugging