What is sed command on Linux?

Many Linux users and Linux’s numerous distributions occasionally or regularly work at the command line level. Some tasks can be done faster or better in the terminal than in the graphical user interface. To do this, however, you need to know the various commands and how they work. One such command in Linux is “sed.”

Although the Stream Editor (sed) is a text editor, it’s not what you might think of regarding this kind of software. In Linux, SED is a command that can be used to read and adjust data streams. Therefore, the tool is mainly used in shell programming. But how does it work?

The command ‘sed’ is part of the basic equipment of every Linux installation, as it belongs to the GNU Core Utilities (coreutils). The tool is called a non-interactive text editor. This means that you don’t make a change directly to the file you are editing. Instead, you first create a temporary file whose contents are then passed to the source file. Linux SED proceeds line by line. Each line of a file is read individually, processed, and then output again. The most important function of SED is to search for certain strings in the file and then replace them with other characters.

In this way you can fundamentally change a complete file – almost automatically – with only one command. If you integrate such commands into a shell script, you can greatly simplify repetitive tasks. For example, databases or extensive source code can be maintained in this way. Instead of adjusting each entry by hand, you can go through the complete file in one go with SED.

The SED command works with commands and is applied to files. Both the command itself and the commands can still be extended by options.

You can either enter commands directly in the command or read them from a file. In the latter case, you then enter the path of the file instead of the command.

As is typical with Linux commands, SED also gives you the option to define parameters. These are particularly important with the SED command, since only with them does it become clear how the subsequent command is to be interpreted. The following options are possible:

The -e and -f options are the most important. They indicate whether the command is directly in the command (then it is an SED script) or whether the command must address an additional file. You can often do without the -e option, since it is the default case. However, as soon as you include more than one command at a time in the command, it is mandatory to enable the option.

Very important – and probably indispensable for your work – is the option -n. If the parameter is not set, every single line of the read text file is displayed in the terminal, which is not very useful especially with large databases. If you activate the option, only those lines are displayed which are also affected by the command.

A command defines the actions that it has to complete with the source file and considering the specified options.

The commands can also be extended with options:

When using SED, it’s important that you have an understanding of regular expressions. The characters are used to tell the command how to handle a sequence of characters. For example, square brackets and round brackets are important:

• [ABC]: A character class is used when looking for a match from a group of letters, digits, or symbols; i.e. either A or B or C.
• (ABC): A character group describes a fixed term; so it’s ABC in this order.

Wildcards allow regular expressions to also search for only parts of terms. You can use two different variants:

• .: The full stop replaces exactly one character.
• *: The asterisk replaces any number of characters.

In addition, regular expressions give you the possibility to determine the frequency of character (-combinations) more precisely.

• ?: The question mark indicates that a term may occur once or not at all.
• +: The plus sign specifies that the character occurs more than once, but at least once.
• {0,n}: With a number inside curly brackets you specify exactly how often the character combination may occur. If you enter two values (separated by a comma), you determine the minimum and maximum repetitions.

Finally, when dealing with Linux and SED, you can also include logical characters that help you, for example, when linking or nesting search queries.

• |: A pipe stands between two terms and symbolises an alternative of the two.
• ^: The circumflex (written directly in front of a term) negates it; this string must therefore not occur.

So, with these characters you modify the input in the SED command and in this way, you can perform complex tasks.

The inputs are called addresses in Linux SED. An address is therefore the goal of the command. You can specify this in different ways. In many situations you will enter search queries embellished with regular expressions. But it is also possible, for example, to select lines of the text file directly. Accordingly, you note the addresses differently.

In the first example, you refer to concrete lines:

You would output lines 10 to 50 in the terminal with this code.

If you don’t know exactly which lines contain the information you want to work with, you can search for the address. It is important here that you always start and end expressions with slashes. This way, you separate the actual search term from other information such as the commands.

This code would give you all the lines that are labelled as example or examples followed by a digit.

The Linux SED command can provide fast assistance in the most diverse situations. Especially when you have to make many changes at once in extensive text files, the tool shows its strengths. In the following three examples, we’ll show you various ways in which SED can be used in everyday Linux life.

The simplest use of the SED command is to search for specific data in a document. This can come in handy, for example, in extensive databases or in source code. In this way, you can quickly find content that you either only want to read or also want to change.

Let’s assume, for example, that you are looking for a Chardonnay in your extensive wine cellar. In addition, you would like to display the first line of the database, in which the individual columns are explained. The following command will help you to find the position of the bottles:

You use two commands in succession here. Both are introduced by -e. The result then looks like this:

If you now want to display only 1980s vintages, this is also possible with a small change to the code.

The wildcard between grape variety and vintage is in principle not important in this example. However, if the database is incorrect or you insert a column later, the information remains correct.

With Linux and the SED command you can also extend databases. So, to make new entries, you don’t have to open the file with a full text editor, change it, and save it. Instead, you can make the change with just one line of code.

For our example, let’s assume that you have received two new bottles of wine for your collection. Understandably, you would also like to enter these into your database. To do this, you can simply add a new line to the end of the text file with SED.

The regular expression $ causes SED to jump to the last line first. The command a causes a new line to be added with the content that eventually follows. We use the -i option to make sure the source file is modified directly. Alternatively, you could create a new database:

If you want to change the structure of large databases with many entries in retrospect, this is almost impossible manually. Linux, however, offers a quick solution with SED. Up to now, the individual columns in your file have been separated by a space. Let’s now assume in this example that you want to replace the space with a hyphen. For this we use the s command:

The g at the end of the address ensures that the command is applied to the entire file.

A similar challenge arises if you want to add additional information within a line. For example, let’s assume that in the future you also want to note whether you have already tasted the wine or not. Before you now add more unknown wines to your range, first mark all wines as already tasted.

Now the marker is inserted in all rows – including the first row where you name the columns. To change this, make one more substitution in the first row.

Linux SED is a powerful command with which you can do many different tasks. However, some things can only be solved very awkwardly and with a few tricks. With similar commands you can then perhaps reach the goal faster and safer.

AWK has established itself as a further development of SED. With this command you also work with regular expressions, but you have additional possibilities, which you may be familiar with from more complex programming languages. This means you can also compose commands with AWK that contain if-else statements or while-do loops.

While AWK is primarily orientated towards C languages, there is also a command that works on the basis of PERL. Although complex systems can be created with the language, PERL is also suitable for smaller tasks within the terminal or in Bash scripts.

If you want to convert individual characters in a text file, this may be easier with a command other than SED. TR (short for: translate) is designed to replace letters, digits, or special characters with others. For example, you can quickly remove duplicate spaces or adjust case. While such smaller tasks are very easy to do with TR, for more complex work other solutions like SED will be beneficial.