Many Linux users and Linux’s numerous distributions occasionally or regularly work at the command line level. Some tasks can be done faster or better in the terminal than in the graphical user interface. To do this, however, you need to know the various commands and how they work. One such command in Linux is “sed.”

Although the Stream Editor (sed) is a text editor, it’s not what you might think of regarding this kind of software. In Linux, SED is a command that can be used to read and adjust data streams. Therefore, the tool is mainly used in shell programming. But how does it work?

When do you use SED in Linux?

The command ‘sed’ is part of the basic equipment of every Linux installation, as it belongs to the GNU Core Utilities (coreutils). The tool is called a non-interactive text editor. This means that you don’t make a change directly to the file you are editing. Instead, you first create a temporary file whose contents are then passed to the source file. Linux SED proceeds line by line. Each line of a file is read individually, processed, and then output again. The most important function of SED is to search for certain strings in the file and then replace them with other characters.

In this way you can fundamentally change a complete file – almost automatically – with only one command. If you integrate such commands into a shell script, you can greatly simplify repetitive tasks. For example, databases or extensive source code can be maintained in this way. Instead of adjusting each entry by hand, you can go through the complete file in one go with SED.

Syntax and function of the SED command

The SED command works with commands and is applied to files. Both the command itself and the commands can still be extended by options.

sed [Option(s)] 'Command(s)' [File(s)]

You can either enter commands directly in the command or read them from a file. In the latter case, you then enter the path of the file instead of the command.

Options

As is typical with Linux commands, SED also gives you the option to define parameters. These are particularly important with the SED command, since only with them does it become clear how the subsequent command is to be interpreted. The following options are possible:

Option Explanation
-e Indicates that one or more SED scripts are used.
-f Specifies that the script is pulled from a file.
-n Results are not output.
-i Creates a more temporary file that subsequently replaces the source file.
-u No data buffer is used.
-s Multiple files are treated separately instead of as one long data stream.
-r The command accepts extended regular expressions.

The -e and -f options are the most important. They indicate whether the command is directly in the command (then it is an SED script) or whether the command must address an additional file. You can often do without the -e option, since it is the default case. However, as soon as you include more than one command at a time in the command, it is mandatory to enable the option.

Note

If you use the -e option, the parameter must be written directly before the first command. If you use other options, write them in front of the first command. If you include other commands in the command, put the option in front of them, too

Very important – and probably indispensable for your work – is the option -n. If the parameter is not set, every single line of the read text file is displayed in the terminal, which is not very useful especially with large databases. If you activate the option, only those lines are displayed which are also affected by the command.

Commands

A command defines the actions that it has to complete with the source file and considering the specified options.

Command Description
a append: Adds one or more other lines to the selected lines.
c change: Replaces selected lines with new content.
d delete: Deletes the selected lines.
g get: Copies the content from the hold spaces into the patten space.
G GetNewline: Pastes the content from the hold spaces into the pattern space.
h hold: Copies the content from the pattern space into the hold space.
H HoldNewLine: Pastes the content from the pattern spaces into the hold space.
i insert: Inserts one or more lines in front of the selected lines.
l listing: Displays all non-printable signs.
n next: Switches to the following command in the command for the next line.
p print: Displays the selected lines.
q quit: Ends Linux SED.
r read: Reads selected lines out loud from a file.
s substitute: Replaces a specified spring with another.
x xchange: Swaps pattern space and hold space with each other.
y yank: Replaces one specified character with another.
w write: Writes lines into the text file.
! Negation: Applies the command to lines that do not apply to the input.
Fact

The two memory types hold space and pattern space have different tasks: The pattern space describes a short-term working memory. It contains the data with which the command is currently working. The hold space is more long-term. Data that is located there can still be retrieved even if SED is already busy with something else.

The commands can also be extended with options:

Option Description
= Specifies the line number of the selected lines.
p Outputs the changed lines.
g Applies the command to the complete file.
Fact

You should always start and end commands with single quotes. This way, you’ll avoid the input from having to be reinterpreted. In principle, the characters are not necessary, but this way you avoid many sources of error.

Regular expressions

When using SED, it’s important that you have an understanding of regular expressions. The characters are used to tell the command how to handle a sequence of characters. For example, square brackets and round brackets are important:

  • [ABC]: A character class is used when looking for a match from a group of letters, digits, or symbols; i.e. either A or B or C.
  • (ABC): A character group describes a fixed term; so it’s ABC in this order.

Wildcards allow regular expressions to also search for only parts of terms. You can use two different variants:

  • .: The full stop replaces exactly one character.
  • *: The asterisk replaces any number of characters.

In addition, regular expressions give you the possibility to determine the frequency of character (-combinations) more precisely.

  • ?: The question mark indicates that a term may occur once or not at all.
  • +: The plus sign specifies that the character occurs more than once, but at least once.
  • {0,n}: With a number inside curly brackets you specify exactly how often the character combination may occur. If you enter two values (separated by a comma), you determine the minimum and maximum repetitions.

Finally, when dealing with Linux and SED, you can also include logical characters that help you, for example, when linking or nesting search queries.

  • |: A pipe stands between two terms and symbolises an alternative of the two.
  • ^: The circumflex (written directly in front of a term) negates it; this string must therefore not occur.

So, with these characters you modify the input in the SED command and in this way, you can perform complex tasks.

Addresses

The inputs are called addresses in Linux SED. An address is therefore the goal of the command. You can specify this in different ways. In many situations you will enter search queries embellished with regular expressions. But it is also possible, for example, to select lines of the text file directly. Accordingly, you note the addresses differently.

In the first example, you refer to concrete lines:

sed -n '10,50p' text.txt

You would output lines 10 to 50 in the terminal with this code.

If you don’t know exactly which lines contain the information you want to work with, you can search for the address. It is important here that you always start and end expressions with slashes. This way, you separate the actual search term from other information such as the commands.

sed -n 'example.[1-9]/p' text.txt

This code would give you all the lines that are labelled as example or examples followed by a digit.

SED explained with three examples

The Linux SED command can provide fast assistance in the most diverse situations. Especially when you have to make many changes at once in extensive text files, the tool shows its strengths. In the following three examples, we’ll show you various ways in which SED can be used in everyday Linux life.

Searching a text file

The simplest use of the SED command is to search for specific data in a document. This can come in handy, for example, in extensive databases or in source code. In this way, you can quickly find content that you either only want to read or also want to change.

Let’s assume, for example, that you are looking for a Chardonnay in your extensive wine cellar. In addition, you would like to display the first line of the database, in which the individual columns are explained. The following command will help you to find the position of the bottles:

sed -n -e '1p' -e '/Chardonnay/p' wine.txt

You use two commands in succession here. Both are introduced by -e. The result then looks like this:

Shelf Growing area Grape variety Year
1 Pfalz Chardonnay 2001
2 Mosel Chardonnay 1983
3 Alsace Chardonnay 1981

If you now want to display only 1980s vintages, this is also possible with a small change to the code.

sed -n -e '1p' -e '/Chardonnay * 198./p' wine.txt
Shelf Growing area Grape variety Year
2 Mosel Chardonnay 1983
3 Alsace Chardonnay 1981

The wildcard between grape variety and vintage is in principle not important in this example. However, if the database is incorrect or you insert a column later, the information remains correct.

Adding information

With Linux and the SED command you can also extend databases. So, to make new entries, you don’t have to open the file with a full text editor, change it, and save it. Instead, you can make the change with just one line of code.

For our example, let’s assume that you have received two new bottles of wine for your collection. Understandably, you would also like to enter these into your database. To do this, you can simply add a new line to the end of the text file with SED.

sed -i -e '$a2 Mosel Dornfelder 2010' -e '$a4 Alsace Pinot Grigio 2011' wine.txt

The regular expression $ causes SED to jump to the last line first. The command a causes a new line to be added with the content that eventually follows. We use the -i option to make sure the source file is modified directly. Alternatively, you could create a new database:

sed -e '$a2 Mosel Dornfelder 2010' -e '$a4 Alsace Pinot Grigio 2011' wine.txt > wine1.txt

Maintain databases

If you want to change the structure of large databases with many entries in retrospect, this is almost impossible manually. Linux, however, offers a quick solution with SED. Up to now, the individual columns in your file have been separated by a space. Let’s now assume in this example that you want to replace the space with a hyphen. For this we use the s command:

sed -i -e 's/[[:space:]]/-/g' wine.txt

The g at the end of the address ensures that the command is applied to the entire file.

A similar challenge arises if you want to add additional information within a line. For example, let’s assume that in the future you also want to note whether you have already tasted the wine or not. Before you now add more unknown wines to your range, first mark all wines as already tasted.

sed -i -e 's/$/-known/g' wine.txt

Now the marker is inserted in all rows – including the first row where you name the columns. To change this, make one more substitution in the first row.

sed -i -e '1s/known/tasted/' wein.txt

Alternatives to SED

Linux SED is a powerful command with which you can do many different tasks. However, some things can only be solved very awkwardly and with a few tricks. With similar commands you can then perhaps reach the goal faster and safer.

AWK

AWK has established itself as a further development of SED. With this command you also work with regular expressions, but you have additional possibilities, which you may be familiar with from more complex programming languages. This means you can also compose commands with AWK that contain if-else statements or while-do loops.

PERL

While AWK is primarily orientated towards C languages, there is also a command that works on the basis of PERL. Although complex systems can be created with the language, PERL is also suitable for smaller tasks within the terminal or in Bash scripts.

TR

If you want to convert individual characters in a text file, this may be easier with a command other than SED. TR (short for: translate) is designed to replace letters, digits, or special characters with others. For example, you can quickly remove duplicate spaces or adjust case. While such smaller tasks are very easy to do with TR, for more complex work other solutions like SED will be beneficial.

Was this article helpful?
Go to Main Menu