Sed (Stream Editor) is a pattern-matching engine that can perform manipulations on lines of text. Its syntax is closely related to text editors such as vim and ed. Here’s how you can quickly find and replace texts using sed.

foo-poo
foo-boo
foo-zoo
foo-bar

Suppose you need to replace the words [poo, boo, zoo] and then replace them with the word bar. The following syntax will modify the file in place.

For BSD/macOS systems: -

sed -i '' 's/[zpb]oo/bar/g' file-a.txt

For GNU/Linux systems: -

sed -i 's/[zpb]oo/bar/g' file-a.txt

The above syntax replaces the occurrences of specified words in the pattern with the word described in replacing the word.

Why should BSD systems send an extra -i '' flag? Why is it different in these two platforms? To explain this, we need to understand some platform-specific dependencies.

Platform Dependencies

If you’re on a BSD (i.e., macOS) system, I need you to install the GNU sed. You can easily do this via your package manager (i.e., homebrew).

brew install gnu-sed

Now, if you’re on a BSD system, you should use gsed instead of sed to follow along.

What does the -i flag do?

The -i/--in-place flag describes editing a file in place. And the string followed by it specifies the file extension of the backup copy (I will explain this later).

First, let’s try to operate the previous sed command without the -i flag. Before that, revert the file content to the previous state.

foo-poo
foo-boo
foo-zoo
foo-bar

And then, we’ll run the following: -

sed 's/[zpb]oo/bar/g' file-a.txt

Did you see that? The file was never modified; instead, it reads the entire file into a buffer in memory and flushes the result to the standard output. In other words, the file was never edited in place. But what if we want to edit it in place, huh?

Editing in place

Well, this is why we use the -i flag. Doing that so writes the entire buffer to the targeted file.

sed -i 's/[zpb]oo/bar/g' file-a.txt

This is great! However, what if we want a backup snapshot of our file before editing? This is where we specify the backup extension immediately after the -i flag.

File backups

In earlier syntax examples, we didn’t specify any extensions immediately after the -i flag. According to the manpage, if supplied with a file extension, it makes a backup file onto the exact location.

GNU and BSD sed support the following syntax to create a backup fileIn both GNU/BSD systems -i'.bak' is another alternative whereas -i '.bak' is only valid for BSD systems. Strangely, in BSD systems -i'' (without a space) simply won’t work, and it will throw an invalid command code error..

sed -i.bak 's/[zpb]oo/bar/g' file-a.txt

If you send an empty string like -i '' it simply avoid creating a backup file with an extension. In BSD, this is the only way to achieve in-place editing without backups.

There’s more to -i flag and for that you should understand its default behaviour.

The default -i behaviour

Now, this is the most crucial bit. The man page describes that sed expressions with this flag will break symbolic links, and they do not preserve the original file-creation date. What do they mean by breaking symbolic links, huh?

Let’s see an example. First, let’s create a symbolic link to file-a.txt in the same directory. You can do this with the ln command.

ln -s file-a.txt file-b.txt

And you can verify it with ls -la that it indeed have created a soft link (symlink) to the file-a.txt. Perfect! Now, let’s operate the previous expressions and see what happens.

sed -i 's/[zpb]oo/bar/g' file-a.txt

Well, so far, all good. Expressions with -i.bak and plain -i will modify the files as expected. But can we do the same with the file-b.txt? Let’s see.

sed -i 's/[zpb]oo/bar/g' file-b.txt

Did you see it? Even though the file-b.txt is modified, the file-a.txt content is still the same. This is because the sed command recursively searches file symlinks uni-directionally from the original source.

When we operate on a symlinked file, the soft link to the original file is broken, and a separate hard link is created. You can see this result by executing ls -la command. But can we preserve the soft link and still edit the file in-place?

Following symlinks

Yes, we can! According to the documentation, you can do this via the --follow-symlinks argument. If we re-execute the following and see the output, we’d get what we want.

sed -i --follow-symlinks 's/[zpb]oo/bar/g' file-b.txt

With the --follow-symlinks argument, we can manipulate the original file in place without breaking the soft link. However, in BSD systems we cannot do this. Simply if you try to in-place edit a symbolic link it will throw an error called sed: FILE.EXT: in-place editing only works for regular files.

Learn more

If you’re interested in learning more about it, I’d prefer you refer the standard POSIX specification and go from there onwards.

Also, there’s three answers by Michael Klement on Stackoverflow related to sed (1 2 3). Those three answers are just :chef-kiss:.