This article describes the bare metal container technology which has taken the computing world by storm. The technology isn’t new but rather has been in use for a very long time. The difference is now that it’s much more cleaner and effective.

Consider for example that you were trying to create your own distribution or packages for several distributions. What we want is to have complete isolation of the development environment of such a package or distribution creation system from our own installed system.

Earlier this was done using chroot which allowed us to change the root file system and move into another root file system which would probably be the distribution or package creation environment and has nothing to do with the installed system. Let’s take a look how this can be done

chroot (Change root / pivot_root) way of isolation

Some of the issues with chroot way are listed below

• only a the filesystem was pivoted but the user would still be the actual system user not the chroot jail user.

• Couldn’t isolate processes, we would still actually see system processes within chroot.

• Couldn’t isolate network. I.e. iptables / ebtables rules were still applied to the system, including the chroot jail.

Implementing a chroot jail

In order to appreciate what namespaces bring to the table, let’s first try the old way chroot command. You may’ve to install additional package(s) to get chroot utility on your system.

Packages Required

Compiling BusyBox

From the shell, cd into BusyBox source directory and run the following command

And make sure you select the Build static binary option under Settings. You may also need to install libncurses if you already haven’t done so to enable the menuconfig GUI appear.

Chroot into the busybox

If the installation was success you’ll have a directory _install. This contains a similar looking directory structure what you would expect on a linux machine. Now with root user you can do the following, (can also use sudo)

You should’ve a pound(#) prompt with a directory structure similar to as shown below,

As you can see there’s no proc directory so doing a ps command will give an error. To fix this first let’s mount procfs so we can see our processes.

Mounting procfs inside chroot jail

Now when we do a ps -ef command, it’ll show you all the processes not only from the chroot but also  of the system. This is one of the major drawbacks of chroot, though it solves the root file system issue (we just changed it above) but it has a lot of limitations. User is also root which is also the system’s root and not isolated to chroot.

Before you exit, unmount the procfs from chroot!

Make sure you do umount /proc before typing exit. That’s to avoid any surprises later!

So what we want …

• To have complete isolation we don’t want at the very least the chroot users to exist on our installed machine.
• It would be nice to have separate mount of proc and sysfs so they would show only the process information about the chroot.
• Network isolation would be nice.
• Controlling resources allocated to such a chroot is much desired.

Enter Namespaces

Just like namespaces in programming languages, namespaces are isolated “places” which are accessible to processes living in that namespace. There are multiple namespaces (7 in all) implemented and all processes are part of exactly one of each of these namespaces.

However some namespaces can be cascaded and some can be shared (mount namespace) but under no circumstance can the hierarchy be reversed, we’ll take a look at this later.

To be able to use the namespaces there are only 2 new system calls. Most of the heavy lifting is done by the Linux kernel at the time of process creation. In order to understand namespaces correctly it’s important that we do some hands-on

Check if namespaces are available in the kernel

What we are looking for are the following options,

Each process in Linux is part of each of these 7 namespaces and can only be part of exactly one of each. So for example if a running process changes it mount namespace it no longer remains part of the previous mount namespace.

The only distinction is the PID Namespace which is actually cascaded, otherwise it would be impossible to control the child processes from parent process. We’ll take a look at what this means. 

These appear to be broken symlinks however you can see that each of the namespace has an id attached to it. Processes whose namespaces are same will have exactly same ids. So for example a command run from the shell will inherit it’s parent’s namespace unless it’s changed explicitly by the command. 

NOTE: Replace $$ with any other process’s pid to see it’s namespace.

Types of namespaces

A process can do either of the following two,

• Create a new namespace , one or all of the above 7 when cloning a child process.
• Become part of an existing namespace(s).

To see the above in action, let’s use the tools already available

The above tools use the following system calls

Creating a simple usable container

Things we’ll need

1. Busybox, so we can have everything we need in a small package.
2. A running Linux kernel with namespaces enabled.
3. Chroot installed on the system.
4. Utilities unshare and nsenter.

After installing BusyBox, we need to do the same steps. i.e. chroot into the BusyBox however we would also change our namespaces while we’re at it so that we are isolated from the rest of the system.

To see what all options are used, see man unshare. I’ll explain some options below,

With the above command successful you should see the same directory structure as we did earlier with chroot. Again first let’s mount the procfs, but this time you’ll see the difference

As you can see, though we are root, we can’t see the processes outside our chroot jail. This is in essence the basic idea of container. 

Now let’s see what the namespace ids are for our init process, in this case the shell (/bin/sh executed via chroot).

Now let’s see what changed, without closing the above busybox terminal,  from another terminal session

Now if I compare the namespaces of the above process (/bin/sh) and the shell I’ve on my other terminal

Apart from the cgroup, ipc namespaces all other are different since we didn’t created a new cgroup and ipc we expected them to be same.

Important Note

Killing the pid 1 inside pid namespace will kill other processes within that pid namespace. In effect all namespace mappings are destroyed of which pid 1 is part of.

Running a process in another process’s namespace

Now that we’ve a basic container running, let’s try to run a process within it. To demonstrate here’s a very simple C program which does nothing but sleeps for 20 seconds.

#include <unistd.h>
int main(int argc, char *argv[]) {
        for(;;) {
                sleep(20);
        }
}

We can compile the above program and let’s call the binary generated as test.

Now to run this inside the busybox’s shell namespace we are going to use the nsenter utility. All it needs is a target pid and the namespace which needs to be changed to that of the target. In our case

1. The target pid = 27135 i.e the pid of the shell /bin/sh started using the unshare command.

2. The namespaces we need to move into. We’ll move into all namespaces of target except cgroups and ipc.

By default nsenter tries to write to uid_map and gid_map files inside the namespace. The issue is that write on these files is allowed exactly one time by exactly one process inside the PID namespace. When we started unshare, using the  -r option we already wrote to the PID 1 (/bin/sh) uid_map and gid_map file which are then only inherited but can’t be modified.

Therefore we would need to tell nsenter not to write to those files, which can be done using the – – preserve-credentials option. So the command now becomes, note that we are running this from our system’s shell,

This process will now be visible inside the BusyBox’s shell as well using the usual ps command,