Using cgroups to limit something's RAM consumption (a practical guide)

December 13, 2013

Suppose, not entirely hypothetically, that you periodically rebuild Firefox from source and that you've discovered that parts of this build process kill your machine due to memory (over)use. You would like to fix this by somehow choking off the total amount of RAM that the whole Firefox build process uses. The relatively simple tool to use for this is a cgroup.

There is probably lots of documentation on the full details of cgroups floating around. This is a practical guide instead.

First we need a cgroup that will actually apply memory limits. Ad-hoc cgroups are created on the fly with cgcreate (which is run as root):

cgcreate -t cks -a cks -g memory,cpu,blkio:confine

I'm doing some overkill here; in theory we only need to limit memory usage. But who cares. As I found out, it's important to specify both -t and -a here; -a lets us set limits, -t lets us actually put something into the new cgroup.

The easiest way to set limits is by writing to files in /sys/fs/cgroup/controller/path. Here we have controllers for memory, cpu, and blkio and our path under them is confine, so:

cd /sys/fs/cgroup
# This is 3 GB
echo 3221225472 >memory/confine/memory.limit_in_bytes

# only gets half of contended CPU and disk bandwidth
# (in theory)
echo 512 >cpu/confine/cpu.shares
echo 500 >blkio/confine/blkio.weight

(Since we gave ourselves permissions with -a, we can set all of these limits directly without being root.)

What parameters controllers take is established partly by poking around in /sys/fs/cgroup, partly by experimentation, partly by Internet searches, and sometimes from the official kernel documentation. Where limits exist (and work) they may have side effects; for example, limiting total RAM here is going to force a memory-hungry program to swap, using up a bunch of disk IO bandwidth.

(If you want this cgroup and its settings to persist over reboot you can make a suitable entry in /etc/cgconfig.conf. On Fedora you may also need to make sure that the cgconfig service is enabled.)

Finally we need to actually run our make or whatever so that it is put into our new 'confine' cgroup and it and its children have their total RAM usage limited the way we want. This is done on the fly with cgexec (run as ourselves):

cgexec -g memory,cpu,blkio:confine --sticky make

You don't need --sticky in various common situations, for example if you're not running the cgroups automatic classification daemon. But I don't think it does any harm to supply it and anyways you may well want to wrap this magic command up in a script so you don't have to remember it.

You can check to see that cgexec is properly putting things into cgroups by looking at /proc/pid/cgroup to see what cgroups a suitable process is part of. In this case you would expect to see memory:/confine among the list. Testing whether your actual cgroup controller settings are working and doing what you want is beyond the scope of this entry.

The good news is that this seems to work for me. My Firefox build process has been significantly tamed.

(I've looked at fair share scheduling with cgroups before, which certainly helped here. People have written all of this information down in bits and pieces and partial explanation and Stack Overflow answers and so on, but since I put it together I want to write it down all in one place for later use. (I'm sure there'll be later use.))

Written on 13 December 2013.
« Some observations from playing with PyPy on DWiki
Why I'm not likely to use Chrome much in the future »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Dec 13 02:00:17 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.