Resource Control: cgroups, ulimit & OOM

On a shared server, one bad program can hurt everything else. A memory leak in Priya's order API slowly eats all the RAM, a backup job keeps every CPU core busy during lunch-time orders, or a script opens files in a loop until the whole system reports "Too many open files". The fix is not to buy a bigger server but to give each program a fair, limited share.

Linux has two layers of limits. Control groups (cgroups) limit a whole group of processes, such as everything belonging to one service, and systemd manages them for you. ulimits (resource limits) restrict a single process, for example how many files it may open. This lesson covers both, and what happens when memory finally runs out: the OOM killer.

Control groups: every service in its own box

A cgroup is a group of processes that the kernel accounts for together: how much CPU time, memory and disk I/O the group uses, and how much it is allowed. Modern systems use cgroup v2, a single tree mounted at /sys/fs/cgroup. Ubuntu 26.04 removed support for the older cgroup v1 layout completely, so everything here is v2.

systemd builds the tree: each service gets its own cgroup inside system.slice, and each logged-in user's processes go into user.slice. A slice is just a cgroup that holds other cgroups. systemd-cgtop shows live usage per group, like top for services:

systemd-cgtop -m -n 1 | head -n 6

CGroup                                   Tasks   %CPU   Memory  Input/s Output/s
/                                          163   38.2     2.9G        -        -
system.slice                                61   36.9     2.1G        -        -
system.slice/sweets-api.service              6   31.4     1.6G        -        -
user.slice                                  14    0.9   302.7M        -        -
system.slice/nginx.service                   3    0.2     6.3M        -        -

-m sorts by memory and -n 1 prints one screen and exits. It is immediately clear that the API, not nginx, is using 1.6 GB. Because every process a service starts stays in its cgroup, even child processes cannot escape the accounting. systemd-cgls prints the same tree as a list of processes. (Output traced from the systemd-cgtop manual page; numbers are examples.)

Limiting a service with systemd

You set cgroup limits as directives in a unit file, best placed in a drop-in made with sudo systemctl edit sweets-api:

[Service]
MemoryHigh=400M
MemoryMax=512M
CPUQuota=50%
TasksMax=100

After saving, restart the service and check the values systemd applied:

sudo systemctl restart sweets-api
systemctl show sweets-api -p MemoryMax -p CPUQuotaPerSecUSec -p TasksMax
cat /sys/fs/cgroup/system.slice/sweets-api.service/memory.max

MemoryMax=536870912
CPUQuotaPerSecUSec=500ms
TasksMax=100
536870912

512M became 536,870,912 bytes, and 50% became 500 ms of CPU time per second. The file under /sys/fs/cgroup proves the kernel itself enforces the limit. Now a memory leak in the API kills only the API (and Restart=on-failure brings it back), while nginx, SSH and the database keep running. To experiment without editing files, sudo systemctl set-property --runtime sweets-api CPUQuota=25% changes a limit until the next reboot.

You can also run a one-off command inside its own limited group. This is a neat way to keep a heavy backup from slowing the website:

sudo systemd-run --scope -p CPUQuota=30% -p MemoryMax=300M tar -czf /srv/backups/www.tar.gz /var/www

Running as unit: run-r4f8c2a1e0b2d4c6e8a0b1c2d3e4f5a6b.scope; invocation ID: 9a1c3e5f7b2d4f6a8c0e1b3d5f7a9c2e

--scope runs the command in your terminal as usual but places it in a temporary cgroup with the limits you gave. The older tools for the same job, nice and ionice -c3 (only use the disk when nobody else needs it), adjust priority rather than set hard caps; see Processes & Signals.

ulimit: limits for one process

Every process also carries resource limits, inherited from its parent. Each has a soft limit, the value actually enforced, and a hard limit, the ceiling up to which an ordinary user may raise the soft one. The best known is the number of open files, because every network connection is also a file descriptor:

ulimit -n
ulimit -Hn
grep 'open files' /proc/$(pgrep -o nginx)/limits

1024
524288
Max open files            1024                 524288               files

The soft limit of 1,024 open files is fine for a shell but too low for a busy web server or database, which then logs Too many open files. Where you raise it depends on how the program starts, and this is the part people get wrong:

• A systemd service: add LimitNOFILE=65536 in a drop-in for the unit. The shell's settings do not apply to services.
• Login sessions (SSH, console): add lines such as priya soft nofile 8192 to a file in /etc/security/limits.d/. These are applied by the PAM module pam_limits at login, explained in PAM & Password Policies.
• The current shell only: ulimit -n 4096 raises the soft limit up to the hard limit, until you close the shell.

Other useful limits are -u (number of processes for a user) and -c (core dump size, often 0 so crashing programs do not fill the disk). ulimit -a lists all of them.

When memory runs out: the OOM killer

If the system (or a cgroup with MemoryMax) runs out of memory and nothing can be reclaimed, the kernel's OOM killer (out of memory) picks a process and kills it, logging a line such as Out of memory: Killed process 2875 (python), as you saw in Logs & journalctl. It chooses using a score, mostly based on how much memory a process uses. You can see the score and nudge it:

cat /proc/$(pgrep -o -f sweets-api)/oom_score
cat /proc/$(pgrep -o -f sweets-api)/oom_score_adj

668
0

A higher oom_score means a process is more likely to be killed. oom_score_adj ranges from -1000 to 1000 and is added in; the API has the default, 0. A value of -1000 means "never kill this one", which is how critical programs such as an SSH server can be protected so you can still log in to a struggling machine. For your own services, set OOMScoreAdjust= in the unit: a positive value for a disposable worker, a negative one for something critical. Some systems, including Ubuntu desktop installs, also run systemd-oomd, which acts earlier when memory pressure stays high.

Common mistakes

• Setting ulimit in .bashrc for a service. Services do not read your shell files; use LimitNOFILE= in the unit.
• Only MemoryMax and no MemoryHigh. The service hits the wall and gets killed with no warning slowdown. Set MemoryHigh a little lower.
• Reading CPUQuota=100% as "the whole machine". It is one core; a 4-core server has 400%.
• Protecting everything with OOMScoreAdjust=-1000. Then the kernel has nothing safe to kill.
• Ignoring OOM kills. They are a signal to fix a leak or add memory, not just to restart.

Key points

• cgroup v2 groups processes; systemd gives every service its own cgroup under system.slice.
• systemd-cgtop and systemd-cgls show usage per service.
• MemoryHigh, MemoryMax, CPUQuota, CPUWeight and TasksMax in a drop-in cap a service; systemd-run --scope -p caps a single command.
• ulimits limit one process; services use LimitNOFILE=, logins use /etc/security/limits.d/.
• The OOM killer picks by oom_score; adjust it with OOMScoreAdjust= in the unit.

[Service]
CPUQuota=100%
MemoryHigh=1G
MemoryMax=1536M
LimitNOFILE=4096
OOMScoreAdjust=500

sudo systemctl daemon-reload
systemctl show report-builder -p CPUQuotaPerSecUSec -p MemoryHigh -p MemoryMax -p LimitNOFILE -p OOMScoreAdjust

CPUQuotaPerSecUSec=1s
MemoryHigh=1073741824
MemoryMax=1610612736
LimitNOFILE=4096
OOMScoreAdjust=500