But I'd Ordered 3 Gigs of Memory

Operating Systems (*nix in particular) are fascinating¹. It does a whole range of cool optimizations for you, the happy programmer. Need some memory? Go ahead! Use a malloc() in C. Here’s the catch: unless you have some idea about how a chunk of memory becomes yours, you’re probably not going to like writing a program to eat a specified amount of memory.

Context

I had to trigger some alarms on AWS instances by reducing the amount of free memory available. Like any other lazy (read: good) programmer, I wanted to automate doing that. With some advice from the DevOps team and a quick hack on the system, deploying the memory eater on the instances and running it would be a piece of cake. One thing remained - writing a memory eater.

Going Retro

One option was to use Java - set min, max JVM memories and so on. However,

1. There’s the possibility of a spurious wakeup (fixable, but undesirable).
2. Thread.sleep() had notable CPU utilization - was too lazy to figure out why it happened.
3. The process could get swapped out soon. swapoff -a² is risky and I didn’t want to do that.
4. There’s Java’s GC - to be safe, I shouldn’t stress the memory of the system beyond a certain extent, and, I postponed reading about the GC’s actions on swapped out pages to another fine day.³

So, not Java.

On a related note, there are some commands you can use to explore memory-related events.

1. Run /usr/bin/time -v <program> on the terminal to see some interesting stats about the program.
2. Run ps -oe cmd min_flt,maj_flt to see the number of minor and major page faults of running processes
3. Run vmstat 1 to see swapping details.

Back to the problem at hand, the biggest issue was page swapping. I was sure C had some weird functions that would give me all the control I needed - which it did, but not without its own share of surprises. My obvious solution was to do a malloc of 3 GB and scanf to wait for releasing memory (by exiting). Well, it didn’t work - it was consuming much less real memory. So I used calloc. Slightly better, but nowhere close to the 3 GB I’d asked for.

What do you do when [m|c]alloc doesn’t work the way you expect it to?

You read a pdf that details how malloc might work. If you’re lazy/not curious, you’d skip right over it to a StackOverflow link that has a labored ELI5 version for malloc. When that doesn’t help, you can either resign to doing manual grunt work, or figure out why it’s happening.

There was something about malloc having an mmap call at some point down the line in one of the StackOverflow pages. From one of the courses ⁴, I remembered a quirk of mmap - the kernel does not allocate the entire memory in one go. It works like an on-demand allocator. With malloc, it has one page of read-only 0s. When the program writes to it, there’s an exception and the kernel maps a new page to the processor’s address space.

Fine, but calloc is supposed to work! Turns out, calloc touches only a few page tables. The solution was to use a memset which touches every page in the allocation. To check my memory, I searched again and found an answer which had everything I needed.

The mlockall function with the right flags would lock the memory pagges from getting swapped out (when run with superuser permissions). And when the process terminates, the OS takes care of unlocking these memory segments and freeing them.

The memory eater looks like this:

#include <sys/mman.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>

#define BLOCK_SIZE 1024*1024

int main(int argc, char **argv){
 if (argc != 3){
  printf("Usage: %s <mem in MB> <fifoPath>", argv[0]);
 }
 int verbose = 0;
 int mem = atoi(argv[1]);
 mlockall(MCL_CURRENT|MCL_FUTURE);
 int target=0;
 while(target < mem){
  void *buf = malloc(1024*1024);
  memset(buf,1,1024*1024);
  if(target%100==0) printf("Mem used: %d\n",target);
  target += 1;
 }

 char buff[1024];
 fscanf(fopen(argv[2],"r"),"%s",buff);
 printf("Done\n");
 return 0;
}

One change here is the use of fscanf for cleaner termination in an automated scenario - this would run in the background. I used mkfifo to create a fifo for each instance of this application and passed it as an argument to the application.

mkfifo hue
./a.out 4000 hue &
sleep 5s
echo hue > hue

fscanf is a blocking call and whatever is echo-d into the fifo is scanned into buff and the program terminates.

Those were 3 interesting hours.