How to Use a 3-Minute Hack for Locally Building a Native Extension

Serverless is hot in the world of software architecture. Many vendors dedicate themselves to serverless, and of course, Amazon, Google, IBM and Microsoft are heavily invested in it. But like any other hot technology, it has some drawbacks.

In this post, I share a super-handy hack for those times when you need to build a local native extension.

Necessity is the mother of Invention and our hack is no different. Rookout helps with production debugging across platforms ranging from monolithic to serverless but many of our customers are already using serverless more extensively, and they’ve driven us to dive deeper into this technology.

Rookout support for Python and Node relies on native extensions to do its magic. If you’ve read this far, you probably know that running native extensions on Lambda and other serverless technologies requires you to prepackage the correct binaries into the function zip file. Since AWS Lambda runs on Amazon Linux, this can be a pain if you are running on Windows, Mac, or Debian Linux.

The Amazon recipe for building native extensions for Lambda builds them on a dedicated EC2 instance, which is far from the most pleasant experience. Many of our clients have asked us to help them build and deploy Lambda functions from any OS.

Following brainstorming sessions, some of our engineers came back with a nifty little hack I’m happy to share with you. Turns out, Docker can easily perform a local task as if it was running on our own computer. The following command line taken from the official Docker guide allows you to run a Docker command within the current folder:

docker run -v `pwd`:`pwd` -w `pwd` -i -t ubuntu pwd

For Docker containers to simulate the AWS Lambda environment, we need to look no further than the LambCI project. To build AWS Lambda compatible native extensions, simply run the following command line:

Node:

docker run -v `pwd`:`pwd` -w `pwd` -i -t lambci/lambda:build-nodejs8.10 npm install

Python:

docker run -v `pwd`:`pwd` -w `pwd` -i -t lambci/lambda:build-python2.7 pip install -r requirements.txt

We can make this even better by hiding the nitty-gritty details of using Docker to build extensions by using scripts. For instance, let’s check out this package.json file that allows you to build extensions on the fly:

{
   “name”: “example”,
   “main”: “index.js”,
   “dependencies” : {
   },
   “scripts” : {
      “install-modules”: “docker run -it -v `pwd`:`pwd` -w `pwd` node:6 npm install”,
      “build-package”: “zip -r package.zip *”,
      “build”: “npm run install-modules && npm run build-package”
   }
}

And to build just run:

npm run build

Both our team and our customers find this hack to be a super-helpful time saver while developing Lambda functions. I hope you find it useful, too.