Securing Environment Variables

In a previous post we showed you how the environment variable system works, and busted some common myths. But as one helpful reader pointed out, we didn't say much about security. Since it's become common to use env vars for storing secret API keys and other valuable information, It's important to understand the security implications. Let's take a look.

In our previous article, The Rubyist's Guide To Environment Variables, we showed you how the environment variable system works, and busted some common myths. But as one helpful reader pointed out, we didn't say much about security.

Since it's become common to use env vars for storing secret API keys and other valuable information, It's important to understand the security implications. Let's take a look:

The Worst Case Scenario

Imagine that a hacker has gained access to your server as root, or as the user who owns your web applications. In that case, your environment variables will be compromised, along with everything else that's not highly encrypted.

The solution here is to keep people from getting root access to your servers. :)

The Trojan Horse

Some guy you met in an alley by the docks gave you a special souped-up version of imagemagick. He claims it'll run twice as fast as the stock imagemagick.

...So you do what anyone would do. You install it in production. The only problem is that it's designed to steal your environment variables.

Here, I'm running our "malicious" mogrify command from Ruby, using the back-ticks syntax. Mogrify then steals my environment variables, and prints a taunting message.

Environment variables get copied from the parent process (irb) to the child pocess (mogrify) Environment variables get copied from the parent process (irb) to the child pocess (mogrify)

This isn't really a "hack." This is just how environment variables work. They get copied from parent process to child process. If you happen to be running malware, they'll inherit environment variables just like any other app.

There are two ways for this to happen:

  • Someone convinces you to install a piece of malicious software in your toolchain

  • A "benign" piece of your toolchain has a vulnerability and is exploited to reveal your environment variables.

The real solution is to be proactive in preventing either of those from happening. But there is a precautionary step you can take to limit damage in case some ENV stealing bandit does wind up in your toolchain.

Sanitizing the Environment

Whenever you shell out of Ruby to run imagemagick or any other program, that program inherits a copy of your app's ENV. If you have any secrets in the ENV, it will get those too. Unless you strip them out.

You can override specific environment variables when you create the new process, by passing a hash to the system method:

How to pass custom environment variables into Ruby's system method How to pass custom environment variables into Ruby's system method

If you want to stop sending any environment variables when you shell out, you might do something like the example below. Here we create a hash that mirrors ENV, except that all its values have been set to nil. It's not pretty, but it's the only way I've found to remove all env vars when executing a shell command.

How to stop sending any environment variables when you run a shell command from ruby How to stop sending any environment variables when you run a shell command from ruby

Don't throw the baby out with the bathwater

Inheritance is one of the most useful features of ENV variables.

Suppose you need to run a command that writes to S3. Environment variable inheritance means  that the command can share the API keys from your Ruby app in a seamless way.   You'd lose that ability if you never allowed inheritance.

Persistence

Every process has its own set of environment variables that die when the process dies. You can set environment variables in your Ruby app, but they'll disappear as soon as that app quits.

If you need an environment variable to stick around after reboot, it needs to be stored somewhere. That somewhere is usually on the filesystem.

Don't use your app's git repo

I'm sure the folks at github are as honest as the rest of us, but do you really want everyone who has access to your source code to be able to get your API keys and run up a $10,000 AWS bill?

Avoid using .bashrc or .bash-profile for secrets

When you store secrets in files like .bashrc, they're sent as environment variables to every single program that you run as that user. Most of these programs don't need to know your secrets. So why give them to them?

Only reveal your secrets to the processes that need them

If your Rails app is the only process that needs to know your HONEYBADGER_API_KEY, then it's a good idea to only make it available to that process.

There are several gems that let you add environment variables to a file that is loaded when Rails starts. These env vars are only available to your Rails process, and its child processes.  There is a section on the figaro and dotenv gems at the bottom of The Rubyist's Guide To Environment Variables.

Secure your configuration files

If you're loading your environment from a configuration file, you'll want to make sure that its permissions are set so that it's only readable by the user running your web app.

Here, I'm making my config file readable and writable only by the user that created it. In this case, it's the same user that owns my Rails application:

Make configuration files readable only by the user that needs to read them Make configuration files readable only by the user that needs to read them

Since you're not checking it into github, you'll need to either SSH into the server and edit it manually or use a tool like Chef to manage it for you.

Build "firewalls" where possible

One of the really great things about AWS is that it lets you create access control policies that are super granular. A lot of service providers have features like this.

What does that mean? It means that you can have one set of API keys that will only allow whoever has them to upload to a single bucket on S3. You should create separate keypairs for separate activities. One keypair for uploading images. Another for your database backup. And so on.

By operating this way you can limit the damage of any one security breach.

What to do next:
  1. Try Honeybadger for FREE
    Honeybadger helps you find and fix errors before your users can even report them. Get set up in minutes and check monitoring off your to-do list.
    Start free trial
    Easy 5-minute setup — No credit card required
  2. Get the Honeybadger newsletter
    Each month we share news, best practices, and stories from the DevOps & monitoring community—exclusively for developers like you.
    author photo

    Starr Horne

    Starr Horne is a Rubyist and Chief JavaScripter at Honeybadger.io. When she's not neck-deep in other people's bugs, she enjoys making furniture with traditional hand-tools, reading history and brewing beer in her garage in Seattle.

    More articles by Starr Horne
    Stop wasting time manually checking logs for errors!

    Try the only application health monitoring tool that allows you to track application errors, uptime, and cron jobs in one simple platform.

    • Know when critical errors occur, and which customers are affected.
    • Respond instantly when your systems go down.
    • Improve the health of your systems over time.
    • Fix problems before your customers can report them!

    As developers ourselves, we hated wasting time tracking down errors—so we built the system we always wanted.

    Honeybadger tracks everything you need and nothing you don't, creating one simple solution to keep your application running and error free so you can do what you do best—release new code. Try it free and see for yourself.

    Start free trial
    Simple 5-minute setup — No credit card required

    Learn more

    "We've looked at a lot of error management systems. Honeybadger is head and shoulders above the rest and somehow gets better with every new release."
    — Michael Smith, Cofounder & CTO of YvesBlue

    Honeybadger is trusted by top companies like:

    “Everyone is in love with Honeybadger ... the UI is spot on.”
    Molly Struve, Sr. Site Reliability Engineer, Netflix
    Start free trial