Ovation Ventures Ovation Developer Docs

Policies

Handling Secrets

By Ben C. Forsberg

Audience All Employees and Contractors
Status RFC

TL;DR

Don’t share passwords with GitHub, Slack, Zoom or email. Instead, use 1Password or one of these tools.

Introduction: Why Is This Necessary?

We are pursuing a SOC2 certification by the end of this year. That means that we need to start getting serious about enforcing best practices around secrets and access to client information. As a small team, we have been able to get buy with very little protocol, and the intention is not to drastically change how you work. But we have had rather poor hygiene when it comes to secrets, and, since we deal with client’s financial information, it is particularly important that we start to develop good habits in this area.

Least Permissions

As a general rule, we should keep in mind that everyone should have the Least Permissions they need to work.

Secrets are the hot potato you do not want to get stuck with. If data get leaked, and you had access to it, you’re a suspect! You don’t want that liability. Get in the habit of refusing to accept secrets you don’t need. Accept only what you need to do your job.

In particular, you do not want any secrets that give you access to client or user personal data (Personally-Identifiable-Information, PII). “With great power, comes great responsibility.” If you can avoid that responsibility—and that liability—you absolutely should.

Also: whenever possible, you want to use your own credentials, not shared credentials. The reason is that when you use your own credentials, there is a log of what you did. If there is a problem, you want to be able to prove that it wasn’t you. (Or, if it was you, to demonstrate why you were only doing your job.)

Role-Based Policies

Deciding who needs access to what is complicated. So a useful abstraction is to base access to secrets on roles. This person is a front-end developer and a mobile developer, so they have two roles. Each role gives them access to some secrets.

In addition, it allows us to easily add and—importantly—remove people from groups to control access to secrets.

Classification: What is a Secret?

Not everything is a secret. It’s important to separate the pieces of information that need to be kept secret from those that don’t. One reason is that, whenever you access a secret, you potentially expose it to leaks. So the less you have to access secret vaults, the less exposure you create.

For example, if you were to encrypt an entire config file with secret and non-secret data, and then you needed to lookup one of the non-secret values, you would have to decrypt the entire file, potentially exposing all those secrets inside. A better approach is to have separate configuration files with secrets and non-secrets. Or replace secret values with 1Password URLs that op inject can replace with the real secret values when you need them, but leave the non-secret values in plaintext.

For example, every AWS API Key has two pieces of information: an ACCESS KEY ID and a SECRET ACCESS KEY. Only the SECRET ACCESS KEY is secret. The ACCESS KEY ID can be shared openly, because it only identifies the key; it doesn’t give anyone the right to use it.

In general, account IDs, key IDs and any other IDs are not secret, and should not be encrypted or obscured. Save the extra encryption or obfuscation work for the real secrets: the passwords, and tokens that actually grant access.

Classification: Who is it For?

With every secret, you should consider the audience: who is this secret for? Who needs to see it?

Within in our 1Password account are several Groups and several Shared Vaults. These are tied together with the concept of Role-Based Policies and Classification. You classify a secret by selecting the correct vault to store it in.

The Classifications I have created consider these factors:

Dos and Don’ts

Do not EVER commit a secret to GitHub!

There are bots and scanners that look for these commits and exploit them automatically. At one company I worked for, a dev (not me!) accidentally committed a secret, immediately rewrote history to remove it, and force-pushed the update. It didn’t stop the bots from jumping on it and installing a cryptocoin miner on our Salt server, which pushed it out to a portion of our on-location appliances.

If you ever do accidentally commit an unencrypted secret: inform the team, including the business stakeholders, immediately, and change the secret value. It is not enough to add a commit removing the secret. It’s not enough to edit the commit history to remove the commit, and force push that to GitHub. As soon as the commit is pushed, you should consider the secret exposed. The key or password MUST be changed immediately.

Instead

sops can be used to encrypt secrets in YAML and JSON files in a way that makes them safe to commit. In combination with age , it can be set up to use a public/private key pair that allows developers to add new secrets, but not decrypt existing secrets. op inject is another tool that allows you to replace secret values in text files with special op:// URLs that can be replaced with their real values from 1Password after they’ve been checked out. These op:// URLs are safe to commit to GitHub.

Do not share secret data in Slack, ClickUp, Email, or any other public channel.

We don’t control the information in these third-party applications. If they get hacked, that’s our secrets on the web. As an exception, we are trusting 1Password not to expose our information. That’s kind of necessary, but it’s their business to be trustworthy, so, grudgingly, I’ll trust them.

Instead

For persistent secrets, you can share them through 1Password. Classify the secret and record it in the proper vault in 1Password. You can send a link to the secret in Slack, and anyone with access can see it. Alternatively, if it’s a secret meant for just one person, you have an “Employee” vault (which I cannot see into) where you can create your own secrets and share them with other individuals in our organization.

For communicating with people in other organizations (e.g. clients), 1Password allows you to create a link with additional controls. You can limit access to someone who can prove they have access to a certain email address. You can limit how long the shared link is active. You can specify that it can only be used once. Limit your shared secrets as much as you think you can get away with.

For ephemeral secrets, like temporary passwords, you can use age along with the other developer’s public key (which GitHub exposes at a certain URL, e.g. find my public keys at https://github.com/unquabain.keys ). You can find more information on this page about tools. With this tool, you can encrypt a secret file, which will make it safe to post in Slack or put in emails. The recipient needs to know how to decrypt it, though. So it’s mostly for dev-to-dev communication.

Do not display files with secrets during demos, pair programming or other screen sharing sessions.

Instead

try to remember to stop sharing if you need to look at a file that might contain a secret. Use tools like sops and op to encrypt or obscure secrets in config files.

In Terraform configs, variables can be marked as sensitive = true , which will prevent Terraform from echoing them to the screen during a plan .

Do not save secrets in config files on your computer.

Instead

You can use tools like aws-vault sops and op to keep your secrets in a secure vault, and extract them only when needed, using your computer’s biometric sensors or other verification as needed.

If you’re writing your own tool that requires secrets, consider using the OS’s system secrets vault rather than a configuration file to store them. For Go, there is go-keyring, which makes storing and looking up secrets very easy, and cross-platform. In Rust, the keyring crate is similar. Here is an example of how to use it.

DO use MFA and/or Passkeys wherever they are offered.

This is not currently a requirement, but it will be soon. In particular, I plan to make it a requirement for GitHub and AWS. Get ahead of the curve now! aws-vault and op can work together to provide a convenient way to access MFA credentials on the command line. (For example, when I run aws-vault exec prod -- terraform plan , a prompt appears on my screen asking for my fingerprint. If I use aws-vault again in the next few minutes, it will remember me and not prompt me for each command. Here is an example of how to set that up.)