Manage your Cloud Run secrets securely with Secret Manager

Manage your Cloud Run secrets securely with Secret Manager

MARTIN OMANDER: Hi, I'm Martin Omander, and today I'll tell you about a pain point from my previous developer jobs and how to solve it with modern technology. I wished I'd had this technology back then. Before my current job as a Developer Advocate at Google, I worked at a string of startups as a software engineer. I always struggled with how to store passwords, API keys, certificates, and other sensitive data. If I hardcoded these secrets into the source code, I had two problems.

One, anyone with access to the source repo could see the secrets. They could copy the secret somewhere else, and we didn't have auditing and access controls. And two, it would be hard to move code between dev, test, and production environments. Secrets typically differ between environments. So in my previous job, I did what the Twelve-Factor App methodology recommends.

I put the secrets in environment variables. That way they were not part of the code base. But the secrets had to be stored somewhere so I put them in a file, which I had to be careful not to check into source control. At one job, we burned secrets to a CD that we kept in the CIO's desk drawer. Handling files without source control is awkward.

I remember thinking that there must be a better way of doing it. Secret Manager and Google Cloud platform provides that better way. It has been generally available since March 2020. We recently added native integration with Cloud Run, which makes it easy to mount secrets in Cloud Run services. Let's see how we can update an existing Cloud Run service to use Secret Manager.

The service is a simple API that returns information about different dog breeds, like this. The code reads database, user, password, et cetera, from environment variables when it creates the database connection pool here. The environment variables are shown here in the Cloud Console in plaintext. So let's say we want a better control and auditing of the database password by moving it into Secret Manger.

We see here that the Cloud Run servers runs as Dog Breed API, a custom service account.

I used the principle of least privilege when I created this account, so it only has the Cloud SQL client role. This means that it can connect to Cloud SQL databases but nothing else. We will need to remember this account for the next step. Next, I will go to the Secret Manager in my project, create a new secret for the database password, and enter its value here. Now, let's see who can access this password.

The Dog Breed API is not in the list yet. Let's add it. Again, we're applying the principle of least privilege. The Dog Breed API account currently has access to db_password, but not any of the other secrets that we see here. This reduces risk.

Now, I'm ready to go back to the Cloud Run service. First, I remove the DB_PASS plaintext environment variable. I feel better already.

 

Then, I'll add it back, but instead of entering the actual password, I will point DB_PASS to the password in Secret Manager. Then I click DEPLOY here so this change takes effect.

And look, the API still works. So, what did we achieve here? Well first, we can now use the principle of least privilege for the password and other secrets. We can define granular permissions for who can access which secrets and who can add new secrets. For example, we just configured it so our Cloud Run service can access, but not write, db_password in production.

And if only a few people in our dev team are allowed to handle secrets in production, we reduce the risk of downtime due to silly mistakes. I know. I once mistakenly deleted the production database at a previous job. Second, we now have auditing of this password.

By default, whenever it is created or updated we'll get a log entry like this.

I usually also turn on additional logging so there will be an entry for whenever a secret is accessed as well. And we achieved all this without changing a single line of source code. Delightful. But what if our existing code reads secrets from a file instead of environment variables? No problem.

We can configure Cloud Run so that secrets look like files to the source code. Pretty neat. I should also mention there is a third way of accessing secrets from source code. Our code could make a call to the Secret Manager client library every time it needs a secret. This obviously means that we have to update our source code, but every time we call the client library we will get the current value of the secret.

So this method is useful if we expect secrets to change often and we don't want to redeploy our Cloud Run servers every time that happens.

The same is true when we read secrets as a file. If we access secrets as environment variables and we update a secret, we need to redeploy the Cloud Run service to get the latest value of that secret. This has been an introduction to using Cloud Run with Secret Manager. The docs have a lot more information, like best practices for how to manage permissions, how to rotate your secrets to reduce risk, and how to version secrets.

Also, in the description below, you will find a link to the source code I used. I really hope that this makes it easier for you to build secure systems. As developers, our code may be entrusted with some very sensitive data. We carry a big responsibility. Let's live up to it.

Please let me know if this video was helpful or if you have any questions.

Also, if you have suggestions for new episodes, please add them in the comments below. Until next time.

Here is a special Free $529 package for you

Thanks for your visit!

https://youtu.be/JIE89dneaGo

Read More: 4 Steps to Build Your Business Network in Austria Fast

https://bit.ly/3sYJDrW