How good are you in keeping secrets?

Why do we need to keep secrets?

I am not sure why I chose this title for this first segment of the article; I am old and experienced enough to not think about it twice - just like you. Perhaps I was looking for a good way to break the ice. Let's start by setting the basics aside: trust is not something you do on the internet. Instead, you implement mechanisms for identifying and authenticating people (users) and services, in order to exchange confidential information. The safe-keeping of credentials and keys is also a very crucial aspect of this process. Without those mechanisms, trust remains fragile and any bad actor can induce chaos on any system or community. So, allow me to rephrase: when, what and how do we need to keep a secret?

Scope of the article and assumptions

The majority of applications in the world wide web are structured based on the client-server model. Simply phrased, client and server communicate over a network, usually on separate hardware, by exchanging messages. On this article, we refer to as "client" any frontend application that runs on a browser and "server" any backend application that delivers information to a frontend. Furthermore, we assume that the backend uses stateless authentication. This eliviates the need to create and manage a session there. Instead, we use tokens which contain information about the user, which the backend verifies each time it is required.

In the next chapters, we will take a look into how we would safely handle credentials from users and API keys from applications. With that out of the way, let's jump straight to it!

User credentials

Most applications need to verify the identity of the user. In order to achieve this, the user often provides a combination of strings (e.g. an email address and a password) that only they and the application know. We refer to this combination of strings as "credentials". Preventing imposters is a major security undertaking and one whose failure can lead to unwanted data access. This is highly important and currently ranks 7 in the OWASP Top 10 list. Next, we'll examine the safe journey of that bundle of information from the client via the network to the server, while also discussing how they ought to reside on the server-side.

Backend (server-side)

Let's imagine a RESTful API which interfaces a database - that's our backend. In our backend we have logic which facilitates our users' authentication and their access to private information. With regard to that, what is the scope of the logic in our backend? In short, our backend API:

• receives a message from the frontend with the credentials the user has provided, either when registering or when authenticating (e.g. logging in)
• stores said credentials into the database when the users are first registered or want to update their information
• retrieves said credentials from the database when they have to be checked against the ones the users provided

With the above scope in mind, we will attempt a very simple threat modelling excersise, in order to analyse our threat vectors and our mitigation techniques.

Firstly, we mentioned that the client and the server (aka frontend and backend) exchange of messages. This needs to happen securely over HTTPS (using TLS). To elaborate on this concept, we are including a digitally signed certificate (either self-signed or provided by a third-party certificate authority), which is basically a pair of public and private keys that are used to encrypt our communication. In addition, certificates do expire at some point, therefore we need to make they are always up-to-date. Furthermore, our server should implement the latest TLS version for better security. Many cloud providers today take up this workload and offer a layer of abstraction that we can configure ourselves.

Apart from encrypting our messages to and from the server (data in transit), we need to consider how credentials are best stored (data at rest) so that not anyone with access to our database can read them. Given this requirement, a strong hashing algorithm in combination with a strong salt would make passwords secure. Keep in mind that hashing is a one-way operation. This means we won't be able to figure out the original password just from the hash we generated in our database. However, this is not entirely sufficient, because attackers can pre-process multiple passwords with said hashing algorithm and generate so-called "rainbow tables" which can match our hashed passwords from the database. That is why an additional string called "salt" fed into our hashing algorithm will prevent this from happening.

Now, if we would query the internet for the "top 10 most used passwords", we soon realise that our users are naturally inclined to choose simple strings for their passwords (e.g. "123456" or simply "password"). This is great news for potential attackers and bad news for us and our users. Therefore, the logic which handles registration (i.e. account creation) needs to prevent that. Enforcing restrictions on what makes up a password is essential to achieve better security. Here's a list of restrictions to consider:

• Setup a minimum length. Length is one of the most important security properties of a password against guessing (brute-force attacks). Currently, 8 characters seems to be a good minimum.
• Disallow sequences (e.g. "123456"). Most of them you'll see in the top 10 most used passwords list.
• Enforce the use of a mix of alphabetic, numeric and special characters.
• Disallow the use of the username or email as the password itself as well.

In addition, we need to properly setup our authentication middleware for the routes and resources we want to protect on the backend. An authentication middleware is logic that accepts user credentials (in the form of a token), which are passed with the initial request, usually in the headers, and checks them for validity.

Last but not least, we want to avoid hinting toward the existence or origin of our secrets. In a practical sense, whenever we validate a user's provided credentials as part of the login process, we shouldn't provide accurate feedback for failed attempts. For example, if their email is registered but their provided password is incorrect, the message from the backend shouldn't read "incorrect password for user", rather "username or password is incorrect". The former message reveals the existence of the account in our system, and an attacker could employ other ways of acquiring said user's password.

Frontend (client)

Here we assume a web application that runs on the browser and therefore interfaces our users. Through which, our users register, login and view the private information stored on our backend.

As we've already discussed earlier, the client should send messages to the server securely over HTTPS. This will make man-in-the-middle attacks extremely difficult. Naturally, we are inclined to trust the client-side less as our web app runs on various browsers and hardware we have no control over. Therefore, a good first principle here is to deliver only the information needed to the frontend from the backend.

In our example, as our backend doesn't implement sessions (stateless), our frontend needs to store a token after the user has logged in as proof of their identity. This token will be passed on with every request to a protected or private resource. For this, we can use JSON web tokens (JWT); they are cryptographically signed data in JSON format which can contain information about the user (claims). Beyond JWTs, there are Simple Web Tokens (SWT) and Security Assertion Markup Language Tokens (SAML), the pros and cons of each of which I'll leave to your own research. As you can imagine, the token is a powerful piece of data that, quite literally, "opens doors" to our backend. Due to that fact, here's a summary of very important points on how to structure, sign and use a JWT token:

• choose whether to sign or encrypt your token; the former verifies the integrity of claims and the latter hides those claims
• choose the appropriate encryption algorithm (symmetric or asymmetric) for signing or encrypting the token. Asymmetric is more secure
• if you don't encrypt your token, do not put sensitive information about the user
• use a short-lived (in the span of minutes) access token in combination with a longer-lived revocable refresh token
• set and validate issuer (iss) and audience (aud) claims

How do we store tokens on the client-side, however? And how do we make sure they don't fall into the wrong hands? The latter question is often the main reason many in the industry speak against using tokens for sessions. However, given that we chose to implement JWT token session management, we have the following options for storing our tokens:

• in the browser's cookies
• in the browser's local storage
• in the browser's session storage
• in the memory of our frontend application (e.g. in a variable)

Regarding cookies, the benefit is that they are sent with each request, meaning we don't need to write logic for setting authorization headers. We can make our cookies more secure by setting the httpOnly and Secure attributes. If you want to read more about restricting access to cookies, check out the mdn docs here. As far as local and session storages are concerned, the former is more permanent than the latter; after the user closes the tab with our application the session storage is wiped. Of course, in both cases data can only be accessed from applications on our unique domain but the ephemeral nature of session storage qualifies it as a slightly more secure option. Storing our tokens in the memory of our application is the most secure but the most inconvenient as well, because the memory is not shared across different tabs. Keep in mind that all options we mentioned thus far are susceptible to Cross-Site Scripting (XSS) attacks.

Application secrets

Users are not the only ones with credentials which need to be kept secret. In the world of distributed applications and services, those very applications need to be able to identify and trust one another. Similar to the previous chapter, how application secrets are handled depends on where our application code is running, whether on the side of client or the server.

Use case

Let's think of the following scenario: you wish to build a web application that displays up-to-date prices of your favorite cryptocoins. Naturally, you are thinking of a way to fetch that information from somewhere, reliably and fast. Your search leads to some big crypto platforms that offer APIs for developers. This enables you to write code which calls certain endpoints that fetch the market price of your selected cryptocoins in return. However, to do this, the platform requires you to create an account with them and to register your application. In exchange, you get an API key with which you can authenticate your application firing the requests. In this way, the platform can identify your application and it will allow you to make requests to its protected API resources.

Now, the question is, how do you safely store and use your API key, so that no other developer can just copy and use it on their own application? Can this be done when your code runs only on the client-side? Or is a backend necessary? Let's discuss.

Handling API keys on the source code

Whether a frontend or backend application, you are most likely using git for version control. You may or may not have a public code repository. Either way, it's best not to hardcode your API keys. If you are working in a team, perhaps not everyone should be able to access those secrets. In case you have a public repository on Github, for example, you wouldn't want anyone to be able to read and use that API key. Actually, Github has a security scanning mechanism for preventing exactly that: the accidental commiting of secrets. In essence, application secrets do not belong inside the code. Instead, you should decouple code and secrets by using environment variables. By extend, you should make sure your environment variables are not checked into your source code.

Handling API keys on the frontend

Spoiler alert: on the frontend, almost everything is visible. You cannot and should not store API keys there. Even if you set environment variables safely on the side of your host/provider, these variables are not safe from prying eyes, as they are bundled up with the rest of the code delivered to the client. Afterwards, they are easily found when inspecting code via the browser's developer console. Well, how can we remedy this situation? How can we safely run code on the front-end that holds keys to APIs?

Implementing a backend proxy

Implementing a backend that acts as a proxy to your request from the frontend to the crypto platform is much safer. This means that whoever wants to get the most up-to-date prices of the cryptocurrency (remember our use case?) needs to use our frontend. You might argue, however, that anyone with access to your frontend can see the network requests being fired and try to replicate them from a different application they created. In this case, we can think of better securing your application by:

• enabling CORS (implemented only by the browser) with strict rules. For example, if your API and frontend are on the same domain, you browser can restrict calls to your backend based on the origin of the network call. In this case, the origin should always be your frontend.
• implementing rate-limiting to your API. This is basically an artificial bottleneck to how fast your backend can serve requests. This is useful to make sure that people cannot spam you with API calls.
• requiring users to register with your backend. This gives you more control over who can and cannot use your application. As for safely storing their credentials, you can read more in the previous section of this article.

A dreaded scenario would be when a bad actor takes hold of your API keys and rakes up cloud consumption costs at your expense.

Closing thoughts

We had a look over some ideas on how we can best protect our secrets and, by extend, the data of our users. These practices mostly apply on the application and network layers in a given client-server architecture. However, secrets can leak from a stolen developer's unencrypted laptop SSD/HDD or from a poorly secured server. Something worth noting is, if you are developing applications in a team with multiple environments, you should make sure you have different keys for each them: development, staging and production.

Beyond that, we need to be able to tell when a secret has been compromised as revealed by our application's logging and auditing. Part of being good at keeping secrets is having an action plan in case any of your secrets are compromised. All in all, secrets are best kept in secure and closely monitored environments.