Intro to Tokens

So now we know how to securely store our user’s passwords, and check a user’s password to see if it’s valid. But we’re still missing a piece. When you successfully log into a website, something changes. Somehow you have access to pages you couldn’t see before.

What actually happens after I type my username and password on the page? How does the server distinguish that I’m someone who has permission, versus all the other people using the site at the same time?

As before, we’re going to look at different approaches and explore the pros and cons of each. But first, we need to take a quick detour to understand some basic facts about web servers.

The server forgets

An interesting thing about web servers is that they have to serve a lot of people at once. Requests to a web server can come from anyone at any time, and the server has to make sure that every person sees exactly what they are supposed to, no matter how many other people are using it at the same time.

Because of this, web servers are designed to forget. Every time someone connects to the server to ask for data, the server starts from ground zero. “Who are you? What are you trying to access? Are you allowed to access that?” Starting from ground zero makes it much easier to think about your server’s design, since every request starts from the same place, no matter who made it or what they were doing before. But it does have some consequences - namely, that the server isn’t going to remember who you are from request to request.

So the server won’t remember us. But we expect to type our username and password once, and have the website “remember” us until we log out. So how do we make that happen?

Approach 1: Send the username/password with every request

It might seem easiest just to make your browser send the username and password with every request. You can store some data in your browser, so why not just save the username and password?

This could definitely work. The problem is that storing a password at all is a really bad idea.

In general, you can never trust browsers to do the right thing or be secure. Anything you store in the browser can be accessed by JavaScript running on the page, which opens you up to various attacks. Browsers have a bunch of security measures in place to keep this relatively safe, but it’s better to never store sensitive data at all.

There’s no way to mitigate the problem, either. Encryption wouldn’t help, because you would still have to decrypt the password in the browser before sending it to the server. Storing the username and password in the browser is a fundamentally bad idea.

And not only is it unsafe, but it’s also actually slower. A secure password hash will naturally be slow to verify, and we would be forcing the server to check that hash on every page load, every image, every CSS file, and every script. It’s more work than we actually need to do, and we’ll see why later.

Approach 2: Send a token

Rather than storing and sending the username and password, let’s take a step back. The reason we have usernames and passwords in the first place is to prove to the server who we are. If I provide my username and password, then the server knows it is talking to Ben, and can show me things it knows I have access to.

What we really need to tell the server is our identity. The way we can achieve that more safely is with a token.

A metaphor

Let’s take a trip to Nickelodeon Universe.

At Nickelodeon Universe, you need a wristband to ride the rides. Each wristband has a certain number of ride points on it. To make this work, each wristband has a barcode that they scan before you enter the ride - if you’re out of points, no ride for you!

This wristband approach might seem pretty obvious, but let’s think about some other ways they could handle ride payment at Nickelodeon Universe.

  • Just pay money at the ride: This would work, but it would be more cumbersome for guests, and it would be more work for the ride operators too. It’s way faster to scan a wristband than actually handle someone’s money!
  • Give each person a wristband with no barcode: This would verify that each person did pay for a wristband, but they wouldn’t know how many points were left for each person. Put another way, they wouldn’t be able to distinguish between individual guests.

There are a few things that make the Nickelodeon Universe wristbands work:

  1. They are fast to use.
  2. They uniquely identity a person, thanks to the barcode.
  3. They can’t be faked, because they are physical objects. (Well, they would be hard to fake.)

When you pay for a wristband at Nickelodeon Universe, you exchange money for a special object that identifies you, and that they can trust. Our “tokens” will do the same thing.

How to make a token

There are lots of ways to make a good token, but it has to hit the same three points as above - it has to be fast to use, it has to identify a person, and it has to be hard to fake.

A very simple approach, and the one we’re going to use, is to take some basic information about the user, and encrypt it. Encrypting it will make it so that only the server will be able to read the information. No one else, including the user’s browser, knows the key used to decrypt it.

For example, we might use the following simple JSON object to identify a person:

{
    "username": "benv",
    "login_time": "2018-08-09 11:34:12"
}

We can include any information we like. In this example, I’ve provided the username, so we can identify the user, and the time that they logged in, so we could, for example, tell the user how long they have been logged in. (If you don’t know what JSON is, check out Appendix B!)

We can then take that data and encrypt it using our algorithm of choice. In this example, I’m going to use 256-bit AES encryption with a key of “secretkey”. Encrypting the JSON above gives us the following string:

gAkDWU4wJ+X05fi3T+6fPC2f6w/alkGF2EnxL2oIW+G0/BR0M/QT4yp4N7xIQHtWUZ6xKB7dbpne24BN1v7JVkMiU7reQYuqoL2Q0JJ60v8=

This value can now be used as a token! The browser can send this to the server with every request, and the server can decrypt it to get at the information it contains.

Let’s see how well it meets our criteria:

  1. Fast to use: Pretty good! Decrypting with AES is a bit faster than checking the user’s password.
  2. Uniquely identifies a user: Yes, because it stores the username.
  3. Hard to fake: Yes, because if you mess with it, or try to fake it, the server won’t be able to decrypt it any more. Only the server knows the key we used to encrypt it!

This is a value that a browser can safely store without fear of password theft. Now it’s time to see how we use this token design in our system.

Putting it all together

Now that we know a way to make a token, let’s see how the whole process would fit together.

Step 1: The browser logs in

The user types their username and password into a form. The form submits its results to the server at a URL we choose - maybe /login or something.

Step 2: The server verifies the password

The server checks the username and password against the database. If it succeeds, then we need to make a token to give back to the browser.

Step 3: The server makes a token

The server builds the necessary information about the user:

{
    "username": "<their username>",
    "login_time": "<the current time>"
}

and encrypts it using some secret value. We now have a token that we can give back to the browser.

Step 4: The server responds with the token

The server takes its usual response and adds the following header:

Set-Cookie: <cookie name>=<the user's token>

When the browser sees this header, it will automatically set a cookie with the new token. That will make the browser include that token in all future requests it makes to the server. (See Appendix A to learn more about headers, and Appendix C to learn more about cookies.)

Step 5: Browse around!

With that cookie in place, the login process is complete! The browser will automatically send the token to the server on each request, so the server can verify it and make sure the user is allowed to see whatever they’ve requested.

Final thoughts

At this point you might be wondering - is this really better than just a username and password? It’s certainly more work, and it sort of behaves like a password as far as the server’s concerned.

Here are a few advantages tokens have:

  • Tokens don’t expose the user’s password. Imagine some future problem with Chrome results in everybody’s cookies being stolen. If the user’s password were stored in a cookie, then an attacker could easily grab that password and use it to log in in the future - or worse, log into other websites where that user had used the same password. Tokens are specific to a particular server, and are not passwords, so it’s not as dangerous if they get stolen.
  • Tokens can store any information you want. Many systems will put more information in their tokens than I used in my example. For example, many websites will put an expiration date in their tokens so that users will be forced to log in again after a certain amount of time. Tokens can store anything, but a password is always just a password.
  • Tokens are more trustworthy. A good token design will be encrypted or signed in such a way that they could only have come from the real server. It’s much harder to fake a token than to guess a password.

Making a login system secure is tricky, and it’s very easy to make naive mistakes. If you’re ever doing login stuff for a more serious website, make sure to seek out some expert advice. But for us, this token design is good enough.

Appendix A: Structure of an HTTP Request

Behind the scenes, all the information your browser gets from the internet is done by an HTTP request. Every HTML page, image, stylesheet, and JavaScript file is fetched using the same format.

To kick things off, before we dive into the details, let’s look at an example:

POST https://fightingcalculators.org/login HTTP/1.1
Content-Type: application/json
Accept: application/json

{"username":"benv","password":"My_Passw0rd!"}

To explain the format, we’ll start with the overall structure and get more specific. All the terminology comes straight from the official spec.

Overall Structure

Here is the overall structure of an HTTP request. You can see how this structure roughly matches the example above. We’ll dig into the specifics of each section below.

request-line
header-field (zero or more)

message-body (optional)

request-line

POST https://fightingcalculators.org/login HTTP/1.1

The request line contains the method, the URL, and the version. The URL is pretty obvious. The method, though, might not be.

HTTP requests have several methods to choose from, each of which carries a different meaning. For example, GET is used for simply retrieving information, like loading an HTML page or an image. POST is used for sending data to a server, usually to take some action or create a resource on the server. There’s many more to choose from, but what they do is mostly arbitrary in practice.

The version is important simply because there have been different versions of HTTP over time. The version we’re looking at here is version 1.1 - hence the HTTP/1.1 in the example. In reality, our browsers and servers might be using HTTP/2, which is a newer version that is more efficient but harder for humans to read.

header-field

Content-Type: application/json
Accept: application/json

HTTP headers are used to provide extra metadata or information with the request. In the example, the Content-Type header tells the server what kind of data we’re sending it, and the Accept header is telling us what kind of information we’d like in return. Each header has its own rules and semantics, so there’s no point in listing them here, but MDN is a good resource if you’d like to learn more about them.

message-body

{"username":"benv","password":"My_Passw0rd!"}

The message body can be whatever you like! (Or rather, whatever the server will understand.) If you’re sending information to the server, this is where you’ll put it. In this example, I’m sending a username and password in the common JSON format. For GET requests, you won’t provide any message-body.

Appendix B: What is JSON?

JSON stands for JavaScript Object Notation. It’s a common text-based format used to store data.

You can learn all the details about JSON at json.org, and it’s actually quite straightforward, but I’ll summarize the basics here.

A JSON value can be one of the following:

  • A number, which just looks like any old number. For example, 3, 1.5, or -10. (Technically, scientific notation is also allowed, but hardly anyone ever uses it.)
  • A string of text, which must be surrounded by double quotes. For example, "bob".
  • The boolean values true and false.
  • null, which is used to represent the absence of a value.
  • An array, which is just a list of values inside square brackets and separated by commas. For example, [1, 2, 3], [true, "bob", 3], or [null, [1, 2, 3], false]. (Note that you can put arrays inside arrays!)
  • An object, which is a set of name/value pairs. Objects use curly braces and separate the name from the value with a colon. For example, { "my_name": "Ben" }, { "x": 2.5, "y": 3.1 }.

These can be combined in any number of ways. The following example is completely valid JSON, and shows how these things can be combined.

{
    "name": "Ben",
    "years_as_student": [2010, 2011, 2012, 2013, 2014],
    "years_as_mentor": [2015, 2016, 2017, 2018],
    "programming_languages": [
        {
            "name": "Java",
            "knowledge": 7,
            "rating": 3
        },
        {
            "name": "PHP",
            "knowledge": 6,
            "rating": 2
        },
        {
            "name": "Go",
            "knowledge": 9,
            "rating": 9
        }
    ]
}

Appendix C: Cookies

Cookies are a feature that lets your browser store information from page to page. Your web browser automatically includes the values of all cookies in its HTTP requests, which makes cookies the perfect choice for identifying information like tokens.

Furthermore, web servers have the ability to automatically set cookies in a user’s browser. Whenever a server sends back a response, it can include a Set-Cookie header, telling the browser the cookie’s name and value.

You can learn more about cookies from their page on the Mozilla Developer Network.