Gmail and OAuth 2.0

July 15, 2014

At FiveStreet.com, we try to make it as easy as possible for new customers to integrate our application into their existing workflow. One way we do this is by grabbing real-estate leads directly from their Gmail inboxes.

When we first built this feature, Google only provided IMAP access through OAuth 1.0. Since then, Google has deprecated OAuth 1.0 in favor of OAuth 2.0. (And they are dropping support for OAuth 1.0 in April of 2015.)

This means we have to:

Migrate OAuth 1 security tokens to OAuth 2 tokens.
Figure out how to connect to Gmail using OAuth 2 tokens.

Google’s documentation on how to do this are fairly confusing, and – in some places – inaccurate.

Converting OAuth 1.0 credentials to OAuth 2.0 credentials.

The code below takes the OAuth 1.0 token / secret assigned to a user and converts it to an OAuth 2.0 refresh token. You will need to run this conversion process once for every user, and save the refresh token to your database.

The refresh token can then be used to generate a short-lived access token. Note that your OAuth 1.0 credentials will stop working an hour after you use the OAuth 2.0 refresh token.

More information: Migrating from OAuth 1.0 to OAuth 2.0

require 'oauth'
require 'net/http'
require 'json'

# OAuth 1 - Application Key / Secret.
oauth1_consumer_key    = "www.site.com"
oauth1_consumer_secret = "..."

# OAuth 1 - User Token / Secret.
oauth1_token           = "..."
oauth1_secret          = "..."

# OAuth 2 - Application ID / Secret
oauth2_client_id       = "..."
oauth2_client_secret   = "..."

# Migration Parameters.
params = {
  "grant_type"             => "urn:ietf:params:oauth:grant-type:migration:oauth1",
  "client_id"              => oauth2_client_id,
  "client_secret"          => oauth2_client_secret,
  "oauth_signature_method" => "HMAC-SHA1"
}

# Create the consumer object.
consumer = OAuth::Consumer.new(
  oauth1_consumer_key,
  oauth1_consumer_secret,
  :site   => 'https://accounts.google.com',
  :scheme => :header
)

# Create the access token object.
access_token = OAuth::AccessToken.new(consumer, oauth1_token, oauth1_secret)

# Post to the migration URL.
resp = access_token.post(
  "/o/oauth2/token",
  params,
  { 'Content-Type' => 'application/x-www-form-urlencoded' })

if resp.code.to_s != "200"
  # Raise an error.
  raise "#{resp.code} - #{resp.body}"
end

# Now you have a refresh token!
 oauth2_refresh_token = JSON.parse(resp.body)["refresh_token"]

Generate an Access Token

The code below uses the OAuth 2.0 refresh token you just generated to obtain an OAuth 2.0 access token. The access token is what you use to actually authenticate the user.

Note: You may need to change the scope based on your application’s needs.

More information: Using an OAuth 2.0 refresh token

require 'oauth2'

# OAuth 2 - Application ID / Secret
oauth2_client_id       = "..."
oauth2_client_secret   = "..."
oauth2_refresh_token   = "refresh-token-from-above"

# Create the OAuth 2 client.
client = OAuth2::Client.new(
  oauth2_client_id,
  oauth2_client_secret,
  {
    :site      => "https://accounts.google.com",
    :token_url => "/o/oauth2/token",
    :token_method => :post,
    :grant_type => "refresh_token",
    :scope      => "https://www.googleapis.com/auth/userinfo.email https://mail.google.com/"
  })

oauth2_access_token = client.get_token(
  "client_id"     => oauth2_client_id,
  "client_secret" => oauth2_client_secret,
  "refresh_token" => oauth2_refresh_token,
  "grant_type"    => "refresh_token")

Connect to IMAP using XOAUTH2

Now that you have an access token, you can use it to authenticate with Gmail through the SASL XOAUTH2 mechanism.

First, we need to update Net::IMAP to add XOAUTH2 as an authenticator method. According to Google’s documentation, this can be done by base64 encoding a specially formatted string containing the user’s email address and a valid access token.

More information: The SASL XOAUTH2 Mechanism

In practice, base64 encoding the string didn’t work, but providing an unencoded string did. It feels like they might quietly change this one day, so be careful if you use this in production.

require 'net/imap'

class Net::IMAP
  class XOAuth2Authenticator
    def initialize(email_address, access_token)
      @email_address = email_address
      @access_token = access_token
    end

    def process(s)
      # HACK!!! - The docs say that we need to base64 encode the
      # following line; but that doesn't work in practice.
      "user=#{@email_address}\x01auth=Bearer #{@access_token}\x01\x01"
    end
  end

  add_authenticator 'XOAUTH2', XOAuth2Authenticator
end

Now that we’ve added an XOAUTH2 authenticator, connecting to IMAP is simple:

# The user's email address.
email_address = "..."

# Connect to IMAP.
client = Net::IMAP.new("imap.gmail.com", :port => 993, :ssl => true)
client.authenticate('XOAUTH2', email_address, oauth2_access_token.token)
# ...do stuff...
client.disconnect()

That’s all. Hopefully this saves you many, many hours of frustration.

Back