How to download Discogs images in Java, using OAuth

Many software developers were flummoxed at the start of the year when it was announced that Discogs would start requiring OAuth authentication for image downloads. At the same time, a new limit of 1000 image downloads per day, per application, was imposed.

As a Premium API user, we at OneMusicAPI were informed of this change in December, and we immediately made plans on how to cope with this change. We knew that, while initially there wouldn't be much change to our operations, given OneMusicAPI simply returned image URLs sourced from Discogs without actually downloading them, our clients would both run out of their image allocation and find it a pain to rewrite their software to authenticate using OAuth.

The result was the image endpoint and a new images plan. By caching Discogs images, we were soon able to serve up way more than the 1000 image limit to each application. Furthermore, we did the tedious OAuth authentication work ourselves.

Since we started counting, in March, we've served almost 400,000 images from our cache. After an initial slow start (as the cache was populating) we now serve between 100,000 and 200,000 images from the cache each month. That's a lot of developer work saved.

If you can't use the OneMusicAPI image cache (and if not I'd like to know why!), though, how do you write the OAuth code to download an image, using Java?

Discogs image download using OAuth and Java

OAuth is a protocol for authorisation. It defines a series of steps and messages between client and server which importantly including the end user's approval. Discogs uses version 1.0a of OAuth.

There are a number of Java libraries that implement the OAuth protocol. The one I use in this tutorial is oauth-signpost. Go ahead and download it or point your pom.xml toward:

<dependency>
  <groupId>oauth.signpost</groupId>
  <artifactId>signpost-core</artifactId>
  <version>1.2.1.2</version>
  <scope>compile</scope>
</dependency>

While that's downloading, a quick overview of the OAuth process. It consists of a number of steps:

1. Instantiate an OAuthProvider and OAuthConsumer
2. Use the OAuthProvider to retrieve a request token
3. Use the request token to ask the user to approve the login, which gleans a code
4. Use that code to retrieve an access token and pass it to an OAuthConsumer
5. Use the OAuthConsumer to sign subsequent HTTP calls

Instantiate an OAuthProvider and OAuthConsumer

Pretty easy this one. For the former, it's most straightforward to use DefaultOAuthProvider which uses a HttpURLConnection to make the calls to Discogs.

OAuthProvider provider = new DefaultOAuthProvider(
    "http://api.discogs.com/oauth/request_token", "http://api.discogs.com/oauth/access_token",
    "http://www.discogs.com/oauth/authorize");

Those URLs are published by Discogs and can be seen in your Discogs applications page. At this point you should also set the User-Agent:

provider.setRequestHeader("User-Agent", userAgent);

The reason for this is that Discogs is quite picky about User-Agents. I've had requests rejected before which do not specify one.

Now create the OAuthConsumer:

OAuthConsumer consumer = new DefaultOAuthConsumer(consumerKey, consumerSecret);

The consumerKey and consumerSecret are, again, listed on your Discogs applications page.

The OAuthProvider and OAuthConsumer objects can now be used to implement the remainder of the OAuth protocol.

Use the OAuthProvider to retrieve a request token

The next step is to retrieve a request token, in the form of a URL:

String verificationUrl = provider.retrieveRequestToken(consumer, OAuth.OUT_OF_BAND);

This URL is used in the next step, to ask the end-user to approve the authentication

Use the request token to ask the user to approve the login, which gleans a verification code

The verificationUrl should now be used to download a page that can be presented to the user so they can approve the authentication. Once the user does so, a new verification code is generated.

Note that the user has to be logged in; if they are not, Discogs will forward them to the login page, and following a successful login they will be able to approve the authentication between your app and Discogs.

You must now ask the user to copy and paste the verification code for the next step.

Use the verification code to retrieve an access token and pass it to an OAuthConsumer

To retrieve the access token, and thus complete the OAuth authentication process, you can do:

provider.retrieveAccessToken(consumer, verificationCode);

Here, verificationCode is the verification code that the user copy and pasted following their approval.

Use the OAuthConsumer to sign subsequent HTTP calls

The consumer is now ready to sign HTTP calls to Discogs.

HttpURLConnection conn = (HttpURLConnection) url.openConnection();
consumer.sign(conn);
conn.setRequestProperty("User-Agent", userAgent);

You can now use the conn as normal - opening an InputStream to it, returning the response code, or however you want to interact.

And, as ever, remember to set the User-Agent!

These instructions are taking on even more importance following the announcement that, from 15th August, access to the search API (/database/search) will require OAuth authentication. Another reason to use OneMusicAPI ;-)

Thanks to JD Hancock who made the the image above available for sharing.