Blog

Developing music apps, OneMusicAPI announcements and more.

Amazon S3 anonymous upload

onemusicapi.com uses S3 to cache images that are loaded from upstream data sources and sent to clients. The bucket used for these images allowed anonymous upload, but this was never a security issue because files were only uploaded from EC2 instances that had a suitable EC2 IAM role.

We created an EC2 instance that didn’t have any EC2 IAM roles, and used it to add some images to the cache. After the process was complete (which took several days), we found that the added images were not accessible, and found that it was due to permissions errors from S3. Further investigation in the AWS console confirmed this, and also showed that the object ACLs weren’t accessible.

Read more...

Posted on 07 June 2021.

Understanding the JVM compiled code cache method listing

We’ve recently been performing some memory optimisations on OneMusicAPI to improve the performance of the service.

One aspect we explored was the memory use of cached code. We use a lot of Scala in the codebase and were concerned that many classes were being generated, cached, and using up memory space.

So we broke out the tools and dived into some lesser-known Java tools to analyse how much memory is being consumed in the code cache.

Read more...

Posted on 13 January 2021.

A PHP package for OneMusicAPI

Currently, OneMusicAPI only has a REST-style HTTP interface. We're now releasing an SDK for PHP which makes performing OneMusicAPI queries and lookups much easier.

How is it easier? Well, before we just exposed a HTTP interface, to which you could send query parameters and the like and receive JSON or images in response. Using the SDK you don't need to know worry about anything at the HTTP level and can just concentrate on using the service in your language of choice.

Read more...

Posted on 14 October 2020.

Fuzzy searching integer ranges in PostgreSQL

onemusicapi.com allows searching for releases by passing in track positions and durations. The search allows the track durations to be off by a couple of seconds, (TODO figure out the reasons for variation), so we need a fast method for performing fuzzy search on the track durations.

Another requirement that makes things more complicated is that not all the input tracks need to match; a subset of the tracks matching (the actual fraction of tracks that need to match depended on other factors) need to be enough for the medium to match. However, this won’t be covered in detail here.

Read more...

Posted on 08 October 2020.

Troubleshooting Tomcat in a memory-lean environment

onemusicapi.com runs on AWS Elastic Beanstalk with a Tomcat 8.5 platform, using t3.micro instances. t3.micro instances have 1GB of RAM, and Elastic Beanstalk’s monitoring puts instances in a Warning state if memory use exceeds 90% of available memory. We found that about a day or two after an EC2 instance was brought online, it would go into Warning state, due to memory use, even if the service wasn’t actually affected.

TL;DR: We found Tomcat was using more memory than expected, and the following helped:

  • Reducing -Xmx
  • Limiting the code cache size
  • Reducing the number of threads

Read more...

Posted on 01 September 2020.

© 2012-2021 elsten software limited, Unit 4934, PO Box 6945, London, W1A 6US, UK | terms and conditions | privacy policy