ASC 2022 - API Specifications Conference

OpenAPI and its associated tool ecosystem have made our lives easier by providing automated mechanisms for API design, documentation, mocking or even code generation, among others. But there’s more to it: an OpenAPI spec can also bootstrap automated testing of APIs. In this talk, I will review some approaches that leverage the OpenAPI Specification to automatically generate test cases, requiring little or no human intervention at all. Then, I will report on my team’s experience on the deployment of an automated testing ecosystem for OpenAPI-described RESTful APIs: over the course of 15 days non-stop, we continuously and automatically generated and executed over one million test cases for 13 popular APIs, including those from YouTube, Stripe and Yelp, among others. We detected over 200 bugs in a highly automated fashion, some of which have been confirmed by API developers, leading to bug fixes and documentation updates in the APIs of Amadeus and YouTube. I will finish my talk discussing the types of bugs that can be found by automated testing techniques, as well as the current limitations of these techniques to make them fully applicable at scale.