Spec-Driven Development at Aryeo

Zero to Launch

When I joined Aryeo in late 2020, the mission was clear: build Aryeo’s first mobile app. After a short discussion with key stakeholders, the engineering team collectively pitched a simple, minimum viable app. The app would expose a subset of Aryeo’s functionality with a focus on real-estate agents and brokerages. Not too hard, right? Well, sort of. There was a catch! Aryeo did not yet have an API, so the would-be app had nothing to consume. Fast forward to today, and we have a fully functioning API and new mobile app called Aryeo Listings. This post outlines how we built the Aryeo API using spec-driven development.

What is Spec-Driven Development?

For many APIs, implementation happens in tandem with burgeoning user requirements. Backend engineers create endpoints as the requirements dictate, but otherwise, there is nothing guiding the development of the API. This can be a workable solution for a short period of time, but quickly an API can spiral out of control. APIs built this way often suffer from code duplication, inconsistent data formats, drifting expectations, and a terrible developer experience for downstream frontend engineers. These problems are compounded when dealing with mobile apps because of the likelihood of mobile users running different versions of an app which all make separate assumptions about the current state of the API.

Spec-driven development takes a different approach. Instead of implementing endpoints, responses, and requests in an ad-hoc manner, each aspect of the API is carefully planned ahead of implementation. Before an engineer writes a single line of code, the details of the API are captured and described in a specification, or “spec” — and sometimes multiple specs. These specs then serve as the de-facto source of truth for backend and frontend engineers as they start their work.

Specs can be constructed in a number of different ways, but for our purposes, we chose to build our spec using Stoplight Studio and the Open API Specification V3. Now for the first step.

Sketching Flows

Like many great ideas, we started our spec-driven process by turning to pencil and paper. Using the same rehashed conversations from our mobile app planning, we sketched a handful of user flows that we foresaw in the mobile app. These sketches formed part of the mobile app’s project pitch (à la Shape Up), and were extremely useful in synthesizing the endpoints and data needed from the eventual API.

One key to this step is maintaining proper fidelity in the sketches. You don’t need to be an artist or use an extreme level of detail for this step. Instead, just focus on the big ideas and data abstractions. Later, once pieces of the API are implemented, your pencil sketches can give way to more permanent ink.

Sketch to Spec

Going from sketch to spec is an exercise in mapping visuals to requests, responses, and endpoints. For example, one of our sketches referenced a screen showing the details of a real-estate listing. That screen needed to include the listing’s address and sections for uploaded images, videos, and other media. From these elements, we imagined a response payload for a listing:

```json

{

        "id": {integer},

        "address": {

                "line_1": {string},

                "line_2": {string},

        },

        "images": [

                {

                       "id": {integer},

                       "thumbnail_url": {string}

                }

        ],

        "videos": [

                {

                       "id": {integer}

                       "thumbnail_url": {string}

                }

        ]

}

```

This JSON-like structure could then be further broken into segments, for each JSON sub-object. Using Stoplight Studio, the primary object and each sub-object mapped nicely to model specs:

‍

In a similar fashion, we imagined an endpoint responding with the details about a listing. To get this data, a GET endpoint like `/listings/{id}` could exist. And again, with the power of Stoplight Studio, specifying the endpoint and response objects is a cinch:

‍

As we reviewed each sketch, we are able to map out nearly a dozen models and endpoints for the API. This included POST and DELETE endpoints too, but we won’t show those examples for brevity. However, if you’d like a more robust look at how to create endpoints and models using specs, we highly recommend checking out Spotlight’s documentation on starting a new API design.

Spec Tools

At this point, we had scaffolded a spec, but it was only as useful as that aging documentation in your company’s internal wiki — you know what I’m talking about! So, how did we avoid this becoming a fruitless endeavor? I’m glad you asked. We introduced two key tools to validate and verify the spec and its eventual concrete implementation. 

Spectral

Buried within Stoplight Studio, you may notice a tool called Spectral. Spectral is a powerful JS-based linting tool that can be used to enforce standards across all of your specs — whether they exist in Stoplight Studio or not. For our specs, we created a custom ruleset by extending the standard ruleset defined for Open API Specification V3 (oas3). Here are a few of our custom rules:

• All models must provide a description
• All model properties must provide a type, description, and example
• All string values must defined a minimum and maximum length
• All property values must be lowercase and snake-cased
• All date properties must be specified in ISO 8601 format

When editing model or API specs in Stoplight Studio, you get automatic feedback about any violated rules:

You can also run Spectral from the command-line — something we utilize heavily as part of our continuous integration and deployment (CI/CD) process. More on this later.

```bash

$> spectral lint models/Video.v1.json 

JSON Schema (loose) detected



models/Video.v1.json

  7:10  warning  model-properties-snake-case       Properties should be lower snake_case.                                  properties.ID

 12:13  warning  model-properties-snake-case       Properties should be lower snake_case.                                  properties.TITLE

 19:21  warning  model-properties-description      Property `type` and `description` must be present and non-empty.        properties.THUMBNAIL_URL

 19:21  warning  model-properties-snake-case       Properties should be lower snake_case.                                  properties.THUMBNAIL_URL

 26:20  warning  model-properties-description      Property `type` and `description` must be present and non-empty.        properties.PLAYBACK_URL

 26:20  warning  model-properties-snake-case       Properties should be lower snake_case.                                  properties.PLAYBACK_URL

 26:20  warning  model-properties-strings-min-max  String properties must specify a `minLength` (0 or 1) and `maxLength`.  properties.PLAYBACK_URL



✖ 7 problems (0 errors, 7 warnings, 0 infos, 0 hints)

```

Spectator

The second tool, and perhaps the most important for enforcing spec-driven development, is Spectator. This nifty PHP package consumes an API spec and allows you to write tests against it. Since our backend is implemented with Laravel, this package gives us the ability to ensure all our Laravel controllers and resources are written against the spec. The package’s README summarizes this well:

> Write [contract] tests that verify your API spec doesn’t drift from your implementation…While functional tests are ensuring that your request validation, controller behavior, events, responses, etc. all behave the way you expect when people interact with your API, contract tests are ensuring that requests and responses are spec-compliant, and that’s it.

Hot Tip

If you’d like to adopt a tool like Spectator, one difficulty you may encounter is the need to use a single spec file instead of many spec files. If you’ve been building separate model and API specs in Stoplight Studio, then this will apply to you. To merge all your specs into a single spec, Stoplight Studio provides a de-referenced export feature for your API spec. This causes all model references (refs), like the ones used to define an endpoint’s response or request body, to be replaced with the entire contents of the corresponding model spec. If you’re not using Stoplight Studio, don’t worry, similar functionality is provided by command-line tools like swagger-cli.

Once you have a fully de-referenced spec, then you can begin using Spectator to write contract tests. Here’s an example of one of the tests we wrote for our `GET /listings/{id}` endpoint:

```php

class ApiListingControllerTest extends TestCase

{

    use RefreshDatabase;



    public function setUp(): void

    {

        parent::setUp();



        Spectator::using('Aryeo.v1.json');

    }



    public function testGetListingResponse()

    {

        // listing object is stubbed

        $this

            ->getJson(route('api.listings.show', $listing->id))

            ->assertValidRequest()

            ->assertValidResponse(200);

    }

}

```



```bash

$> ./vendor/bin/phpunit --filter ApiListingControllerTest::testGetListingResponse

PHPUnit 8.5.14 by Sebastian Bergmann and contributors.



. E                                                                  2 / 2 (100%) 



Time: 3.34 seconds, Memory: 82.51 MB



There was 1 error:



1) Tests\Feature\Api\Listing\Spec\ApiListingControllerTest::testGetListingResponse

ErrorException: get-listings-id json response field data does not match the spec: [ required: {"missing":"address"} ]

Failed asserting that true is false.



./vendor/laravel/framework/src/Illuminate/Support/Traits/Macroable.php:114

./vendor/laravel/framework/src/Illuminate/Testing/TestResponse.php:1324

./vendor/hotmeteor/spectator/src/Assertions.php:83

./vendor/laravel/framework/src/Illuminate/Support/Traits/Macroable.php:114

./vendor/laravel/framework/src/Illuminate/Testing/TestResponse.php:1324

./tests/Feature/Api/Listing/Spec/ApiListingControllerTest.php:177



ERRORS!

Tests: 2, Assertions: 6, Errors: 1.

```



```bash

$> ./vendor/bin/phpunit --filter ApiListingControllerTest::testGetListingResponse                    

PHPUnit 8.5.14 by Sebastian Bergmann and contributors.



. .                                                                  2 / 2 (100%) 



Time: 4.22 seconds, Memory: 82.51 MB



OK (2 tests, 7 assertions)

```

These tests can be run manually, but the real power is running them automatically as part of CI/CD. That’s the next step.

Automatic Validation

After identifying the right tools, we were ready to make them part of an automated workflow. Instead of manually using Spectral and Spectator, we wanted to include them in our existing CI/CD workflow. For Spectral, we created a custom GitHub workflows:

```yaml

name: Lint Specs



on:

  pull_request:

    branches: [develop]



jobs:

  lint-specs:

    runs-on: ubuntu-latest

    steps:

      - name: Check out api repo

        uses: actions/checkout@v2



      - name: Setup node

        uses: actions/setup-node@v1

        with:

          node-version: '12'



      - name: Install node packages

        run: npm install



      - name: Install spectral

        run: npm install -g @stoplight/[email protected]



      - name: Lint spec

        run: spectral lint --ruleset spec/.spectral.json --resolver spec/skip-resolver.js 'spec/reference/*'



      - name: Lint spec models

        run: spectral lint --fail-severity warn --ruleset spec/.spectral.json 'spec/models/*'

```

And for Spectator, no change was required. All Spectator tests could be automatically included as part of our existing PHP unit tests (`./vendor/bin/phpunit`)!

Asynchronous Development

With all these pieces in-place, our backend engineers were able to confidently begin API development. In our Laravel-based world, that meant building API controllers and response/request objects while using the spec as a guide; but, even if we weren’t using Laravel, the process would have looked the same for any other stack. The key was having the right pieces in place to enforce the spec as the source of truth — this is also what gives this step the ability to be somewhat asynchronous. For us, the frontend (mobile) engineers defined the initial spec, the backend engineers implemented it, and occasionally collaboration was necessary to smooth out parts of the spec that needed to change.Here’s an example of a the development feedback loop possible with Spectator:

Release!

Ah, the final step. Once our backend engineers finished their API implementation and it passed all testing, it was safely merged into master. But, we didn’t stop there. We wrote a few more GitHub workflows to create handy resources from the spec:

```yaml

name: Generate Postman Collection



on:

  push:

    branches: [ master ]



jobs:

  generate-postman:

    env:

      REPOSITORY: {repository}



    runs-on: ubuntu-latest



    steps:

      - name: Check out api repo

        uses: actions/checkout@v2



      - name: Check out postman repo

        uses: actions/checkout@v2

        with:

          repository: ${{ env.REPOSITORY }}

          path: postman

          token: ${{ secrets.TOKEN }}



      - name: Setup node

        uses: actions/setup-node@v1

        with:

          node-version: '12'



      - name: Install openapi to postman tool

        run: npm install -g openapi-to-postmanv2



      - name: Install swagger cli

        run: docker pull jeanberu/swagger-cli



      - name: Replace spec $refs with definitions

        run: docker run --rm -v ${PWD}:/local jeanberu/swagger-cli swagger-cli bundle -r -o /local/spec/out/Aryeo-deref.v1.json /local/spec/reference/Aryeo.v1.json



      - name: Create postman collection

        run: openapi2postmanv2 -s spec/out/Aryeo-deref.v1.json -o postman/aryeo-postman.json -O requestParametersResolution=Example,exampleParametersResolution=Example,folderStrategy=Tags



      - name: Commit files

        continue-on-error: true

        run: |

          cd postman

          git config --local user.email "[email protected]"

          git config --local user.name "GitHub Action"

          git add .

          git commit -m "update postman collection"



      - name: Push changes to postman repo

        uses: ad-m/github-push-action@master

        with:

          directory: ./postman

          github_token: ${{ secrets.TOKEN }}

          repository: ${{ env.REPOSITORY }}

```



```bash

name: Generate SDKs



on:

  push:

    branches: [ master ]



jobs:

  build_swift_client:

    env:

      LANGUAGE: swift5

      REPOSITORY: {repository}

      PROJECT_NAME: AryeoModels



    runs-on: ubuntu-latest



    steps:

      - name: Check out api repo

        uses: actions/checkout@v2



      - name: Check out sdk repo

        uses: actions/checkout@v2

        with:

          repository: ${{ env.REPOSITORY }}

          path: sdk

          token: ${{ secrets.TOKEN }}



      - name: Install swagger cli

        run: docker pull jeanberu/swagger-cli



      - name: Replace spec $refs with definitions

        run: docker run --rm -v ${PWD}:/local jeanberu/swagger-cli swagger-cli bundle -r -o /local/spec/out/Aryeo-deref.v1.json /local/spec/reference/Aryeo.v1.json



      - name: Install openapi generator cli

        run: docker pull openapitools/openapi-generator-cli



      - name: Clear the sdk repo

        run: |

          sudo rm -rf sdk/*



      - name: Generate swift sdk

        run: docker run --rm -v "${PWD}:/local" openapitools/openapi-generator-cli generate -t /local/spec/codegen/templates/swift5 -i /local/spec/out/Aryeo-deref.v1.json -g swift5 --additional-properties projectName=AryeoModels,readonlyProperties=true --skip-validate-spec -o /local/sdk



      - name: Remove extraneous files

        run: |

          cd sdk

          sudo rm -rf .openapi-generator

          sudo rm -rf docs

          sudo rm -rf ${{ env.PROJECT_NAME }}/Classes/OpenAPIs/APIs

          sudo rm .openapi-generator-ignore

          sudo rm ${{ env.PROJECT_NAME }}.podspec

          sudo rm Cartfile

          sudo rm git_push.sh

          sudo rm project.yml

          sudo rm README.md

          sudo touch README.md

          sudo chmod 777 README.md

          sudo echo "# Aryeo SDK" > README.md



      - name: Commit files

        continue-on-error: true

        run: |

          cd sdk

          git config --local user.email "[email protected]"

          git config --local user.name "GitHub Action"

          git add .

          git commit -m "update sdk"



      - name: Push changes to sdk repo

        uses: ad-m/github-push-action@master

        with:

          directory: ./sdk

          github_token: ${{ secrets.TOKEN }}

          repository: ${{ env.REPOSITORY }}

```

These workflows produce our API’s documentation, an interactive Postman collection, and even an SDK for the Swift/mobile app. And, as the API is updated in the future, we’ll continue to get these byproducts for free! Not bad for a few lines of code.

Staying Spec-Driven

At first, adopting spec-driven development felt a little clunky. We had to set up numerous tools and adjust the way we worked as an engineering team. However, the proof is in the results. With spec-driven development, we’ve been able to iterate on our API while keeping backend and frontend engineers in sync. All of our API decisions begin with our specs and flow into our implementations. This has reduced code duplication, unified data formats, and made for a really pleasant developer experience. We feel like spec-driven development is the future, and we hope this post has given you a starting point if you’re considering a switch!