Using Dreamfactory 2.x as REST API with Docker

For a new project I’m working on as Systems and Platform Architect, we were looking for an comparably easy and fast to implement, yet scalable and reliable way to build a RESTful API for various services we need to expose to mobile device users via iOS/Android Apps aswell as internal services.

Usually you have several MySQL/NoSQL Databases or similar which you need to query. We even have to deal with a 3rd party SOAP service. So wrapping all this into CRUD APIs and having ACLs for user roles etc would be quite a big story.

Fortunately there exists a great tool/framework which does alot of work for you nearly automagically. Its called Dreamfactory

Read on after the jump how Dreamfactory works and how we integrated it as BaaS into our docker-based PaaS.

Overview

So to give you a basic idea of what Dreamfactory does, have a look at this infographic and check the features page I linked above.

DreamFactory Infographic

DreamFactory Infographic

The auto-generation of APIs is really helpful and maps databases, tables, fields, file storages, 3rd party services etc to REST endpoints and it also does that for SOAP services based on their WSDL. Which makes working with SOAP services way comfortable  than it usually is.

You can also create Custom Scripting Services (in PHP, V8Js, Node.js, Python) for adding some more custom logic. Theres also the possibility to add pre/post-process scripts for any API endpoint so you can hook into any auto-generated API easily.

It uses the OpenAPI Specification formerly known as swagger.io , and auto-generates API Docs for you so you can try the APIs instantly without writing a single line of code.
For your custom services you have to write the swagger definition yourself but its quite simple and you can auto-generate client SDKs for dozens of programming languages out of them. I’ll talk about this some more in the upcoming articles.

Running Dreamfactory with Docker

As the platform I’m working on should be immutable and is based on and built around microservices and containers, the API should also fit into this picture. I was happy to find https://github.com/dreamfactorysoftware/df-docker however had to find out it had a few bugs which rendered the image useless. The current image available on DockerHub has the same issues. DockerHub image should be fine now.

I fixed up the bugs and made some enhancements for using the same and untouched Docker image in different environments aswell as using it with orchestrators like docker-compose, Kubernetes, Mesos, AWS ECS..
My PRs got merged into their develop master branch, so you can also clone a working copy from there. I’ll update this post when there are significant changes or new release builds get available.

The included docker-compose.yml will run DF with a linked MySQL and Redis container, however we needed to connect it to external services, in our scenario AWS RDS and AWS ElastiCache.
Here is an example env_file for docker-compose but you can aswell supply those variables via any other orchestrator or directly via docker run … -e DB_HOST=… , see df-docker/README.md for more detailed instructions and command examples.

DB_HOST=foobar.eu-central-1.rds.amazonaws.com
DB_USERNAME=df_admin
DB_PASSWORD=p4ssw0rD
DB_DATABASE=df2
REDIS_HOST=foobar.cache.amazonaws.com
REDIS_DATABASE=0
APP_KEY=SUPER_SECRET_KEY

You can now scale the containers horizontally and run them in a load-balancer pool. The containers are immutable, so you can just throw them away and create a new one. The data is living in the DB unless you create a local SQLite DB as a service or a local “app”. In that case you would have to make paths storage/app and storage/databases a VOLUME.

I came across one culprit tho which wasnt straight-forward and not exactly clear in the documentation:

The APP_KEY is used to hash certain internal things and if you run DF with a new APP_KEY against an existing DF database, it wont work properly, or in other words, you have to set the same APP_KEY for all DF containers.
Unfortunately the df-docker Dockerfile used to generate a new APP_KEY on every image build.
But now you can supply an existing APP_KEY and DF will reuse it instead of generating a new one.

This should be it for my first article about Dreamfactory. I’ll write some more and show you how to actually use it as BaaS with thousands of mobile users.

Update: Dreamfactory also has a related article on their blog now http://blog.dreamfactory.com/scaling-dreamfactory-with-docker

Update 2: Also see my followup article Orchestrating Dreamfactory with docker-compose and a LoadBalancer