Understanding the 12-Factor App Methodology

Gavin Terpstra
July 12, 2022

Understanding the 12-Factor App Methodology

At reinteractive we build, maintain and host Ruby on Rails web applications.

Some of the applications we host have over a million visitors a month. Others power entire businesses, are time critical and require the utmost speed of response - any down time or interruptions have significant consequences. Notwithstanding our decade of experience, we draw on the experience gained by following the 12-Factor App methodology for our hosting platform and tools.

What is the 12-Factor App Methodology?

The 12 factors are a methodology that drive decisions behind the coding and architecture of Software as a Service (SaaS) applications. Building with these 12 factors means the application:

  • uses declarative formats for setup automation.
  • has a clean contract with the underlying operating system, offering maximum portability between execution environments;
  • is suitable for deployment on cloud platforms, simplifying server hardware requirements.
  • is built for agility and continuous deployment by creating environments for development, staging and production that share the same DNA.
  • is architected to scale without changing tooling or practices.

We have 12 years of experience building these apps and keeping them performant for our users.

What are the 12 Factor App Design Principles?

The 12 factors are:

Codebase Dependencies Config Backing Services Build, Release, Run Processes Port Binding Concurrency Disposability Dev/Prod Parity Logs Admin processes

1. Codebase

Your application codebase should exist in one location (repository). This is not your production server. The repository is in a separate environment, and managed with a version control system. We primarily use Github for version control, but there are other options, both SaaS and self-hosted.

The codebase is the central data-store for the application. Developers clone copies of the code, make their changes, and submit their changes back to the version control system.

The codebase will contain branches to organise work on the application for major releases, new features, or the work individual developers are doing. From this central code-base these branches can be deployed to staging servers or production servers, and merged into the current version.

2. Dependencies

Today software, more often than not, depends on external code libraries that are included in the project to add functionality. For example, users sending emails from an application call ruby gems like ActionMailer and Mail to properly format and send emails. This allows a developer to leverage the work done by others to properly format an email. The developer need only supply the basic details like recipient, subject and body content – the gem does the rest.

Declaring dependencies is done in Ruby on Rails, by adding the gem to the project’s gemfile. We recommend going a step further and explicitly declaring the version(s) of the dependencies that are tested with your project.

This applies not only to Rails, but to NPM packages for Javascript, Python, PHP, GoLang packages used in any software project.

Popular dependencies are generally well maintained. This maintenance brings equal measures of caution and optimism. Optimism because any bugs or edge cases are rapidly resolved. Caution because newer versions may alter the performance of the methods in use, and break your code.

3. Config

External services, such as web servers, processes, databases, mail servers, Amazon S3 buckets and other API endpoints are just a few of the endpoints a web application requires. The credentials are stored in environment variables such as config/database.yml for Rails.

Another aspect to the config is being able to separately configure workspaces for development, staging and for production servers. Staging and development databases should be different to production servers, likewise payment gateways will have configuration variables that differ between production and development and so forth.

All of these configuration variables reside outside the code repository.

4. Backing Services

Backing services, such as SMTP mail servers, databases, S3 buckets or other storage and other API endpoints are treated as attached resources, even if served locally. These are not hard-coded into the application. The last thing you want in the event of an SMTP server failure is to be searching through the code of your application to find the connections to your SMTP server.

By defining all backing services as attached resources in your config you can rapidly scale or repair your application without the need to push patches to your codebase.

5. Build, Release, Run

The build, release, and run stages are kept separate from each other.

After code is written, it is compiled in the build stage and released. Released code is the only code pushed to runtime. There is no place for editing code in the runtime. Any changes to the runtime code are not replicated back to the build stage or code stage.

The app will have multiple release versions of the code, versioned and stored by your automated deployment processes. if there is a need to roll-back a change, the Ops team can push the last valid release to runtime without having to patch code.

6. Stateless Processes

All processes for the application are stateless and share nothing. Any data that needs to persist is stored in stateful storage service, typically a database and/or cloud storage for files.

The Memory and filesystem of the processes (application instances) can be used as a temporary, single transaction cache. However, for any of the results to persist, these are stored in the database or other storage. A twelve-factor app never assumes anything is cached in the process memory or stored on a local disk. A restart of the process to deploy code, change config or relocate the processes will generally reset all memory and filesystem changes to the release state.

7. Port Binding

Configured services are bound to ports, not hard coded resources. The binding of ports for external services is performed in the execution environment.

HTTP is not the only service that can be exported by port binding. Nearly any kind of server software can be run via a process binding to a port and awaiting incoming requests. Examples include ejabberd (speaking XMPP), and Redis (speaking the Redis protocol).

The port-binding approach means that an app can become the backing service for another app (ie. microservices), by providing the URL to the backing app as a resource handle in the config for the consuming app.

8. Concurrency

Processes in the 12-factor app are most like unix processes running service daemons. Using this model, the app handles diverse workloads by assigning each type of work to a process type. HTTP requests are generally handled by web processes, and long-running background tasks handled by worker processes.

Individual processes can scale internally via multiple threads inside the runtime virtual machine instance. But individual virtual machines have limitations (vertical scaling), so the application must also be able to support processes running on multiple physical or virtual machines (horizontal scaling).

This horizontal scaling is where 12-factor methodology shines. Adding or removing server resources is a simple and reliable operation. Equally, where multiple processes exist for the application, web server resources can be scaled independently of worker processes to balance computing resources based on request volume.

9. Disposability

Application processes are disposable. They can be spun up or wound down rapidly, scaling with application need. They should be able to spin-up relatively quickly and should respond to system commands to close.

For a web process, shutdown is achieved by ceasing to listen on the service port (refusing new requests), allowing any current requests to finish, and then terminating the process.

10. Development/Production Parity

A 12-factor app maintains as many architectural similarities as possible between the development, staging and production environments.

The 12-factor app is designed for continuous deployment.

Even when adapters theoretically nullify differences in backing services, the 12-factor app seeks to maintain parity in the environments. Differences in backing services mean that small incompatibilities crop up, causing code that passed tests in development or staging to fail in production.

Agile processes and rapid feature development are aided when all parts of the software development process share the same environments.

11. Logs

A 12-factor app never concerns itself with routing or storage of its output stream. It should not attempt to manage logs. Each process sends its event stream to stdout.

In development, the developer can view output in their terminal to observe the app’s behavior. In staging or production deploys, each process’ stream is captured by the environment, collated together with other process streams, for viewing and long-term archival. These log destinations are not visible to or configurable by the app. They are completely managed by the execution environment. Open-source log routers are available for this management.

The event stream for an app can be routed to a file, or watched via tail in a terminal. The stream can be sent to a log indexing and analysis system such as Splunk, or to a general-purpose data warehousing system such as Hadoop/Hive. These systems allow for great power and flexibility analysing app’s behavior over time, including:

Finding specific events in the past. Application wide graphing of trends. Alerting by user-defined metrics (such as an alert when the quantity of errors per minute exceeds a certain threshold).

12. Admin Processes

One-off admin processes should be run in an identical environment as the regular processes of the app. They run against a specific release, using the same codebase and config as any process running the same release. Admin process code must ship with application code to avoid synchronisation issues.

The Dependency isolation techniques should be used on all process types. For example, if the Ruby web process uses the command bundle exec thin start, then a database migration should use bundle exec rake db:migrate.

In a local deploy, developers invoke one-off admin processes by a direct shell command inside the app’s checkout directory. In a production deploy, developers can use ssh or other remote command execution mechanism provided by that deploy’s execution environment to run the admin process.

12-Factor App for Microservices

The 12-factor methodology applies to microservices as any other app. All of these factors apply, but notably microservice processes should be stateless, disposable and bound together by ports.

Microservices can add greater flexibility for scaling large applications and where systems are incredibly large and distributed, they make sense. That said, in most instances a well-written 12-factor monolithic application will horizontally scale to serve many thousands of concurrent requests without the need of adding the complexities of managing microservices. A monolith is generally easier for new developers to move into to understand all the moving parts. So, unless the application becomes incredibly large or the user volume vast, a more traditional approach is recommended.

Key Takeaways

The 12-factor methodology provides a robust, transportable, scalable and agile framework for application development and maintenance. It’s ideally suited to today’s cloud computing model.

12-factor apps are generally easier to migrate to new platforms and easier to horizontally scale and to serve many thousands of requests.

At reinteractive, we built our OpsCare using these 12-factors as the guiding principle.