Intro

My notes on what characteristics make up a good DevOps build system.   These are in no particular order

No Ops Team

The practice of having an Ops team to deploy developers' code becomes a bottle neck.  There shouldn't be an Ops team.  Developers need to own the entire pipeline.

Fully Reproducible Builds

At any point in time, any developer should be able to faithfully reproduce any previous build with the same exact artifact versions (including compilers, libraries, OS, system properties, etc).  Full traceability of builds, including dependencies graphs, environment data should be easily maintained and viewable.

A good system must give the developers the freedom to use their preferred languages and tools.  This means that a good DevOps system must accommodate support for a large number of programming languages and their packaging formats/repos (or even higher level constructs like Docker images). 

Auditable

Certain certifications and governmental agencies require build be auditable so one can prove the provenance of every bit of code in a production release.  Artifact servers are a good way to aggregate all components (internal and external) and provide the audit trails needed.

Third-Party Dependencies

Most software today has dependencies on external packages/libraries.  It is not been an uncommon practice for developers to check in these libraries, reference a file share or internet server where they reside during a build.  Doing this makes it really difficult to provide reproducible builds, plus auditing for license or governmental requirements.  Using an artifact server for these dependencies solves these issues

Build Comparisons

It should be a trivial task for any developer to compare builds to determine what changes have been made.  This can be used to show incremental feature and bug fixes from one release to another.

Release Management

The ability to perform continuous deployment of builds is key in a good pipeline.  It should allow for staging deployments where testing and monitoring can determine the health of a build before being promoted to a production stage.

Integrate with the target cloud platform(s) as much as possible and in an automated fashion.

Caching

Deploying final products into a production plane could put a heavy toll on artifact servers.  One should consider caching servers located close to the deployment zones.  For example, deploying to an East Asia production plane from a US artifact server will likely experience long download delays and sometimes even unavailability of the connection.  Therefore, caching local to the deployment region makes sense.

Testing

A good DevOps pipeline must include large amounts of testing:

• Security
• Performance
• Integration
• Unit
• Regression
• Test the complete process

Logging

Must provide good logging to get better at detecting issues.

Feedback

Find a way to provide automatic feedback to all stakeholders.  Developers must have a way to change the process/tools.

Tools

Tools must be swappable.  New tools will be created and old ones will become useless.

They should be hosted in the cloud BUT also allowed for on-premises hosting. Cloud is the ultimate platform target that most enterprise applications will end up.  Optimize for them.

Each tool must be testable as a unit (outside of the pipeline).  We want to make sure the tool is running as expected.  Then move on to test a cluster of tools (integrated with others)

Select best-of-breed tools.

Metrics

Automate metrics capture.  You want to capture metrics on the entire cycle, from code commit to production.

Lead time tracks the total amount of time it takes from when work is requested until it's delivered. Cycle time tracks the amount of time we spend working on it while it's on our board.

Template/Library Management

Find ways to re-use build templates so each team is not re-inventing the wheel.  Look for ways to provide a "service catalog" where developers can find code to be re-used.

Syntax And Formatting Conventions

Add tools to check for syntax and formatting conventions.

Training

Must provide training for developers and other stakeholders for the pipeline to be successful.  People are everything.  Don't force change.  People fear change so allow them to understand what the process will be.