Portable code enables a smooth flow to production for the product

What is portability? The application or platform you are currently developing needs to work on your machine but also in the production environment where all the users are using it. The measure of Portability is the ability of the application to behave predictably in all environments. 

In this article, I talk about portable configuration and using code branching effectively with your team when developing software products. By adding some DORA metrics to your reporting, it's easy to quantify the effect portability has on your overall quality

why do I need to do this? 

The reliability of the software increases when it works in production without modification, so this effort starts when developing on your local machine. When developing an application there is going to be a need to build the app locally and deploy it to your production environment for the users to use the app. There could be testing, staging, and other environments along the way. Enabling some good practices for the configuration of the app makes developing and testing a lot easier and reduces the stress of deployment.

Keys to success:

• A new person should be able to pull, run tests, start app with just .local.env, and can hook into secrets in a vault when in dev, staging and production environments
• Keep environment config versions and use the environment to use a specific file.

how do I do this?

The application code shouldn’t change, the configuration of the code is what changes. This configuration is unique to each environment (local, dev, staging, production). Reduce the complexity of moving to production removes barriers and increases velocity

• Collaborative development with PRs
• The sooner the smallest architecture changes are established, the easier the validation in the build process. The maintainability of the codebase is fundamental to portable software
• Configuration of the System. 
• By deploying to the integration environment and testing the system, you can gain confidence that the system will work the same way in production
• use env files and keep secrets safe in cloud storage
• Configuration of the product
• Use Feature Flagging to limit access to incomplete functionality

• Align on the product level for releasing the user-facing feature. Release internally first so docs and support know what the customer is getting. 
• https://jseller.blogspot.com/2019/01/software-releases-are-lot-more-than-code.html

Branches of Code

How to integrate my work with the code from other developer teams? Use Trunk-based development for collaboration and velocity. This process allows integration of PRs and rapid deployment of the outcomes.

What is the trunk? it is the main branch and should match the deployed branch in the production environment. When changes are in the main branch, expect those changes to work immediately in staging and production environments.

• Ensures smaller changes
• features that aren’t ‘ready’, that don’t fulfill the full intention of the feature and don’t fully solve the user problem don’t need to be in a separate branch,
• Use feature flags to only allow users that have the alpha, beta or GA access, but remove them and tie them to the license source of truth for access and usage stats to understand the impact (and usefulness) to the user
• sometimes an alpha version is required to show progress and demoing the functionality through the alpha-beta-GA cycle 
• Prevents over-engineering through smaller changes
• Favours rebasing changes on the main branch instead of large merges. This keeps the history cleaner on the main branch and makes rollbacks much easier if needed. 

Feature Branches

• building a feature in a separate branch sounds like a good idea, but incurs a lot of maintenance effort when keeping the branch up to date
• frequent updating and merging
• merge conflicts

Release branches

• When a deployment to production is complete, the semver of the release is known. It can ease the transition to trunk-based development to create a Release branch to handle any hotfixes made to the production system between releaseses. This allows small changes in prod so users aren’t blocked by using the product while the large feature branches are merged and validated. 
• the same change is applied to the main branch with a test so the main branch has the changes integrated already for the next deploy

More on specific git commands here 

https://jseller.blogspot.com/2020/03/source-control-saves-time-and-complexity.html

Automation

Use automation to validate this portability and test the observability at the same time.

• Use a certified CI/CD for package and dependency validation

Use pipelines in bitbucket or github to automatically deploy your main branch to a development server, for larger production deployments there is Harness and other tools that can deploy to many nodes to reduce the complexity of setting up many production instances.

• Run tests appropriate to the environment

1. local tests and mock data for local development
2. Integration tests and synthetic data for staging
3. Production smoke tests to validate

Use an automated test to add a quality step to your deployment scripts. The deployment is finished when it is validated, not just deployedValidate your observability when validating the deployment. This is essential to certify the system is up to date for deprecation or security fixes

Metrics to measure?

Good portability helps velocity and collaboration, but how does it do that? Use DORA metrics to understand how the portability is affecting deployments and rework

• Change Failure Rate
• Deployment frequency

https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance

Conclusion

No matter how careful you are about the portability of the app; things are going to happen when many people are working on the application. 

Adding quality through Portable patterns like code branching and automation will go a long way to ironing out the issues before deploying to production

Also see:

https://jseller.blogspot.com/2019/01/feature-definition-with-quality-model.html

https://jseller.blogspot.com/2020/03/packaging-code-for-development-and.html

https://dora.dev/capabilities/trunk-based-development/

Continuously integrate software and everything around it

Continually integrating code increases velocity and quality

While you are working away on a project or a paper, its easy to handle all the modifications on your own, since you are the only author. Things get interesting when many authors need to use the same source.

Integrate early and often.

Especially when you have the very first thing working, and you have a unit test and functional test doing the bare minimum; take a moment to integrate with the larger project, commit changes to the branch and generally harden the edges around what is newly build. 

Once the resulting leaner version of feature is in the build, expanding on that feature is done more effectively as you have the build cycle moving with everything intact.

Automate the building of the software. Once the scripts are done (and they are way easier than everyone seems to think they are) the project builds itself every night. Why not do this? Everyone is home sleeping, and the computer at the office are just running idle all night. Use that time.

 
The build script, like all scripts, start small and go from there. The build script should do the basics:

• Clean the source directory.
• Sync code into staging area for building
• Build all source in staging area. Full build too; no incremental, or half build shortcuts. Build dependencies, and any inclusions.
  Build target libs and place into target directories.
  Build binaries and link all necessary libraries.
• Clean all intermediary files
• Package binaries and libs for distribution
• Configure target machine for deployment
• Check network configuration
• Deploy binaries and libs

Each step some reporting can be done and the results archived. The script runs at night when everyone is asleep. The results are ready the next day, and everyday.
Polish your build!

 
CONTINUOUS INTEGRATION

• Integration is the assembly of all the parts to end up with a deliverable 
• building, testing, packaging, deploying.. 
• One person doesn’t need an integration effort, but more than one person does 

INTEGRATION

• Artifacts are created: source code, and other dependencies 
• Source from different places has to end up in one place 
• There are many options, but using a source control tool is the best practice. 

SOURCE

• A common, neutral build is necessary. 
• ‘Works on my machine’ works if you are distributing your machine 
• A common build machine/environment eliminates local dependencies and gives confidence it can work elsewhere 
• When to build all this? Weekly? Daily? 

BUILDING FROM SOURCE

• Once the source compiles, how do you know it works at all? 
• Manually running the build and exploratory testing can take you so far 
• Having unit tests can give more confidence that the code that is tested will work 

TESTING THE BUILD

• You can just zip up the compiled source, email it somewhere and run it… 
• Error prone, it’s easy to forget menial steps 
• Time consuming as new dependencies are usually found as a result of things ‘not working’ 
• Run those tests to give confidence that all tests will pass. Tests that aren't in the dependency tree shouldn't be affected. Make unit, integration and functional tests run separately or all at once

PACKAGING FOR DEPLOYMENT

• Building, testing, packaging, versioning and deploying define a process 
• If it is a defined process, it can be repeated. 
• It’s ‘defined’ if you did it once. Never wait for a master plan to start, just evolve the plan as you go. 
• The build is ‘broken’ if the entire process doesn’t complete. 

IT’S A PROCESS

• Once the process is automated, run the process as much as you can, not ‘when you need to’.
• Always look to automate any repeatable process. 
• Run the process continuously and be aggressive about finding more tasks to automate 
• Why do all that work when a computer can do it for you? Computers just sit around all night and they don’t get tired. 

AUTOMATE EARLY AND OFTEN

• Spend your time actually building the technology, not doing the housekeeping around it.
• Automate the source and change control, the build and packaging, the testing and verification, and the deployment 
• Only one trigger is needed: when the source changes. 
• Run on every change, every hour, all night… all the time. Who cares? It’s automated. 

CONTINUOUSLY INTEGRATE EVERYTHING

• Continuous integration tool: ’Hudson’ 
• Source Control: git 
• Building: using compilers specific to the code base 
• Metrics: running analysis tools on codebase 
• Testing: running unit tests on every build 
• Metrics: running more analysis tools on built codebase 
• •Packaging: auto deploying to development environment 

PRODUCT

• Allows builds anytime that have aligned versions

Wrap it up

Continuous integration is a great strategy to automate a lot of quality into your application, and over time you will end up saving a lot of time and increasing the general quality of the technology that you are making.

Its a process; do a little each week and the results will quickly build up over time.