Reducing Manual Deployment Work by 70%

At Softcraft, I worked on some systems where deployments depended a lot on a few people knowing the exact sequence. Most releases worked, but someone still needed to build the correct version, check the configuration for each environment, coordinate database changes, deploy the services and then verify if everything was healthy.

We had checklists, but they didn’t contain everything that the more experienced people knew. A step like “validate the environment” could mean several checks to the person who wrote it. Staging and production also had their differences, and when a deployment failed, the recovery process was sometimes clear only to whoever had dealt with a similar problem before. This made even a normal release depend on the right people being available.

The First Pipelines

We didn’t try to automate the complete process at once. We started with the steps that were repeated in almost every release, including builds, tests and packaging, and then continued with deployment and some of the checks we performed after it. We used Docker, Kubernetes, GitHub Actions and Azure DevOps across these systems.

Setting up the tools wasn’t the part that generated most of the discussion. We needed to agree on which tests had to pass, how configuration should be provided for each environment and what the pipeline needed to check before considering a deployment successful. A command could finish without an error while the application was still unhealthy, so the verification after the release also became part of the work.

Database changes needed their own place in the sequence. Some versions could run with both the old and new schema for a while, while others needed closer coordination between the migration and application deployment. We avoided putting every change behind the same generic step because a rollback was also different when data had already been modified. For the normal cases, the pipeline made the order explicit. For the more sensitive ones, it showed where somebody still needed to confirm the next action.

The Parts That Stayed Manual

Some steps remained manual for a while because we didn’t understand them well enough yet or because they needed a decision from somebody in the team. When a process exists mostly in people’s memory, automating it forces everyone to describe what actually happens, and sometimes we found that two people followed slightly different versions of the same checklist.

We moved these parts gradually instead of hiding every possible case inside a very large pipeline. This gave us time to use the automation in real releases, see where the assumptions were wrong and change the next part without making the complete delivery process difficult to understand.

The team also needed to get used to changing the pipeline when the release process changed. At first, it was easy to treat the CI/CD configuration as something that belonged to DevOps and work around it when a new requirement appeared. We started reviewing those changes with the application code because the build, tests and health checks depended on decisions made by both sides. That reduced the number of one-off commands executed only during a release.

After the Change

Over time, the manual deployment work was reduced by around 70%. Fewer steps depended on the setup of someone’s machine and we didn’t need as many messages to find out who knew what came next. New people could also understand more of the release by looking at the pipeline instead of learning the complete process during their first production deployment.

Important releases still had some tension, especially when they included database changes or behavior we couldn’t reproduce completely outside production. The repeated work became much more predictable, and normal releases stopped requiring so much coordination from the team.

The 70% reduction came over several changes, not from one pipeline replacing the complete process in a week. Some releases saved more time than others and there were still applications with older steps that we hadn’t moved yet. I prefer describing it that way because the result was useful even before every part was automated. Each repeated step we removed was one less thing someone needed to remember on release day.