Building and Operating My Own Production Systems

In freelance projects, I often owned more than the implementation. I designed the backend, modeled the data, built features, deployed the application, and responded when production did something unexpected. Sometimes the first alert was a client message rather than a dashboard. That changed my relationship with the code. A vague error message was no longer a small detail someone else would handle. A difficult deployment was a process I’d repeat. A hidden background failure could become a client asking why an operation hadn’t completed. I knew I’d need to understand these small decisions again when something happened in production, probably at an inconvenient time.

Error Messages in Production

During development, a stack trace and local debugger can provide enough context. In production, the useful question is often which user operation failed and what the system had already done. I started giving important workflows identifiers that appeared across logs and persisted state. Errors distinguished invalid input from a temporary dependency failure. Logs recorded relevant decisions without exposing sensitive data. This didn’t make incidents easy, but I spent less time searching unrelated logs while somebody waited for an answer. Instead of searching by timestamp and guessing which messages belonged together, I could follow the operation through the application.

Background Job States

Several products included scheduled tasks, notifications, integrations, or other background jobs. In code, it was easy to think of them as functions that continued outside the user-facing request. Then a notification went missing or an integration stopped halfway through, and that model was clearly incomplete. Had the job been scheduled? Was it running? Did it finish? Was it safe to retry? Would a failure remain visible, or would the user simply notice missing behavior later? I started storing meaningful processing states and enough information to understand incomplete work. If a background job fails silently, somebody will only find it later as a support problem. Owning deployment made me more careful about changes that crossed code and data. A database migration needed to work with the application version active during rollout. A new required field couldn’t assume every existing record already had a value. A long-running job might still be using the previous behavior while the new version started. These situations encouraged smaller, compatible steps instead of one change that required everything to switch at exactly the same moment.

I also invested more in automated builds, tests, and deployment pipelines. I had repeated enough manual releases to know that familiarity wasn’t the same as reliability.

Keeping the System Small

When I operated what I built, every additional component had a very visible cost. A separate service meant another deployment and more configuration. A new database meant backups and another set of access patterns to understand. For smaller products I usually stayed with a modular application and a few dependencies I knew well, partly because there wasn’t an operations team behind me. I was the person who would receive the client message, open the logs and try to remember why the code behaved that way. Knowing that changed small details in the implementation: better operation identifiers, clearer errors, visible job states and deployment steps I could repeat. I was mostly leaving information for my future self, but it also helped the next engineer who needed to understand the project.