More effective use of AWS through operational excellence

Have you ever had the feeling of constantly encountering obstacles in your work in the IT landscape? Perhaps you have experienced projects with great potential suddenly stalling due to unforeseen events. Or, even more frustrating, you've put months of hard work and effort into a project only to find that the end result doesn't meet the customer's expectations.  

Technology plus mindset

Managing resources effectively becomes an ever-increasing challenge as you want to ensure that every hour spent and every euro spent adds value. There's also the added pressure of keeping the team motivated and energized, especially when faced with these complications. You may have noticed that technical expertise alone is no guarantee of success, but needs to be combined with the right methods to add value. This is where the concept of "Operation Excellence" comes into play.

Excellence is about more than just doing things right - it's about reaching your highest potential at every step and working towards a clear, common goal. Even if you achieve this goal, there is always room for improvement. In the pursuit of excellence, reaching the goal is not the end, but the new starting point for further improvement.  

Integrating Operational Excellence (OpEx) into your IT projects means adopting this mindset in your daily work. Excellence is pursued through an iterative process throughout the life of the project. At each stage, a detailed review is conducted to ensure that everything is going according to plan, followed by constructive feedback from everyone involved, from team members to stakeholders, including the customer. The process is not static: OpEx emphasizes ongoing evaluation and adjustments. This constant cycle of planning, implementation, review and action ensures that projects not only progress, but evolve. In addition, this participative method encourages collective ownership and collaboration, with team members strongly committed to the success of the project.

Adopting the Operational Excellence methodology in cloud environments, especially on platforms such as AWS, reinforces its benefits. Cloud computing, with its dynamic nature, requires constant evaluation and adaptation. By applying OpEx in your cloud strategy on AWS, you can maximize services, reduce waste, and standardize your workflows and infrastructure to get the most out of your AWS project. In addition, AWS promotes OpEx as part of its "Well Architected Framework".

Pillars of operational excellence for AWS projects

Operational Excellence is not a strict method that you have to follow as closely as possible, as is the case with SCRUM, for example. Operational Excellence is a set of guidelines and ideas that can put you on the right track.

Feedback alone is not enough - create a healthy error culture
Every IT project on AWS starts with a clear goal. However, to achieve this goal, it is not enough to set a direction and work towards it. Rather, it is about continuously ensuring that you are on the right track, while being aware that mistakes are inevitable.

You will have noticed that people don't usually like to talk about mistakes and may even sweep them under the carpet. This then leads to other people making the same mistakes and these mistakes having much more serious consequences than they should have. If you enable your team to talk openly about mistakes as soon as they occur, everyone involved can learn from them. Seeing mistakes as opportunities for improvement is crucial!

This mindset can easily be applied to the world of AWS. You want to make data-driven decisions and therefore need feedback not only from your colleagues, but also from the IT infrastructure. AWS offers some tightly integrated services that are designed to deliver insights. For example, use AWS Cloudwatch's built-in alerting system to be notified when something goes wrong. Check your deployments with AWS Inspector and eliminate vulnerabilities. Apply AI via Amazon Mazie to scan your data for sensitive information you need to act on.

Continuous improvement
The concept of continuous improvement becomes particularly relevant when you apply operational excellence in your AWS project. Because the nature of the cloud is one thing above all else - dynamic. New services and functions are added on a weekly basis. In addition, your cloud infrastructure is also very dynamic: instances may need to be scaled during periods of high or low traffic, infrastructure may need to be refined frequently, and user requirements may change unpredictably.

As a developer working in the AWS environment, you want to adapt to all these changes as quickly as possible and deliver the best possible product. But since even the "best" product is only two weeks away from being just "good" due to the ever-changing environment, you need to take this into account when planning your project.

The perfect application cannot be created, and if it is, then only for a short period of time. To cope with all influencing factors, it is necessary to implement a process of continuous improvement. Only then will you have the chance to stay ahead of the curve.

To do this, you should iteratively improve your product and processes using methods such as the PDCA cycle. In this way, you can planyour goal, carry outthe necessary implementations, review and analyze your results - and become activebased on the knowledge gained. Remember that the end of a PDCA cycle is only the starting point for the next iteration with new goals

Reduce waste

Becoming efficient is difficult, but there is a path you can follow to get you moving in the right direction. Looking at our AWS projects, we can identify seven types of waste that prevent efficiency:

1. Transport: Unnecessary data movements between systems and locations.
2. Inventory: Accumulation of unfinished work, unfinished functions, excessive data.
3. Movement: Inefficient tools, unavailable information, complex interfaces.
4. Waiting time: delays in approvals, unresolved dependencies and blockades.
5. Overproduction: Unnecessary functions.
6. Overfulfillment: Carrying out work on a task after it has been completed that does not offer any added value.
7. Defects: Bugs and errors.

Now that we have an idea of what to look out for, we can apply the 5S method, which provides a clear framework for reducing waste. This method comprises five phases that we have adopted for working with AWS projects:

1. Sort: Remove unnecessary elements. Delete superfluous files and data. In AWS, ensure that only active, necessary resources are executed. This not only saves costs, but also reduces potential sources of error. Delete old logs and prevent large amounts of data from slowing down operations.
2. Set: Structure and organize the remaining elements. Archive old data regularly, compress it if possible and select the correct storage class.
3. Shine: Maintain and implement updates for your tech stack and work environment.
4. Standardize: Create consistent AWS deployments using existing software and Infrastructure-as-Code modules.
5. Sustain: Be disciplined, carry out the 5S method regularly and thus improve the process.

For example, you could do 5S in your team for 30 minutes every Monday morning and share your results.

‍Qualitythrough standardization
If everyone follows their own approach, the results can be unpredictable. The larger the team and the project become, the more important a clearly defined standard process becomes.

One way to achieve consistency is to use Infrastructure as Code (IaC) through tools like Terraform. With Terraform, teams can define and deploy their AWS infrastructure using code and automation. Instead of manually configuring resources, developers rely on configuration files to ensure that the infrastructure is version-controlled, shareable and consistent. Terraform offers an important advantage: the ability to use pre-built Terraform modules. These modules, which have already been created for specific AWS configurations, provide basic code templates. Developers can then configure these modules to meet specific project requirements. This approach not only speeds up deployment, but also supports standardization and reduces the likelihood of errors.

At celver, we have developed an extensive catalog of Terraform modules that are already available for use. These pre-built modules allow you to create and manage your infrastructure efficiently and significantly reduce the time and effort required for deployment and maintenance.

Once you have the building blocks ready, create a CI/CD pipeline with AWS CodePipeline and CodeBuild to deploy the infrastructure in a production environment and you can significantly reduce the risk of human error.

Conclusion

A great tech stack is an excellent foundation for developing a good product, but that's only half the battle. If you shift your mindset towards excellence, you can achieve even more than just reaching your goal. In the fast-paced AWS environment, this is exactly the right approach. Because there is waste in all processes and many people are working without adding value. This is not good for the company, nor is it meaningful work as a developer. Striving for continuous improvement with the ideas that Operational Excellence offers and the AWS services that support this will have a positive impact on your projects.

Of course, it is also a challenge for everyone to live in a culture of failure, learning and continuous improvement. Achieving these goals is not easy and often requires a lot of time and effort. However, the principles of operational excellence applied to your AWS project through methods such as PDCA and 5S, combined with the standardization of Infrastructure as Code, provide a good start to get you on the right track.

Do you also feel like you're not getting the most out of your AWS projects and development process? Are your developers frustrated by rigid target definitions and arbitrary metrics for a good product? Then let's discuss together how we can optimize your processes and provide the right tools.