Validating Python code with a CodeCatalyst pipeline

CodeCatalyst is a unified development environment created by AWS. It has many features such as blueprints to assist in writing code, integrated Git repositories, dev environments which can be pre-defined and now AI integration. However, for myself, one of the most useful things is being able to define and use pipelines stored in the code repository. Pipelines are one of the most commonly used tools that many of us working with code and Cloud use, allowing us to automate tasks to be carried out when we make changes to our code, whether that’s checking that our code works, building artefacts and packages, and deploying to our environments. Continue reading

Generating cloudwatch alarms using 'metric math' via CloudFormation and Terraform.

I spend a lot of time working as a consultant with GlobalLogic UK&I with different client teams to deploy AWS infrastructure, and not surprisingly, I see differing levels of maturity and experience within these teams. While we work with teams with a lot of knowledge, often they concentrate on deploying the applications and infrastructure, but they won’t think about how they can understand how well an application is working. This is an important aspect of working within the Cloud, usually termed monitoring or observability. Continue reading

Use VSCode to write Terraform? AWS AI can now help you write your code!

As I sit here writing this article, AWS’s annual re:Invent is just starting in Las Vegas. This huge event pulls together cloud enthusiasts from around the world to learn about the largest cloud providers offerings. Not surprisingly, AWS use this time to announce new and improved services, and we’ll see hundreds of articles over the next week - in fact, there are so many announcements, they have to start drip-feeding them out in advance of the main event. Continue reading

Handling partial batch failures with the SQS Lambda Poller

In my previous post, I described how I’d been asked to help a developer who was having some issues. His team was logging all S3 accesses to a central log bucket and then using Athena to analyse the data. They needed to partition this data to allow Athena to process the data more efficiently. However, a small number of the entries they were trying to process seemed to be missing when they used the architecture below: Continue reading

Now You See Me, Now You Don't - the Mystery of the Vanishing S3 Objects

One of the great things about my role as a consultant at GlobalLogic is that sometimes I’ll be asked to help out on what at first glance can be a simple problem, but as I investigate, I get a chance to uncover some unusual or forgotten features. Recently, I was working on a project and was asked if I could help solve an issue that had been puzzling a developer. They had deployed a system where some objects uploaded to an S3 bucket seemed to disappear and then reappear; it was time to start digging. Continue reading

Reviewing AWS SSM DHMC - too many acronyms or a useful tool?

AWS Systems Manager (SSM) is an integral component for managing EC2 and other compute fleets, offering capabilities such as patch management, parameter store, and managing changes across a fleet of servers. It also offers a service called Session Manager allowing secure, audited access to EC2 instances without needing to expose the instances on the Internet. In February 2023, AWS announced a new solution called Default Host Management Configuration or DHMC, to simply the setup of the core SSM capabilities by providing a method of ensuring that SSM was available for all instances in an account. Continue reading