Audit logs in the cloud: who was it?!
Why keep and process audit logs honestly and what tools to use for their analysis?
Martin Gavanda
IT guy 1: “Please, do you know why it’s not working now? It was running yesterday!”
IT guy 2: “I traced from the log that someone logged in with an admin account, but I don’t know what changed.”
IT guy 1: “Who changed anything in there?”
Chorus: “Not me.”
Similar situations have probably happened to all of us. Personally, I hope that it is rarely the case anymore and that we all have the right change management process in place. Keeping and processing environmental audit logs honestly is also important, perhaps because your internal guidelines require it, or you are even obliged to process audit logs due to external regulations (e.g. CNB regulation, critical infrastructure of the state or different ISO standards). So why should we store audit logs and what tools should we use? That’s what we’ll talk about in today’s article.
Basic division of audit logs
In principle, audit logs can be divided into two basic groups. Each is equally important and we should not forget one or the other.
Remember that audit logs should always be stored and processed, regardless of the technology used or where the application is run from. Furthermore, we must not forget the topic of data retention. Due to the fact that audit logs are typically retained for regulatory reasons, it is necessary to retain this data for a long period of time (sometimes several years).
- Environment audit logs: typically contain information about who has manipulated, accessed or modified the environment and how. These are typically logs that reflect administrative actions (who changed which component or resource and how, which data in the database or which files were accessed).
- Application audit logs: these logs map and describe user activity within the application. For example, this may include information about what application functionality the user accessed, what user inputs were used to generate the offer, or what document was downloaded.
As part of our Cloud services, we can help you define and set up the necessary rules.
Audit logs and the cloud
A major benefit of the cloud environment is that audit logging tools are a direct part of all major cloud platforms. In the cloud (as opposed to a typical on-premise environment), all actions are performed through APIs – either by calling these APIs directly or indirectly through UI interfaces or various software development kits. If we perform all operations “over one place” (the API), it is very easy to log and process all these actions.
Environment audit logs
So what tools do individual cloud providers offer us? In the case of AWS, it is AWS CouldTrail, while Azure offers Activity & Resource Logs.
The goal of these tools is to capture any activity (operation) performed over all resources. These activities can be triggered by the user or directly by the cloud environment or another service.
For example, here we see activity in one of our internal Azure environments and we can clearly see that some operations were performed by the user (here specifically by me) and others are performed directly by the cloud environment (automatic backup of a virtual server).
A detailed audit trail is available for each operation, which stores all the information about the individual operations performed. Typically, who (identity or role), when (when the operation was called), where (environment and region identification), from where (IP address, endpoint identification) and what (what operation over what specific resource) was performed within the environment.
Processing and visualization of audit logs
The first part is over – so we have detailed logs of all operations. Both of the tools mentioned above (AWS CloudTrail and Azure Monitor) offer basic filtering and browsing capabilities for these logs, but you will likely want to implement some additional (either native or 3rd party) tools in your environment to visualize this data.
If you choose to use pure cloud provider tools, you’ll likely reach for tools like AWS CloudWatch, AWS QuickSight, or Azure Log Analytics. All of them aim to process any volume of source data and visualize and offer it to users based on your preferences and requirements.
The next step for optimal work with audit logs will probably be the definition of specific notification rules. For example, it would be interesting to see the rate of failed attempts (access denied) over individual accounts, which may indicate that a user is trying to do something they shouldn’t.
Another example might be the notification of key operations over production resources, but there is no limit to your imagination and you can certainly come up with many other interesting scenarios.
External tools for processing audit logs
Nowadays, there are a wide range of external tools that can be used to analyze audit logs. The primary difference from native cloud tools is a certain higher level of “intelligence”, pre-made repots and templates, and support for different environments, which is useful for those of you running multi-cloud environments.
For me personally, I would recommend taking a look at the following tools:
The individual tools differ from each other, especially in terms of the additional security features offered. There is certainly no perfect tool, each has its pluses and minuses. I definitely recommend to deploy the selected tool in a Proof of Concept implementation and thoroughly test all the required features before the final implementation.
Application audit logs
The second area you should cover is audit logs within the application itself. Here the situation is a bit more complicated because each application is unique. In general, you should be able to identify within the audit logs every user action performed in the application itself, such as:
- Execution of a business event
- Access to application data
- Calling a key operation
- Change of data
But how to achieve this? Here, it will probably be up to the application development team to decide what technology or approach to use. Functionality for collecting application audit logs can be implemented purely “internally”, i.e. within the development of the application itself, or you can use services offered directly by cloud providers.
Here I would like to mention AWS X-Ray and Azure AppInisghts. Although these services are not focused on the area of application auditing (their primary use is for application monitoring), but thanks to the fact that with proper implementation they “see in detail” into the application itself, it can be relatively easy to create your own “audit logic”.
How to process and evaluate these audit records? It’s similar to environmental audit trails – you can use, for example, Azure Log Analytics and visualize the data or reach for standard log management tools such as Splunk or ELK Stack (Elasticsearch, Logstash and Kibana).
What do you use?
Audit logs are not magic, and they are remarkably easy to work with in the cloud because they are available to you automatically. Don’t forget them, store them and process them. They will come in handy! And the next time someone asks you, “Who was that?!”, you won’t have to think long.
What tools do you use? Do you only use the native tools of each cloud provider, or do you also use other technologies and tools? I will be glad for any comments and especially personal experience.
And what can you look forward to next? In the following article we will take a look at the detailed possibilities of cloud monitoring and my colleague Jakub Procházka will introduce in detail some of the tools I mentioned in this part of the Cloud Encyclopedia. All previous chapters of the series can be found here.
This is a machine translation. Please excuse any possible errors.
