The Latest CrowdStrike News
Product and Solution Information, Press Releases, Announcements

Visibility in the cloud is an important but difficult problem to tackle. It differs among cloud providers, and each one has its own positives and negatives. This blog reviews some of the logging and visibility options that Amazon Web Services (AWS) and Google Cloud Platform (GCP) offer. We also discuss their blind spots and how to eliminate them.

Visibility Options

Cloud providers typically offer some sort of default logging/monitoring at no extra cost, but it is never enough to gain sufficient visibility into what’s going on across your organization. They also provide additional paid services that allow you to gain more visibility into your environments for a variety of use cases. Each cloud provider does things slightly differently, meaning blind spots and lack of visibility will differ across providers.

The Cloud Control Plane

The “control plane” offered by a cloud provider is essentially what handles “cloud operations,” or API calls. The control plane needs to be accessible from anywhere on the internet, and it is what allows users to interact with the cloud and the resources in it. For example, API calls made through the AWS Command Line Interface (CLI) are routed through the AWS control plane, allowing you to take actions such as launching a new EC2 instance. In GCP, the control plane would handle things such as calls made through the “gcloud” CLI.

Visibility into what API calls are being made in your environment is incredibly important, which is why many of the free, default logging services are provided. This could include CloudTrail Event History in AWS (a 90-day history of API calls made in your account) or something like activity logging in GCP, which provides a broad overview of activity in a project. These default offerings have shortcomings, meaning that without additional configuration you will miss things.

To monitor the cloud control plane, you will need to use built-in services provided by your cloud provider, but those logs should then be exported to an external SIEM, such as Splunk, for further analysis and alerting. These services include things like an AWS CloudTrail trail being configured for every region and for every event type across your organization or GCP audit logs applied to all supported services at the organization level.

Network and Hosts

When it comes to activity within your cloud network or the hosts within that network, there is typically no free default offering to gain visibility. There may be default logging on your hosts, such as bash history, but there is nothing aggregating those logs and providing you access to all of your hosts through a unified interface.

There are many built-in and third-party offerings available to gain visibility into activity in your network and on your hosts. The CrowdStrike Falcon® platform is one option for host-based monitoring and visibility that allows you to detect and prevent threats in real time. Other offerings, such as AWS VPC (Virtual Private Cloud) Traffic Mirroring or GCP Packet Mirroring, can help with full packet capture within your cloud network. VPC flow logs are another useful tool for network visibility, and offerings such as AWS GuardDuty can monitor those flow logs, as well as DNS logs and CloudTrail logs, to detect threats and unusual activity within your environment.

Pain Points and Blind Spots

Regardless of the monitoring and visibility that is in place, even if you are utilizing all of the built-in services your cloud provider offers (and even some third-party services), there are likely missing pieces to the puzzle.

Data-Level Events

Data-level events differ from control plane/management events because they are actions performed on specific data rather than a cloud resource. For example, AWS Simple Storage Service (S3) data events log activity for objects in an S3 bucket, meaning you get insight into who’s interacting with what objects. Without additional configuration (and therefore additional cost), these types of events are not logged. Some services offer data-level event logging, such as S3 bucket access logs, but others do not, such as AWS EBS Direct APIs.

To ensure you are not losing insight into data-level events, you should enable them where possible. Where it is not possible, we recommend that you deny all users access to those data-level APIs.

Undocumented APIs

Every cloud provider publishes documentation for their supported APIs, which allows you to identify what is possible in that specific cloud. The problem is that there are often undocumented APIs supported by each cloud provider that may be exposed when they should not be. Often these undocumented APIs are used to enhance user experience, such as providing a good UX through the web console the provider supplies. Because these are undocumented and generally not for direct access, they are not logged where other documented APIs are logged. This could allow an attacker to perform certain actions in your environment without any way of knowing they did so.

One example of an “undocumented” API in GCP that did not seem to be logged is the API that lists all indexed Google Cloud resources that you have access to. This used to be able to be executed with the “gcloud alpha resources list” CLI command, but it was recently removed. It was also removed from the documentation, as can be seen on this gcloud alpha page, where there is no “resources” group available (although I did have the actual API reference page favorited, and it seems to still be valid as of 11/5/2020 — it can be viewed here). Because this is not linked in the official documentation and it did not seem to show up in a project’s audit logs, it could pose a serious risk to an environment.

To prevent undocumented APIs from being used maliciously in your environment, it is important to grant permissions on a granular level. That means: Don’t grant permissions using wildcards (such as using a “*” in AWS), don’t use “managed” permissions sets, as they are often overly permissive, and regularly review the permissions you grant your users to ensure they are properly restricted. By allowlisting permissions on a granular level, you can avoid much of the risk that undocumented APIs expose.

Pre-GA Services

Another potential blind spot in the cloud is pre-GA (general availability) services. This could include alpha, beta, gamma, pre-production or whatever-you-want-to-call-them services. Often these services are blocked from the public and only permit allowlisted users access, but there are also many times where they are available for public use while still in pre-GA status.

One example of how to access pre-GA services is the commands available through the “beta” and “alpha” groups of the “gcloud” CLI. These can be accessed with “gcloud beta ” and “gcloud alpha ,” respectively. Just because they are pre-GA doesn’t mean they are not logged, but this is where you may encounter services that do not yet support logging and have a potential for abuse. Access to many of these beta and alpha services in GCP can be prevented by denying users access to enabling new APIs in their projects and only providing them access to an allowlist of trusted services.

Permissions must be set at a granular level so they only grant access to services and APIs that are necessary. These risks often arise when wildcards or managed permission sets are applied in an environment, where you can’t be 100% sure of every service and action you have granted to your user. When setting the permissions granularly, you can ensure that nothing unexpected can be accessed by your users.

The Principle of Least Privilege

Without taking advantage of logging and visibility offerings provided by cloud providers, you can lose insight into what’s going on in your environment. Visibility gaps can still exist even with the proper services enabled, as we discussed with data-level events, undocumented APIs and pre-GA services, so it’s necessary to follow the principle of least privilege to avoid granting access to services and APIs that escape your monitoring capabilities.

It may seem like the cloud provider’s responsibility to not release undocumented APIs or pre-GA services, but until that is demonstrated as fact, it is your responsibility to delegate permissions and access in such a way that these APIs do not expose your environment to risk.