Introduction to Amazon CloudWatch
Last Updated : 17 Jun, 2024
Amazon CloudWatch is a service used for monitoring and observing resources in real-time, built for DevOps engineers, developers, site reliability engineers (SREs), and IT managers. CloudWatch provides users with data and actionable insights to monitor their respective applications, stimulate system-wide performance changes, and optimize resource utilization. CloudWatch collects monitoring and operational data in the form of logs, metrics, and events, providing its users with an aggregated view of AWS resources, applications, and services that run on AWS. The CloudWatch can also be used to detect anomalous behavior in the environments, set warnings and alarms, visualize logs and metrics side by side, take automated actions, and troubleshoot issues.
Amazon CloudWatch Agent
Amazon CloudWatch is an open-source lightweight tool that is used to collect the data of the resources in which they are deployed. Some of the data is as follows
- Metrics: Amazon CloudWatch agent will record the data of CPU utilization, memory usage, disk i/o other system-level stats.
- Logs: It will collect all the logs which are used for the further analysis
- Events: Launching of significant instances, modifications to security groups, and other events.
What is Amazon CloudWatch?
Amazon CloudWatch is a monitoring and observability service provided by Amazon Web Services (AWS) that enables users to collect and track metrics, monitor log files, set alarms, and automatically react to changes in AWS resources. It helps users gain insights into the operational health, performance, and resource utilization of their AWS infrastructure and applications.
Why Amazon CloudWatch?
Amazon Cloud Watch is a monitoring service offered by Amazon Web Services to monitor applications like the following.
- Performance.
- Health of the application.
- Monitors the resource use, etc.
You can set the alarm to the to the resource use of the applications when the limits are exceeded then you will get the notification to the mail automatically.
How Amazon CloudWatch Works
At first Amazon Cloud watch will configured to the resource that you want to monitor from there the agents that are configured will be used to collect the logs from the resources the service may be run on-premises or AWS. CloudWatch also provides the overall view of the resources with the help of a dashboard from where you can troubleshoot the issues. CloudWatch also performs the operational changes depending on the changes made to the resources they will also perform the AWS auto-scaling of the resources depending on the changes that occurred. CloudWatch performs real-time analysis based on the logs that have been received.
Amazon CloudWatch Features
Metrics
- It represents a time-ordered set of data points that are published to Amazon CloudWatch.
- All data point is marked with a timestamp.
- Metric is a variable that is monitored and data points are the value of that variable over time.
- They are uniquely defined by a name, namespace, and zero or more dimensions.
- Metric math is used to query multiple CloudWatch metrics and use math expressions to create new time series based on these metrics
Dimensions
- A dimension is a name/value pair which uniquely identifies a metric.
- Dimensions are the unique identifiers for a metric, so whenever you add a unique name/value pair to one of the metrics, you are creating a new variation of that metric.
Statistics
- Statistics are metric data aggregations over specified periods of time.
- The few available statistics on CloudWatch are maximum, minimum, sum, average, and sample count.
Alarm
- It is used to automatically initiate actions on our behalf.
- It watches a single metric over a specified time period and performs one or more specified actions based on the value of the metric.
- The estimated AWS charges can also be monitored using the alarm.
Percentiles
- It represents the relative weightage of the data in a dataset.
- It helps the user to get a better understanding of the distribution of metric data.
CloudWatch dashboard
- A user-friendly CloudWatch console is available which is used for monitoring resources in a single view.
- There is no limit on the number of CloudWatch dashboards you can create.
- These dashboards are global and not region-specific.
CloudWatch agent
CloudWatch Events
- CloudWatch events help you to create a set of rules that match with any event(i.e. stopping of EC2 instance).
- These events can be routed to one or more targets like AWS Lambda functions, Amazon SNS Topics, Amazon SQS queues, and other target types.
- CloudWatch Events observes the operational events continuously and whenever there is any change in the state of the event, it performs the action by sending notifications, activating lambda, etc.
- An event indicates a change in the AWS environment. Whenever there is a change in the state of AWS resources, events are generated.
- Rules are used for matching events and routing to targets.
- Target process events. They include Amazon EC2 instances, AWS Lambda functions, etc. A target receives the events in JSON format.
CloudWatch logs
- Amazon CloudWatch logs enable you to store, monitor, and access files from AWS resources like Amazon EC2 instances, Route53, etc.
- It also helps you to troubleshoot your system errors and maintain the logs in highly durable storage.
- It also creates log of information about the DNS queries that Route 53 receives.
Getting started with Amazon CloudWatch
Notifying gfg website management team when the instance on which gfg website is hosted stops Whenever the CPU utilization of instance (on which GeeksForGeeks website is hosted ) goes above 80%, CloudWatch event is triggered. This CloudWatch event then activates the SNS topic which sends the alert email to the attached gfg subscribers.
Create an SNS Topic
Step 1: Let us assume that you have already launched an instance with the name tag 'instance'.
Step 2: Go to SNS topic dashboard and click on create a topic.
Step 3: You will be directed to this dashboard. Now specify the name and display name.
Step 4: Scroll down and click on create the topic.
Step 5: The SNS topic is created successfully.
Add Subscribers
Step 1: Go to the SNS topic dashboard and click on gfgtopic link.
Step 2: Under the subscriptions section, Click on Create subscription.
Step 3: Select Email as protocol and specify the email address of subscribers in Endpoint. Click on create the subscription. Now Go to the mailbox of the specified email id and click on Subscription confirmed.
Craft the CloudWatch Alarm
Step 1: Go to the CloudWatch dashboard on the AWS management console. Click on Metrics in the left pane.
Step 2: In All metrics section click on EC2
Step 3: Click on Per-instance metrics.
Step 4: Select the instance you launched.
Step 5: Go to Graphed metrics, click on the bell icon.
Step 6: This dashboard shows the components of Amazon CloudWatch such as Namespace, Metric Name, Statistics, etc.
Set the Threshold
Step 1: Select the greater threshold. Also, specify the amount( i.e. 80 ) of the threshold value. Click on Next.
Connect the Dots
Step 1: Click on Select an existing SNS topic, also mention the name of the SNS topic you created now.
Step 2: Specify the name of alarm and description which is completely optional. Click on Next and then click on Create alarm.
Monitor your Instances using CloudWatch
Step 3: The alarm is successfully created.
Step 4: You can see the graph which notifies whenever CPU utilization goes above 80%.
Use Cases for CloudWatch
- CloudWatch can be used to monitor the performance of AWS resources, applications, and infrastructure components in real-time
- CloudWatch allows users to set up alarms that trigger notifications or automated actions in response to changes in the state of their resources.
- CloudWatch can be used to store, search, and analyze log data from various AWS services, applications, and infrastructure components.
- CloudWatch can be used to monitor the performance of EC2 instances, RDS databases, and other resources, which can then be used to trigger automatic scaling events.
Benefits of Amazon CloudWatch
- A large amount of data is produced by web applications nowadays so amazon CloudWatch acts as a dashboard that contains the organized collection of whole data.
- It improves the total cost of ownership by providing alarms and also takes automated actions when there is an error in limits provided.
- Applications and resources can be optimized by examining the logs and metric data.
- Detailed Insights from the application are provided through data like CPU utilization, capacity utilization, memory utilization, etc.
- It provides a great platform to compare and contrast the data produced by various AWS services.
Draw Backs of Amazon CloudWatch
- Cloud Watch can be expensive, especially for large-scale monitoring and logging needs.
- Cloud Watch may not be able to handle large amounts of log data, especially during spikes in usage, making it difficult to maintain a consistent level of monitoring and logging.
- The monitoring and logging processes of CloudWatch can consume significant system resources, impacting the overall performance of an application.
- Integrating CloudWatch with other AWS services and third-party tools can be challenging.
- Setting up and managing CloudWatch can be complex, especially for users who are not familiar with cloud-based systems.
Challenges of CloudWatch
- Complexity in Setup: Setting up CloudWatch monitoring and configuring alarms can be challenging, especially for users who are new to AWS. Understanding which metrics to monitor and how to interpret them effectively requires familiarity with AWS services and best practices.
- Limited Visibility and Granularity: CloudWatch provides metrics and logs at a high level, which may lack the granularity needed for detailed analysis and troubleshooting. Users may encounter difficulty in pinpointing the root cause of issues due to limited visibility into specific system components or resources.
- Cost Management: CloudWatch costs can accumulate, particularly when monitoring a large number of resources or enabling detailed logging and retention settings. Users need to carefully manage and optimize their CloudWatch configurations to avoid unexpected charges while ensuring adequate monitoring coverage.
Amazon CloudWatch Pricing
Amazon cloud watch offers different pricing as following.
- Free Tier: Amazon cloud watch offers free tier up to 7 metrics, 3 alarms and 500 custom dashboards per month and log storage up to 5 Gb per month.
- Pay-as-you-go: You will be charged according to the base charge like each metric had its base charge and log will be charged based on per gb for dashboard you will be charged according to the per dash board. You will basically charged according to how much you use.
CloudWatch vs. CloudTrail
| Aspect | CloudWatch | CloudTrail |
|---|
| Function | Monitoring and observability service for AWS resources, collecting and tracking metrics, and managing alarms. | Auditing and logging service capturing API activity and providing a history of AWS API calls for governance, compliance, and security analysis. |
| Use Cases | Monitoring performance metrics, logs, and events to troubleshoot issues, optimize resource utilization, and maintain application health. | Tracking API activity and changes to AWS resources, auditing user activity, and generating actionable insights for security analysis and compliance auditing. |
| Key Features | Metric collection, dashboards, alarms, logs, and events for real-time monitoring and automated responses. | Logging of API calls, including details such as the identity of the caller, the time of the call, the source IP address, and the request parameters. Enables analysis, compliance reporting, and troubleshooting. |