In this article, we will see how we can add and use customer metrics in AWS CloudWatch for EC2 instances. There are several metrics that are available when an EC2 instance is launched. These metrics can be monitored on a host level. But what if they are not monitoring what you are interested in?

Well, you have the possibility to add your own metrics, such as memory, swap and disk utilization, availability, and usage, that can be monitored at system level (operating system level). These metrics might provide you with better monitoring of the EC2 instances.

VMware Training – Resources (Intense)

The monitoring scripts can be installed for Linux or Windows operating systems and they are written in Perl. In order to make the scripts work, additional Perl modules needs to be installed on the EC2 instance.

So let’s see how this is done.

First before we enable the monitoring scripts, let’s check what metrics types we have in the AWS CloudWatch console:

As said earlier, we will use an IAM role that will be assigned to the EC2 instance. In this way, we will have full access to the EC2 instance without providing the AWS credentials.

From the IAM console, select “Roles” and then “Create New Role”:

Enter the name of the role:

Next, we will need to specify the role type. Because this will be applicable to EC2 instances, we will select “Amazon EC2”:

Next, we need to attach a policy. Because this policy will allow access to CloudWatch, we need to look for a policy that allows access to the CloudWatch metrics. To avoid any complication about the access, we will allow full access:

Review the role information and go ahead and create the role:

Now the role is created:

It’s time to create the EC2 instance whose disk/memory/swap metrics will be monitored by CloudWatch.

The EC2 launch process is the regular one with the exception that on Step 3, where instance details are provided, the role with which the EC2 will be launched needs to be specified:

The rest is the same and shortly the instance will be available for you to connect and continue with the scripts installation and configuration.

Let’s connect to the EC2 instance:

lab@UBUNTU:~/AWS$ ssh -i "AMAZON_LINUX.pem" ec2-user@52.3.129.99
The authenticity of host '52.3.129.99 (52.3.129.99)' can't be established.
ECDSA key fingerprint is 2d:8b:0a:cb:02:a9:28:a9:f4:13:eb:31:35:bc:e2:89.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '52.3.129.99' (ECDSA) to the list of known hosts.

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2015.03-release-notes/
32 package(s) needed for security, out of 64 available
Run "sudo yum update" to apply all updates.
[ec2-user@ip-172-31-60-16 ~]$

Next we need to install some Perl modules so that the script can work:

[ec2-user@ip-172-31-60-16 ~]$ sudo yum install perl-LWP-Protocol-https perl-Sys-Syslog perl-DateTime
Loaded plugins: priorities, update-motd, upgrade-helper
amzn-main/latest                                                                                                                             | 2.1 kB     00:00
amzn-updates/latest                                                                                                                          | 2.3 kB     00:00
Resolving Dependencies

===== output cut for brevity =====

--> Running transaction check



  perl-Try-Tiny.noarch 0:0.12-2.5.amzn1               perl-URI.noarch 0:1.60-9.8.amzn1                   perl-WWW-RobotRules.noarch 0:6.02-5.12.amzn1
  perl-libwww-perl.noarch 0:6.05-2.17.amzn1

Complete!
[ec2-user@ip-172-31-60-16 ~]$

Next, the scripts need to be downloaded from AWS:

[ec2-user@ip-172-31-60-16 ~]$ wget http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip
--2015-08-01 10:56:55--  http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip
Resolving aws-cloudwatch.s3.amazonaws.com (aws-cloudwatch.s3.amazonaws.com)... 54.231.1.145
Connecting to aws-cloudwatch.s3.amazonaws.com (aws-cloudwatch.s3.amazonaws.com)|54.231.1.145|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24089 (24K) [application/zip]
Saving to: ‘CloudWatchMonitoringScripts-1.2.1.zip’

CloudWatchMonitoringScripts-1.2.1.zip    100%[====================================================================================>]  23.52K  --.-KB/s   in 0.001s

2015-08-01 10:56:55 (32.2 MB/s) - ‘CloudWatchMonitoringScripts-1.2.1.zip’ saved [24089/24089]

[ec2-user@ip-172-31-60-16 ~]$

Decompress the archive:

[ec2-user@ip-172-31-60-16 ~]$ unzip CloudWatchMonitoringScripts-1.2.1.zip
Archive:  CloudWatchMonitoringScripts-1.2.1.zip
 extracting: aws-scripts-mon/awscreds.template
  inflating: aws-scripts-mon/AwsSignatureV4.pm
  inflating: aws-scripts-mon/CloudWatchClient.pm
  inflating: aws-scripts-mon/LICENSE.txt
  inflating: aws-scripts-mon/mon-get-instance-stats.pl
  inflating: aws-scripts-mon/mon-put-instance-data.pl
  inflating: aws-scripts-mon/NOTICE.txt
[ec2-user@ip-172-31-60-16 ~]$

And let’s see what is inside the archive:

[ec2-user@ip-172-31-60-16 ~]$ cd aws-scripts-mon/
[ec2-user@ip-172-31-60-16 aws-scripts-mon]$ ls -l
total 96
-rw-r--r-- 1 ec2-user ec2-user    30 Mar  6 22:57 awscreds.template
-r--r--r-- 1 ec2-user ec2-user 17021 Mar  6 22:57 AwsSignatureV4.pm
-r--r--r-- 1 ec2-user ec2-user 22487 Mar  6 22:57 CloudWatchClient.pm
-rw-r--r-- 1 ec2-user ec2-user  9124 Mar  6 22:57 LICENSE.txt
-rwxr-xr-x 1 ec2-user ec2-user  9739 Mar  6 22:57 mon-get-instance-stats.pl
-rwxr-xr-x 1 ec2-user ec2-user 18144 Mar  6 22:57 mon-put-instance-data.pl
-rw-r--r-- 1 ec2-user ec2-user   138 Mar  6 22:57 NOTICE.txt
[ec2-user@ip-172-31-60-16 aws-scripts-mon]$

There are two scripts: “mon-put-instance-data.pl” will collect the data from the EC2 instance and then it will push it to CloudWatch; “mon-get-instance-stats.pl” will retrieve the same metrics from the CloudWatch and present them in a simple way.

We will use the scripts to monitor the disk (the disk availability, utilization and usage). But first we need to specify what will be the device where the disk is mounted and on what unit to report the metric.

To find out what are the disk devices we have, you can use this command:

[ec2-user@ip-172-31-60-16 aws-scripts-mon]$ lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0   8G  0 disk
└─xvda1 202:1    0   8G  0 part /
[ec2-user@ip-172-31-60-16 aws-scripts-mon]$

So in our case, it will be /dev/xvda1.

Let’s check the current disk availability:

[ec2-user@ip-172-31-60-16 aws-scripts-mon]$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      7.8G  1.2G  6.6G  15% /
devtmpfs        490M   56K  490M   1% /dev
tmpfs           499M     0  499M   0% /dev/shm
[ec2-user@ip-172-31-60-16 aws-scripts-mon]$

We have 6.6GB, that is 6600MB, available.

The scripts allow running in the dry mode to test the command and not send the data to CloudWatch. You need to add “–verify” as parameter.

Ok, so let’s run the script and push the data to CloudWatch:

[ec2-user@ip-172-31-60-16 aws-scripts-mon]$ /home/ec2-user/aws-scripts-mon/mon-put-instance-data.pl --disk-path=/dev/xvda1 --disk-space-avail --disk-space-util --disk-space-used --disk-space-units=megabytes

Successfully reported metrics to CloudWatch. Reference Id: 2d407f1b-383d-11e5-ba84-c5cdfd57eba2

[ec2-user@ip-172-31-60-16 aws-scripts-mon]$

This means that the script reports the data in megabytes for /dev/xvda1 for disk availability, disk usage, and disk utilization.

If we go back to the CloudWatch console and see the metrics again, we will see that a new category is showing up:

And the new category has three metrics:

By using the script as we did above, we will get only one value of the metrics; we will get more useful data if the script is running at determined intervals so we will know how the metrics vary.

For this, we need to use crontab and add the script there to run at the desired intervals.

To edit the crontab file, use this command:

[ec2-user@ip-172-31-60-16 aws-scripts-mon]$ crontab -e

Then add the following line in order to run the script every minute of every day of the week of every month:

*/1 * * * * /home/ec2-user/aws-scripts-mon/mon-put-instance-data.pl --disk-path=/dev/xvda1 --disk-space-avail --disk-space-util --disk-space-used --disk-space-units=megabytes

I left the script running for 11 minutes during which I added files to the disk and deleted them. You can see how the available disk varied during these 11 minutes:

We can retrieve the metrics that we pushed to CloudWatch using the other script. Because we pushed only disk space metrics, we can retrieve only those:

[ec2-user@ip-172-31-60-16 aws-scripts-mon]$ /home/ec2-user/aws-scripts-mon/mon-get-instance-stats.pl

Instance i-25dadb8d statistics for the last 1 hour.

CPU Utilization
    Average: 1.73%, Minimum: 0.00%, Maximum: 24.33%

Memory Utilization
    Average: N/A, Minimum: N/A, Maximum: N/A

Swap Utilization
    Average: N/A, Minimum: N/A, Maximum: N/A

Disk Space Utilization for /dev/xvda1 mounted on /
    Average: 25.91%, Minimum: 14.71%, Maximum: 53.46%

[ec2-user@ip-172-31-60-16 aws-scripts-mon]$

And this will be all about monitoring scripts for EC2 instances. Now you should know what monitoring scripts are, how you can use them to monitor system level metrics that are not added by default by CloudWatch when an instance is launched.

References