As you know, the monitoring platform Cloudwatch does NOT include disk and memory monitoring in AWS. However, AWS provides the software/tool to do it called “Cloudwatch agent”.

And, there is additional pricing involved 😉 See more: Cloudwatch Pricing. However, in May 2020, expect to pay just around 40 cents US$ a month for up to 5 metrics with 1-minute interval, i.e., not expensive. I guess, it is a “justified cost”, together with those 10 cents per alarm cost.

DifficultyMedium
ToolsNone needed
Requirement
  • Access to AWS console
  • Access to instances

So, this article is not for replacing the official documentation on disk and memory monitoring in AWS. But, this is just a big picture outline to help you through those massive docs only on this matter. See more:AWS Cloudwatch Agent Installation

And we proceed with the most straightforward route to do it. For example, installing additional “StatsD” or “collectd” tools are optionally unnecessary.

Installing CloudWatch Agent for Disk and Memory Monitoring in AWS

There are 3 separate items to setup:

  1. The credentials: all the security policy involved in the AWS ecosystem
  2. The collector: the CloudwatchAgent who collect all the metrics and information from the server
  3. The submitter: the part who submit the information collected to Cloudwatch

The credentials

  1. Create IAM credential for instance:
    • The “Cloudwatch Agent” is designed to work on either the native EC2 instances or outside server (on-prem instances)
    • For EC2 instances on AWS: create an IAM role to attach to EC2
    • For on-prem server: create an IAM user with the appropriate policy
    • The policy needed to be included for either above are: CloudWatchAgentServerPolicy and AmazonSSMManagedInstanceCore
    • See here for AWS guide on creating IAM roles for Cloudwatch Agent
  2. Attached the role or user above to the instances.
    • For EC2: Go to EC2 Dashboard, select the running instance and click Action=>=>Instance Setting=> Attached/Replace IAM Role.
    • Restart the instances (to be sure)
  3. Create an IAM user for the “submitter” who run the software on the server. Be prepared to have its access key and secret key when creating the configuration key down below.
    • The necessary policy to attached are CloudWatchAgentAdminPolicy and AmazonSSMManagedInstanceCore.

The collector: Cloudwatch Agent

AWS documentation on installing the Cloudwatch Agent is here. It is basically:

  1. Download the agent package. See here
  2. Create and Modify the CloudWatch agent configuration file and specify the metrics that you want to collect. See here
     sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
    
  3. Activate the agent
     sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a start
    
  4. Make sure /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log doesn’t contain any error.

The submitter: the software and the user who run it

  • You can choose “cwagent” (recommended) or root or your other username to collect the metrics. (This is one of the questions in the configuration wizard above).
  • In your installation directory, you have a folder named aws-script-mon. Inside it, rename the file awscreds.template to awscred.conf and insert the access key and the secret key of the collector.
  • Setup the “cwagent” (or root or your chosen collector user-id) credential
  • Make the home directory as mentioned in /etc/passwd.
  • Create the config files for cwagent. These files below would be the standard configuration if the user installed the AWS CLI. If your chosen user already has that install, you can skip this part.
  • Create the directory: mkdir .aws
  • Create the credentials file: vi ./aws/credentials
      aws_access_key_id = <key_id>
      aws_secret_access_key =<secretkey>

Also create the config file vi ./aws/config

       [default]
       region = us-west-2
  • Set the proper ownership sudo chown cwagent:cwagent /home/cwagent
  • Verify all settings are good by run below command (you can modify the metrics as you need): ~/aws-scripts-mon/mon-put-instance-data.pl --mem-used-incl-cache-buff --mem-util --mem-avail --disk-space-avail --disk-path=/ --verify
  • Then install the command above with ‘–from-cron’ option in CRON using crontab -e */5 * * * * sudo ~/aws-scripts-mon/mon-put-instance-data.pl --mem-used-incl-cache-buff --mem-util --mem-avail --disk-space-avail --disk-path=/ --from-cron

Done. Disk and Memory Monitoring in AWS is Ready.

Now we should be able to start getting the disk and memory monitoring data in the Cloudwatch. Not a simple piece, is it? Now just wait in Cloudwatch for the update.

Troubleshooting Cloudwatch Agent

Just, in case…. Because it was not too simple, some trouble might come. See this: AWS Troubleshooting. Some things you can start with:

  1. Check if the Cloudwatch agent is active sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status You should have the result as:
{   "status": "running",
    "starttime": "2017-12-12T18:41:18", 
    "version": "1.73.4"
}
  1. If it is active, check the log /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log for any error message.
  2. And also check the initial config if there is an obvious problem: /etc/init/amazon-cloudwatch-agent.conf
  3. Check any incorrect parameter in the file below against any error message given from the above command /opt/aws/amazon-cloudwatch-agent/bin/config.json
  4. If you got this mouthful error in the log, the is mostly from missing .aws/credentials in the user home directory.
      refresh EC2 Instance Tags failed: NoCredentialProviders: 
      no valid providers in chain caused by: EnvAccessKeyNotFound: 
      failed to find credentials in the environment.SharedCredsLoad: 
      failed to load profile, .EC2RoleRequestError: no EC2 instance role foundcaused by: 
      EC2MetadataError: failed to make EC2Metadata request
  1. If you got just one or 2 dots on Cloudwatch, then the cron job might not run properly. (That dot was the data when you tested the submission manually)
    • Check if cron service is active service cron status
    • Check cron’s log: could be /var/log/cron, var/log/syslog or var/log/auth.log (depend on the error and the flavor of Linux)

I hope this helps!

AWS Cloudwatch