This series will discuss the AWS Command Line Interface (CLI). AWS CLI is a tool that can be used to configure and manage different AWS services. It allows users to automate different tasks using scripts. Not all AWS services support AWS CLI, but many of them do and are actually the most used ones.

This first part of the series will discuss how to:

  • Install AWS CLI
  • Configure AWS CLI
  • Use parameters and options
  • Control the output returned by AWS CLI

VMware Training – Resources (Intense)

Installing AWS CLI

As you might have already figured out, the first step is to install AWS CLI. The installation method varies depending on the type of instances where you plan to install AWS CLI. For instance, if it’s Amazon Linux AMI, then the tool comes preinstalled. You can go ahead and start using it without doing anything else.

If you are installing AWS CLI on Windows instances, then you would have to use the MSI installer. As with any Windows installation, this is pretty straightforward. Just click ‘Next’, ‘Next’ and you are done.

Another method of installing AWS CLI is by using pip utility. Pip is a Python utility used to install, upgrade and remove Python packages. Amazon recommends that you use this method to install AWS CLI on Linux and MacOS.

The last method, and another way to install AWS CLI on Linux/MacOS, is to use the bundled installer. This method is simple because you only have to deal with installing the bundle package that contains the AWS CLI tool.

So let’s come back to our chosen method, by using pip. These are the steps to install AWS CLI using pip:

1. Install Python in case you don’t have it. To check, use this command:
[ec2-user@ip-172-31-21-197 ~]$ python
Python 2.7.5 (default, Feb 11 2014, 07:46:25)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
[ec2-user@ip-172-31-21-197 ~]$
2. Download and install pip:
[ec2-user@EC2-REDHAT-01 ~]$ wget https://bootstrap.pypa.io/get-pip.py
--2014-09-05 11:22:44--  https://bootstrap.pypa.io/get-pip.py
Resolving bootstrap.pypa.io (bootstrap.pypa.io)... 199.27.76.175
Connecting to bootstrap.pypa.io (bootstrap.pypa.io)|199.27.76.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1340903 (1.3M) [text/x-python]
Saving to: âget-pip.pyâ

100%[=======================================================================================================================================>] 1,340,903   --.-K/s   in 0.1s

2014-09-05 11:22:45 (12.5 MB/s) - âget-pip.pyâ saved [1340903/1340903]

[ec2-user@EC2-REDHAT-01 ~]$ sudo python get-pip.py
Downloading/unpacking pip
  Downloading pip-1.5.6-py2.py3-none-any.whl (1.0MB): 1.0MB downloaded
Installing collected packages: pip
Successfully installed pip
Cleaning up...
[ec2-user@EC2-REDHAT-01 ~]$
3. Install AWS CLI:
[ec2-user@EC2-REDHAT-01 ~]$ sudo pip install awscli
Downloading/unpacking awscli
  Downloading awscli-1.4.2.tar.gz (239kB): 239kB downloaded
  Running setup.py (path:/tmp/pip_build_root/awscli/setup.py) egg_info for package awscli

===== The output is verbose because other packages are installed together with AWS CLI so I’m removing most of it and keeping only the beginning and the end confirming the correct installation. =====

Successfully installed awscli botocore bcdoc colorama docutils rsa jmespath python-dateutil pyasn1
Cleaning up...
[ec2-user@EC2-REDHAT-01 ~]$

And those are the steps to install AWS CLI on Redhat. The installation procedure is similar for any Linux operating system or MacOS.

Although it’s not part of the installation, the next thing you should do after you installed AWS CLI would be to confirm that it has been correctly installed. You can check this by using:

aws help

If everything is correct, the help for ‘aws’ command will be displayed.

Configuring AWS CLI

Let’s say that you want to go ahead and use this tool and that you would like to see what EC2 instances you have running in AWS cloud. The command would be ‘aws ec2 describe-intances’. If you didn’t configure AWS CLI, you would get this error:

[ec2-user@EC2-REDHAT-01 ~]$ aws ec2 describe-instances
Unable to locate credentials. You can configure credentials by running "aws configure".
[ec2-user@EC2-REDHAT-01 ~]$

So what are these credentials? When you set up a new account in AWS, each user receives an access key made up of an access key ID and a secret access key. You should make sure that nobody has access to them. If they get lost, you can generate another one.

What happened when you executed the command in our example is that AWS tried to use the credentials but wasn’t able to find them. AWS CLI looks for the credentials and configuration, and these can be in multiple places. That’s why there is a specific order in which different locations are checked for credentials and configuration presence.

This is the order:

  • * Environment variables: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
  • * AWS credentials profile file
  • * CLI configuration file
  • * Instance profile credentials

Two other things that need to be configured besides credentials are the region name and the output format. The default response of the AWS CLI command is in JSON format. You might want to change it to ASCII table format. We will see later the differences between the two.

So, there are multiple ways to set the credentials. The easiest and recommended one would be to use this command:

aws configure

This is an interactive command which will ask you for the access ID and secret access key. What it actually does is it writes these two values to the “~/.aws/config” file on Linux or the “C:\Users\USERNAME\.aws\config” file on Windows. This corresponds to method three from above.

You can also configure the region using “aws configure” as well. This is how you can configure the credentials and region name:

[ec2-user@EC2-REDHAT-01 ~]$ aws configure
AWS Access Key ID [None]: AKIAISYAZSOA63BDNBIA
AWS Secret Access Key [None]: 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
Default region name [None]:
Default output format [None]:
Default region name [None]: us-east-1
Default output format [None]: json
[ec2-user@EC2-REDHAT-01 ~]$

Using Parameters and Options

At this point you have installed and properly configured AWS CLI. It’s time to see what it can do.

The format of AWS CLI is ‘aws

As you can see, it always starts with ‘aws’.

To see what commands you can use with ‘aws’ command, you can type anything as the command and the possible commands you can use will be returned:

[ec2-user@EC2-REDHAT-01 ~]$ aws ?
usage: aws [options] 

The same can be used for subcommands.

It’s possible to get detailed information about options and parameters of an AWS command. For instance, this command will give you detailed information about the command as well as descriptions of the parameters:

aws ec2 describe-tags help

How do you pass parameters to AWS CLI? You can do it like this:

aws ec2 create-tags --resources i-da73a431 --tags Key=Name,Value=EC2_AMAZON_LINUX

This changes the tag for one instance. Before the command was run, this was the tag of the instance:

{
            "ResourceType": "instance",
            "ResourceId": "i-da73a431",
            "Value": "EC2_AMAZON_AMI_LINUX_01",
            "Key": "Name"
        },

After the command was run, this was the output:

{
            "ResourceType": "instance",
            "ResourceId": "i-da73a431",
            "Value": "EC2_AMAZON_LINUX",
            "Key": "Name"
        },

You can have multiple parameters passed to the CLI. For instance, you want to find all Windows-based instances for which monitoring is enabled. I’m running two Windows instances and the monitoring is enabled only for one of them. You can do so below. I used ‘grep’ to match on ‘Platform’ string to confirm that I will get only one string.

[ec2-user@EC2-REDHAT-01 ~]$ aws ec2 describe-instances --filters "Name=platform,Values=windows" "Name=monitoring-state,Values=enabled" | grep Platf
                    "Platform": "windows",
[ec2-user@EC2-REDHAT-01 ~]$

If I used the second filter criteria, the returned output would have been:

[ec2-user@EC2-REDHAT-01 ~]$ aws ec2 describe-instances --filters "Name=platform,Values=windows" | grep Platf
                    "Platform": "windows",
                    "Platform": "windows",
[ec2-user@EC2-REDHAT-01 ~]$

Controlling the Output

I mentioned in the beginning of the article about the output format that is returned. By default, it’s set to ‘JSON’ but you can change it to either ‘text’ (delimited text by TAB) or ‘table’ (ASCII table).

Why would you use one over the other? There are many reasons. For instance, some operations require that the data be formatted in JSON format, which can be decoded easily by many programming languages.

You could use ‘text’ format because it works well with UNIX tools like grep, awk and sed.

The ‘table’ format can be read the easiest by humans though, as you will see in the example.

How do you switch between these formats? You can do it by running ‘aws configure’ again or by specifying the parameter ‘–output’ and then one of the three possibilities mentioned above.

Let’s see how the output looks like in different output formats when we are trying to get all the tags.

This is for ‘json’:

[ec2-user@EC2-REDHAT-01 ~]$ aws ec2 describe-tags
{
    "Tags": [
        {
            "ResourceType": "volume",
            "ResourceId": "vol-78522c31",
            "Value": "VOL_WINDOWS",
            "Key": "Name"
        },
        {
            "ResourceType": "volume",
            "ResourceId": "vol-6f146a26",
            "Value": "VOL_LINUX",
            "Key": "Name"
        },
        {
            "ResourceType": "instance",
            "ResourceId": "i-da73a431",
            "Value": "EC2_AMAZON_AMI_LINUX_01",
            "Key": "Name"
        },
        {
            "ResourceType": "instance",
            "ResourceId": "i-158c5afe",
            "Value": "EC2_UBUNTU_01",
            "Key": "Name"
        },
        {
            "ResourceType": "instance",
            "ResourceId": "i-d5895f3e",
            "Value": "EC2_REDHAT_01",
            "Key": "Name"
        },
        {
            "ResourceType": "instance",
            "ResourceId": "i-68126343",
            "Value": "EC2-Linux",
            "Key": "Name"
        },
        {
            "ResourceType": "instance",
            "ResourceId": "i-aa71a641",
            "Value": "EC2_WINDOWS_2012_01",
            "Key": "Name"
        },
        {
            "ResourceType": "instance",
            "ResourceId": "i-0fbfce24",
            "Value": "EC2-Windows",
            "Key": "Name"
        },
        {
            "ResourceType": "image",
            "ResourceId": "ami-76817c1e",
            "Value": "EC2_AMAZON_LINUX",
            "Key": "Name"
        }
    ]
}
[ec2-user@EC2-REDHAT-01 ~]$

This is for ‘text’ format:

[ec2-user@EC2-REDHAT-01 ~]$ aws ec2 describe-tags --output text
TAGS    Name    ami-76817c1e    image   EC2_AMAZON_LINUX
TAGS    Name    i-da73a431      instance        EC2_AMAZON_AMI_LINUX_01
TAGS    Name    i-158c5afe      instance        EC2_UBUNTU_01
TAGS    Name    i-d5895f3e      instance        EC2_REDHAT_01
TAGS    Name    i-68126343      instance        EC2-Linux
TAGS    Name    i-aa71a641      instance        EC2_WINDOWS_2012_01
TAGS    Name    i-0fbfce24      instance        EC2-Windows
TAGS    Name    vol-78522c31    volume  VOL_WINDOWS
TAGS    Name    vol-6f146a26    volume  VOL_LINUX
[ec2-user@EC2-REDHAT-01 ~]$

And this is for ‘table’ format:

[ec2-user@EC2-REDHAT-01 ~]$ aws ec2 describe-tags --output table
-----------------------------------------------------------------------
|                            DescribeTags                             |
+---------------------------------------------------------------------+
||                               Tags                                ||
|+------+---------------+---------------+----------------------------+|
||  Key |  ResourceId   | ResourceType  |           Value            ||
|+------+---------------+---------------+----------------------------+|
||  Name|  vol-78522c31 |  volume       |  VOL_WINDOWS               ||
||  Name|  vol-6f146a26 |  volume       |  VOL_LINUX                 ||
||  Name|  ami-76817c1e |  image        |  EC2_AMAZON_LINUX          ||
||  Name|  i-da73a431   |  instance     |  EC2_AMAZON_AMI_LINUX_01   ||
||  Name|  i-158c5afe   |  instance     |  EC2_UBUNTU_01             ||
||  Name|  i-d5895f3e   |  instance     |  EC2_REDHAT_01             ||
||  Name|  i-68126343   |  instance     |  EC2-Linux                 ||
||  Name|  i-aa71a641   |  instance     |  EC2_WINDOWS_2012_01       ||
||  Name|  i-0fbfce24   |  instance     |  EC2-Windows               ||
|+------+---------------+---------------+----------------------------+|
[ec2-user@EC2-REDHAT-01 ~]$

One very useful feature is ‘–query’. To understand what it does, take another look at the JSON output for all the tags and then use this command to filter only the first tag:

[ec2-user@EC2-REDHAT-01 ~]$ aws ec2 describe-tags --query 'Tags[0]'
{
    "ResourceType": "instance",
    "ResourceId": "i-da73a431",
    "Value": "EC2_AMAZON_AMI_LINUX_01",
    "Key": "Name"
}
[ec2-user@EC2-REDHAT-01 ~]$

Using ”Tags[*]” would be just like not using the ‘–query’ parameter. It matches everything.

As another example, let’s say that you want to see all the tags’ values and to what type of resources they are attached. You can use this command:

[ec2-user@EC2-REDHAT-01 ~]$ aws ec2 describe-tags --query 'Tags[*].{Type:ResourceType,Value:Value}'
[
    {
        "Type": "instance",
        "Value": "EC2_AMAZON_AMI_LINUX_01"
    },
    {
        "Type": "instance",
        "Value": "EC2_UBUNTU_01"
    },
    {
        "Type": "instance",
        "Value": "EC2_REDHAT_01"
    },
    {
        "Type": "instance",
        "Value": "EC2-Linux"
    },
    {
        "Type": "instance",
        "Value": "EC2_WINDOWS_2012_01"
    },
    {
        "Type": "instance",
        "Value": "EC2-Windows"
    },
    {
        "Type": "image",
        "Value": "EC2_AMAZON_LINUX"
    },
    {
        "Type": "volume",
        "Value": "VOL_WINDOWS"
    },
    {
        "Type": "volume",
        "Value": "VOL_LINUX"
    }
]
[ec2-user@EC2-REDHAT-01 ~]$

You can play around with this ‘–query’ option as there are many other things that you can filter out.

This brings us to the end of the first part of the series. In this part we discussed the basics of AWS CLI, how to install it, how to configure it and how to use its most common features.

In the second part of the series, we will discuss how you can use AWS CLI to work with different services from AWS.

References

  1. Getting Set Up with the AWS Command Line Interface(link to http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html)
  2. Using the AWS Command Line Interface (link to http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-using.html)