Creating Snapshots using the AWS CLI

The AWS Command Line Interface is an interface for managing AWS services. It contains a tool that allows you to control many AWS services from command line, and because it’s a command line – you can automate your tasks with scripts.

Installing AWS CLI for Managed Services

Installing AWS CLI can be done in different ways – depending on your environment:

  1. For Windows: with MSI package available from those places:
  2. For Linux /Mac – you can install CLI using python package manager – pip:

sudo pip install awscli

or use bundled installer:

$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"

$ unzip awscli-bundle.zip

$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

Note: The bundled installer does not support installing to paths that contain spaces.

To check the install you can use this command:

aws -version

which should return information about AWS version and also python version and your platform:

awscli/1.10.8 Python/2.7.9 Windows/8 botocore/1.3.30

After installing CLI – the next step is to configure to be able to use it with your AWS account. In order to do that – you need to run

aws configure

You will be asked to put:

  • AWS Access Key
  • AWS Secret Access Key
  • Default Region name
  • Default output format

You don’t have to put default region now but when executing commands – you will need to provide region you want to connect to with –region param.

Now you are ready to use Amazon Command Line Interface to work with your amazon resources.

With CLI you can do most of the things that you can do within web console. To see a help with available options and also to list available services that you can interact with type in:

aws help

To check all options for particular service – just type

aws <service> help

For example:

aws ec2 help

will show you all options available to use with EC2 service.

Also, you can go deeper and check help pages for particular command within service:

aws <service> <command> help

For example:

aws ec2 describe-images help

How can we use CLI – examples:

The best part about AWS CLI  and also about other Amazon client libraries that you can use – if you have a coding backgroud (as myself) is that it is very consistent across all of them. For example – if you ever used php to interact with your s3 bucket your code would look like that:

use Aws\S3\S3Client;

 $client = S3Client::factory(array(
      'key' =>"your key",
      'secret' => "your secret"
    ));
 $client->putObject(
        [
          'Bucket' => "bucket_name",
          'Key' => "example-folder/myfile.txt",
          'ACL' => 'public-read',
          'Body' => $your_file_body,
          'ContentType' => "text/plain"
        ]
      );

If you’re doing the same operation from CLI the command will look like that:

aws s3api put-object --bucket bucket_name --key example-folder/myfile.txt --body myfile.txt --acl public-read --content-type text/plain

As you can see experience within one type of client application/library might save you a lot searching for a commands that you need to use to perform AWS operations.

I’ve selected a couple of basic operations that can be performed with Amazon Command Line Interface.

Creating an S3 bucket:

aws s3api create-bucket --bucket this-is-buckeeettttt

Output:

{
    "Location": "/this-is-buckeeettttt"
}

Uploading file to a bucket:

aws s3api put-object --bucket this-is-buckeeettttt --key my_files/my_file.txt --body my_file.json --acl public-read --content-type application/json

Output:

{
    "ETag": "\"d63b8a5de28a3a86a9d36b63ae0291bb\""
}

Listing s3 buckets:

aws s3api list-buckets
{
    "Owner": {
        "DisplayName": "cloudranger-text",
        "ID": "a9b79fb518f9d0c68bd1fa19191bedbb94bf7a1f386b9678ee8f189fac98a59f"
    },
    "Buckets": [
        {
            "CreationDate": "2016-02-18T09:33:35.000Z",
            "Name": "adfsdafdsssjfkljfsdluerfsfds"
        },

        {
            "CreationDate": "2016-02-27T12:57:26.000Z",
            "Name": "this-is-buckeeettttt"
        }
    ]
}

Listing objects in bucket:

aws s3api list-objects --bucket this-is-buckeeettttt

Output:

{
    "Contents": [
        {
            "LastModified": "2016-02-27T13:15:55.000Z",
            "ETag": "\"d63b8a5de28a3a86a9d36b63ae0291bb\"",
            "StorageClass": "STANDARD",
            "Key": "my_files/my_file.txt",
            "Owner": {
                "DisplayName": "dave+testaccount",
                "ID": "a9b79fb518f9d0c68bd1fa19191bedbb94bf7a1f386b9678ee8f189fac98a59f"
            },
            "Size": 28565
        }
    ]
}

Retrieving object from s3 to a disk:

aws s3api get-object --bucket this-is-buckeeettttt --key my_files/my_file.txt my_file_downloaded.txt

Output

{
    "AcceptRanges": "bytes",
    "ContentType": "application/jsonn",
    "LastModified": "Sat, 27 Feb 2016 13:15:55 GMT",
    "ContentLength": 28565,
    "ETag": "\"d63b8a5de28a3a86a9d36b63ae0291bb\"",
    "Metadata": {}
}

Deleting object from s3:

aws s3api delete-object --bucket this-is-buckeeettttt --key my_files/my_file.txt

This command doesn’t have any response – even if there is no object already. You will get an error if for example, the bucket doesn’t exist or it can’t connect – which  is standard across the CLI.

To delete multiple objects use

 aws s3api delete-objects 

command.

For example:

aws s3api delete-objects --bucket this-is-buckeeettttt --delete file://files_to_delete.json

files_to_delete.json is a list of files to delete

 {
     "Objects": [
       {
         "Key": "test1.txt"
       }
     ],
     "Quiet": false
   }

Output:

   {
       "Deleted": [
           {
               "DeleteMarkerVersionId": "mYAT5Mc6F7aeUL8SS7FAAqUPO1koHwzU",
               "Key": "test1.txt",
               "DeleteMarker": true
           }
       ]
   }

Listing snapshots:

aws ec2 describe-snapshots

Example output:

     [

        {

            "Description": "My machine backup snapshot 2",
            "Encrypted": false,
            "VolumeId": "vol-xxxxx",
            "State": "completed",
            "VolumeSize": 1,
            "Progress": "100%",
            "StartTime": "2016-02-11T23:10:43.000Z",
            "SnapshotId": "snap-xxxxxx",
            "OwnerId": "123456"

        },

        {

            "Description": "My machine backup snapshot 1",
            "Encrypted": false,
            "VolumeId": "vol-xxxxxx",
            "State": "completed",
            "VolumeSize": 1,
            "Progress": "100%",
            "StartTime": "2016-02-13T19:01:36.000Z",
            "SnapshotId": "snap-xxxxxx",
            "OwnerId": "123456"

        }
    ]

This command also will list a images that are available for you from Amazon as a source to create new machines.

Creating snapshot:

aws ec2 create-snapshot vol-123456aa --description “My machine backup snapshot 3”

That will output:

     [

        {

            "Description": "My machine backup snapshot 3",
            "Encrypted": false,
            "VolumeId": "vol-123456aa",
            "State": "pending",
            "VolumeSize": 1,
            "Progress": "10%",
            "StartTime": "2016-02-29T19:01:36.000Z",
            "SnapshotId": "snap-xxxxxx",
            "OwnerId": "123456"

        }
    ]
Alternatives to scripting

As we have shown above, you can use scripts to take scheduled snapshots and automate operations, but sometimes custom scripts are simply not sustainable or practical for organisations on a long-term basis.

When you start to scale up your backup operations, you require an AWS backup solution that will provide you with more control of your automated backup processes. For example, you may need various backup frequencies, and varying retention periods for your AWS snapshots etc. Therefore, you would benefit more from a policy-based system that allows you to control your specific business requirements.

About CloudRanger

CloudRanger are specialists in automation and scheduling for your Amazon Web Services (AWS) cloud resources. Easily schedule your AWS cloud servers to start, stop and manage snapshots automatically. We also offer a range of cloud consulting services that are focused on helping small and medium businesses move to Amazon Web Services to gain strategic advantage for their business through lower costs or scalability.

We offer a 14-day free trial, so you can try out our services for yourselves. This will help you get a feel for what we do, and how we can help.

Steven

This entry has 0 replies

Comments are closed.