Creating Snapshots using the AWS CLI

The AWS Command Line Interface is an interface for managing AWS services. It contains a tool that allows you to control many AWS services from command line, and because it’s a command line – you can automate your tasks with scripts.

Installing AWS CLI for Managed Services

Installing AWS CLI can be done in different ways – depending on your environment:

  1. For Windows: with MSI package available from those places:
  2. For Linux /Mac – you can install CLI using python package manager – pip:

sudo pip install awscli

or use bundled installer:

$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"

$ unzip awscli-bundle.zip

$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

Note: The bundled installer does not support installing to paths that contain spaces.

To check the install you can use this command:

aws -version

which should return information about AWS version and also python version and your platform:

awscli/1.10.8 Python/2.7.9 Windows/8 botocore/1.3.30

After installing CLI – the next step is to configure to be able to use it with your AWS account. In order to do that – you need to run

aws configure

You will be asked to put:

  • AWS Access Key
  • AWS Secret Access Key
  • Default Region name
  • Default output format

You don’t have to put default region now but when executing commands – you will need to provide region you want to connect to with –region param.

Now you are ready to use Amazon Command Line Interface to work with your Amazon resources.

With CLI you can do most of the things that you can do within web console. To see a help with available options and also to list available services that you can interact with type in:

aws help

To check all options for particular service – just type

aws <service> help

For example:

aws ec2 help

will show you all options available to use with EC2 service.

Also, you can go deeper and check help pages for particular command within service:

aws <service> <command> help

For example:

aws ec2 describe-images help

How can we use CLI – examples:

The best part about AWS CLI  and also about other Amazon client libraries that you can use – if you have a coding backgroud (as myself) is that it is very consistent across all of them. For example – if you ever used php to interact with your s3 bucket your code would look like that:

use Aws\S3\S3Client;

 $client = S3Client::factory(array(
      'key' =>"your key",
      'secret' => "your secret"
    ));
 $client->putObject(
        [
          'Bucket' => "bucket_name",
          'Key' => "example-folder/myfile.txt",
          'ACL' => 'public-read',
          'Body' => $your_file_body,
          'ContentType' => "text/plain"
        ]
      );

If you’re doing the same operation from CLI the command will look like that:

aws s3api put-object --bucket bucket_name --key example-folder/myfile.txt --body myfile.txt --acl public-read --content-type text/plain

As you can see experience within one type of client application/library might save you a lot searching for a command that you need to use to perform AWS operations.

I’ve selected a couple of basic operations that can be performed with Amazon Command Line Interface.

Creating an S3 bucket:

aws s3api create-bucket --bucket this-is-buckeeettttt

Output:

{
    "Location": "/this-is-buckeeettttt"
}

Uploading file to a bucket:

aws s3api put-object --bucket this-is-buckeeettttt --key my_files/my_file.txt --body my_file.json --acl public-read --content-type application/json

Output:

{
    "ETag": "\"d63b8a5de28a3a86a9d36b63ae0291bb\""
}

Listing s3 buckets:

aws s3api list-buckets
{
    "Owner": {
        "DisplayName": "cloudranger-text",
        "ID": "a9b79fb518f9d0c68bd1fa19191bedbb94bf7a1f386b9678ee8f189fac98a59f"
    },
    "Buckets": [
        {
            "CreationDate": "2016-02-18T09:33:35.000Z",
            "Name": "adfsdafdsssjfkljfsdluerfsfds"
        },

        {
            "CreationDate": "2016-02-27T12:57:26.000Z",
            "Name": "this-is-buckeeettttt"
        }
    ]
}

Listing objects in bucket:

aws s3api list-objects --bucket this-is-buckeeettttt

Output:

{
    "Contents": [
        {
            "LastModified": "2016-02-27T13:15:55.000Z",
            "ETag": "\"d63b8a5de28a3a86a9d36b63ae0291bb\"",
            "StorageClass": "STANDARD",
            "Key": "my_files/my_file.txt",
            "Owner": {
                "DisplayName": "dave+testaccount",
                "ID": "a9b79fb518f9d0c68bd1fa19191bedbb94bf7a1f386b9678ee8f189fac98a59f"
            },
            "Size": 28565
        }
    ]
}

Retrieving object from s3 to a disk:

aws s3api get-object --bucket this-is-buckeeettttt --key my_files/my_file.txt my_file_downloaded.txt

Output

{
    "AcceptRanges": "bytes",
    "ContentType": "application/jsonn",
    "LastModified": "Sat, 27 Feb 2016 13:15:55 GMT",
    "ContentLength": 28565,
    "ETag": "\"d63b8a5de28a3a86a9d36b63ae0291bb\"",
    "Metadata": {}
}

Deleting object from s3:

aws s3api delete-object --bucket this-is-buckeeettttt --key my_files/my_file.txt

This command doesn’t have any response – even if there is no object already. You will get an error if, for example, the bucket doesn’t exist or it can’t connect – which is standard across the CLI.

To delete multiple objects use

 aws s3api delete-objects 

command.

For example:

aws s3api delete-objects --bucket this-is-buckeeettttt --delete file://files_to_delete.json

files_to_delete.json is a list of files to delete

 {
     "Objects": [
       {
         "Key": "test1.txt"
       }
     ],
     "Quiet": false
   }

Output:

   {
       "Deleted": [
           {
               "DeleteMarkerVersionId": "mYAT5Mc6F7aeUL8SS7FAAqUPO1koHwzU",
               "Key": "test1.txt",
               "DeleteMarker": true
           }
       ]
   }

Listing snapshots:

aws ec2 describe-snapshots

Example output:

     [

        {

            "Description": "My machine backup snapshot 2",
            "Encrypted": false,
            "VolumeId": "vol-xxxxx",
            "State": "completed",
            "VolumeSize": 1,
            "Progress": "100%",
            "StartTime": "2016-02-11T23:10:43.000Z",
            "SnapshotId": "snap-xxxxxx",
            "OwnerId": "123456"

        },

        {

            "Description": "My machine backup snapshot 1",
            "Encrypted": false,
            "VolumeId": "vol-xxxxxx",
            "State": "completed",
            "VolumeSize": 1,
            "Progress": "100%",
            "StartTime": "2016-02-13T19:01:36.000Z",
            "SnapshotId": "snap-xxxxxx",
            "OwnerId": "123456"

        }
    ]

This command also will list a images that are available for you from Amazon as a source to create new machines.

Creating snapshot:

aws ec2 create-snapshot vol-123456aa --description “My machine backup snapshot 3”

That will output:

     [

        {

            "Description": "My machine backup snapshot 3",
            "Encrypted": false,
            "VolumeId": "vol-123456aa",
            "State": "pending",
            "VolumeSize": 1,
            "Progress": "10%",
            "StartTime": "2016-02-29T19:01:36.000Z",
            "SnapshotId": "snap-xxxxxx",
            "OwnerId": "123456"

        }
    ]
Alternatives to scripting

As we have shown above, you can use scripts to take scheduled snapshots and automate operations, but sometimes custom scripts are simply not sustainable or practical for organisations on a long-term basis.

When you start to scale up your backup operations, you require an AWS backup solution that will provide you with more control of your automated backup processes. For example, you may need various backup frequencies, and varying retention periods for your AWS snapshots etc. Therefore, you would benefit more from a policy-based system that allows you to control your specific business requirements.

About CloudRanger

CloudRanger provides the world’s easiest to use backup and recovery solution for Amazon Web Services. We make it easy to manage your backups & servers running on AWS cloud. Using CloudRanger, you can easily manage backups and retention of your RDS, EC2 and Redshift resources with snapshots and AMIs.

With an easy to use interface, managing your routine AWS tasks is simple and effective. CloudRanger saves your team time and hassle, making the day-to-day management of your AWS resources easier and more automated. CloudRanger can also help you save on your EC2 costs by starting/stopping non-production instances automatically when you need them.

Try CloudRanger for Free

Whether you are a small business, a medium business, or an enterprise user of Amazon Web Services, CloudRanger has the backup and recovery features that will meet your specific needs. Our easy-to-use AWS cloud management system is simple to install, which means your business can be up and running with CloudRanger in no time. And best of all, you can even try CloudRanger for free for 14 days. So why not experience all the cost-saving features and time-saving benefits CloudRanger has to offer?

 

cloudranger

 

 

Steven

This entry has 0 replies

Comments are closed.