AWS Archive

awsarchive is a Python toolkit for archiving files using AWS services.

The goal of this project is to provide best practice methods for archiving files using the products provided by AWS. This project will provide both command line utilities and Python API’s.

Further Documentation: http://awsarchive.readthedocs.org/

Used AWS Services

awsarchive makes use of the following AWS services:

The project assumes that you have a single AWS access and secret key that can be used across the AWS services used.

Installation

Install via pip:

$ pip install awsarchive

Install from source:

$ git clone git://github.com/billyshambrook/awsarchive.git
$ cd awsarchive
$ python setup.py install

Contribute

awsarchive is an open-source software originally written by BillyShambrook and released under the MIT licence. The project is hosted on Github, where everyone is welcome to contribute, ask for help or simply give feedback. Please fork project, add your changes and open a pull request.

Command line utilities

awsarchive can be accessed via the command line using awsarchive followed by the command you wish to perform.

To list all the commands available do:

$ awsarchive --help

or to get help for a specific command do:

$ awsarchive <command> --help
Commands Description
glacier Move files to Glacier.

API examples

To archive a file to AWS Glacier, you can do the following:

>>> import awsarchive
>>> metadata = awsarchive.to_glacier(filename='file.mpg', vault_name='myarchive', access_key='abc', secret_key='xyz', region='us-west-2')

This will first create the vault if not already present, then upload the file to it.

It will then return metadata relating to that archive.

>>> metadata
ArchiveMetadata(archive_id='123ABC', filename='file.mpg', vault='myarchive', region='us-west-2')

Glacier does provide an inventory of your archives using the AWS management console or Glacier API, however it is recommended to keep your own inventory.

awsarchive gives you a way to roll your own inventory using SimpleDB, which is done by simply passing in the ArchiveMetadata instance.

>>> awsarchive.log_archive(metadata, domain_name='myarchive', access_key='abc', secret_key='xyz', region='us-west-2')

This will first create the domain if not already present, then add the archive metadata as a new item, using the archive_id as its item name.

API reference

class awsarchive.__init__.ArchiveMetadata
Variables:
  • archive_id – AWS Glacier archive ID.
  • filename – Filename.
  • vault – AWS Glacier vault name.
  • region – AWS region name.
awsarchive.__init__.log_archive(metadata, domain_name, access_key, secret_key, region)

Log archive to AWS SimpleDB. The archive ID will be used for the item name along with filename, the vault name and the region where the vault is located as its attributes.

Parameters:
  • archive_metadata (awsarchive.ArchiveMetadata) – Information about the archive.
  • domain_name (string) – AWS SimpleDB domain name.
  • access_key (string) – AWS SimpleDB access key.
  • secret_key (string) – AWS SimpleDB secret key.
  • region (string) – AWS SimpleDB region name.
awsarchive.__init__.to_glacier(filename, vault_name, access_key, secret_key, region)

Archive file in glacier. On completion, the archive will be assigned a ID which is returned and should be stored to be used for retrieving the file at a later stage.

Parameters:
  • filename (string) – Local file to archive.
  • vault_name (string) – AWS Glacier vault to store file in.
  • access_key (string) – AWS Glacier access key.
  • secret_key (string) – AWS Glacier secret key.
  • region (string) – AWS Glacier region name.
Return type:

awsarchive.ArchiveMetadata
Returns:

Archive metadata.

Indices and tables