MSP360 For IT PROs
Articles about cloud backup, cloud storage and more
BLOG MENU
Don’t miss new articles!
Thank you for subscribing!
Every week we deliver quality content about cloud storage, backups and security. Leave your email to get weekly and monthly digests

Upload to S3 Glacier with Amazon S3 Lifecycle Rules

Upload to S3 Glacier with Amazon S3 Lifecycle Rules

There are a number of storage classes for every need and budget in Amazon Web Services cloud storage. The cheapest method to store the backup for a long time is Amazon Glacier. However, there are two different ways to upload files to that storage: direct and using the lifecycle policy. In this article, we will demonstrate why the lifecycle policy method is more practical.  

Everything about backing up to Amazon S3: New call-to-action

Table of Contents

    Why Uploading to Amazon S3 Glacier?

    Amazon S3 Glacier is the most cost-effective storage solution for the infrequently accessed data Amazon Web Services has to offer. You can store any amounts of data for $0.004 per gigabyte per month, so it proves itself cheaper than the on-premises solutions.

    It is also secure: by default, only you can access your data. Additionally, you can use the IAM policies to define the additional circle of users who can access your data.

    Further reading How to Configure IAM Users for Amazon S3 Glacier

    All the uploaded data is encrypted on the server side using 256-bit Advanced Encryption Standard (AES-256). Besides, you can also set up the client-side encryption on your side.

    With that said, Amazon Glacier is one of the best options for the data archival.

    FREE WHITEPAPER
    Archiving to the Cloud: Cold Storage

    How to use cold storage - like Amazon S3 Glacier - cost-effectively and efficiently? Find out in our whitepaper:

    New call-to-action
    Cold Storage WP icon

    What Is Amazon S3 Lifecycle Policies?

    AWS lifecycle rules are a set of options aimed to manage the data stored within the S3. It allows to move or to delete the objects after the certain number of days by configuring your own lifecycle rules.

    Previously, the lifecycle configuration could only be implemented by creating an XML file, but nowadays it is possible to create it using a pretty intuitive web GUI within the AWS management console.

    Upload to Glacier Through S3 Lifecycle Policies

    Amazon S3 Glacier offers a few data upload options:

    • AWS control web interface.
    • AWS (SDK) software or Amazon S3 Glacier API.
    • Amazon S3 allows setting lifecycle management rules to archive unused data from Amazon S3 to S3 Glacier.
    • Own solutions of AWS (APN) network partners that are already embedded in S3 Glacier.

    Let’s consider archiving using Lifecycle Policy in S3 in detail.

    The cost of data storage in S3 or S3 Glacier is just a part of the price for these services. An extra pay is charged for requests for data stored in S3 and download traffic. In S3 Glacier, this list also includes the cost of data retrieval and the price also depends on the retrieval speed.

    The point is that all information in Glacier is stored on reasonably priced tape drives, but this information is copied to more expensive media before its transfer to the user. So, when data is archived directly to Glacier, it takes half a day to index it and 3-5 hours more to make it available for retrieval (if standard retrieval is used). If data is uploaded to S3, all files will be instantly available.

    It is possible to store several data versions and manage their lifecycle in S3. Upon expiration, data is deleted or it can be transferred to S3 Glacier. If you set a storage class equal to 0 days, information will be immediately sent to S3 Glacier. It is of use when information is rarely accessed in everyday life but its storage life is limited.

    Though, it might seem that uploading data to S3 first and go with it to Glacier afterward might be more expensive, AWS has ensured that this exact scenario leads to no more expenses than direct Glacier upload.

    Applying AWS Lifecycle Rules via AWS Management Console

    The lifecycle rules can be set up in the AWS management console directly.

    Here is our step-by-step guide:

    1. Open  your AWS console, click “Services” in the right upper corner and select “S3”
    2. Select the bucket you want to set up lifecycle rule for and click on it. You will see the following menu on the right to it.
      Adding rules for glacier transition S3
    3. Click “Management”, and then “ Add lifecycle rule” button.
      How to add lifecycle rule glacier
    4. Select the name and add the tags/prefixes for your lifecycle rule if necessary.
      Adding a rule for S3-glacier lifecycle
    5. Check the versions of data you want to be transferred with the lifecycle rule, then select “Transition to Amazon S3 Glacier after” and specify the required amount of days (note that if you set different setting from "0", the costs of storage and file transfer will be bigger).
      Amazon S3 to Glacier via Console
    6. After that comes the “Expiration” step. Since the lifecycle is set up for the bucket in S3, you can just skip it and click “Next”.
      S3-Glacier transition
    7. See the “Review” step and click “Save”.
      Lifecycle to Glacier

    Automating AWS Lifecycle Rules with MSP360 Backup

    After setting up the backup plan and cloud storage in S3, you should enable Lifecycle Policy in MSP360 Backup. Go to Tools and click Lifecycle Policy or use the same parameters in the left pane. For the step-by-step guide please check out this article:

    Further reading How to Archive Amazon S3 Data to Glacier with MSP360 Backup

    In the dialog box select the source of data that will be transferred to the storage.

    As a result of such transfer, the files would first get to S3 and after that would get to Glacier in 60 days. If you set 0 days - they will be archived automatically.

    Please keep in mind that if your setting is different from "0", the costs of storage and file transfer will be bigger. AWS will charge additional costs for storing files in S3 storage.

    This approach has an advantage over the pure Glacier transfer. With the pure Glacier transfer, you have to wait 3-5 hours for the initial inventory to complete. With the S3-Glacier lifecycle policy set to 0 days, the inventory happens as soon as the files are in the storage.

    Summary

    The need to use Amazon S3 and Glacier individually or in combination depends on the objectives of each separate project. We prepared a comparison table for you to evaluate expenses for both methods of data archiving:

    Amazon S3Amazon Glacier
    Data transfer to Amazon (per GB)$0.000$0.000
    Data transfer from S3 to S3 Glacier (1,000 requests)$0.055X
    PUT (1,000 requests)$0.0055Free
    COPY (1,000 requests)$0.0055Free
    POST (1,000 requests)$0.0055Free
    LIST (1,000 requests)$0.0055Free
    GET (10,000 requests)$0.0044Free
    UPLOAD (1,000 requests)$0.0044$0.055
    RETRIEVAL (1,000 requests)$0.0044$0.055
    Other requests$0.0044Free
    Data storage (first 50 TB / month)$0.025 per GB$0.0045 per GB
    Data access rateInstantlyPrice
    Data transfer from AmazonPricePrice

    Amazon Web Services provides a possibility to use 5 GB of S3 storage (unfortunately without Glacier) for FREE for 12 months – this time period is enough to get acquainted with system features. We recommend trying MSP360 Backup available as a free trial for 15 days to assess archiving features in full.

    You are welcome to share your experience and ask questions in the comments section below.

    FREE WHITEPAPER
    Smart Guide to Amazon S3 Glacier
    • How to avoid the hidden costs of S3 Glacier
    • How to actually save on data cloud storage with lifecycle policies
    • Why data recovery planning can save you a dime
    New call-to-action
    Whitepaper icon