{"id":14717,"date":"2017-10-16T20:00:45","date_gmt":"2017-10-16T16:00:45","guid":{"rendered":"http:\/\/www.msp360.com\/blog\/?p=14717"},"modified":"2023-12-14T14:17:44","modified_gmt":"2023-12-14T10:17:44","slug":"compare-amazon-glacier-direct-upload-and-glacier-upload-through-amazon-s3","status":"publish","type":"post","link":"https:\/\/www.msp360.com\/resources\/blog\/compare-amazon-glacier-direct-upload-and-glacier-upload-through-amazon-s3\/","title":{"rendered":"Direct Upload to Amazon S3 Glacier vs Upload Through Amazon S3"},"content":{"rendered":"<p>Amazon <a href=\"https:\/\/www.msp360.com\/resources\/blog\/amazon-s3-glacier\/\">S3 Glacier<\/a> is a cloud service dedicated to storing archived data that is not likely to be retrieved often. In other words, it is designed for infrequently accessed data. Glacier has a high latency of data retrieval but offers low pricing and high safety for stored archives. In this article, we are going to explain Glacier\u2019s data uploading nuances.<br \/>\n<!--more--><\/p>\n<div class=\"table-of-content \">\n\t\t\t\t<p>Table of Contents<\/p>\n\t\t\t\t<ul><\/ul>\n\t\t\t\t<\/div>\n<h2>Working with S3 Glacier<\/h2>\n<p>Amazon Glacier is a quite cost-effective solution for the prolonged keeping of important data that is not used often. It is a nice choice for a company that possesses a lot of outdated electronic documentation and wants cheap but safe storage. Amazon does not urge its customers to store more or less there, though Glacier's optimal usage model foresees archives to be kept for a longer period of time.<\/p>\n<div class=\"call-to-action\">\n<div class=\"call-to-action__left\">\n<div class=\"call-to-action__tag\">FREE WHITEPAPER<\/div>\n<div class=\"call-to-action__title\">Archiving to the Cloud: Cold Storage<\/div>\n<div class=\"call-to-action__text\">\n<p>How to use cold storage - <em>like Amazon S3 Glacier<\/em> - cost-effectively and efficiently? Find out in our whitepaper:<\/p>\n<\/div>\n<!--HubSpot Call-to-Action Code --><span class=\"hs-cta-wrapper hs-cta-deferred\" id=\"hs-cta-wrapper-87c13c37-bf67-436c-8ca8-de28e54e32d7\" data-portal=\"5442029\" data-id=\"87c13c37-bf67-436c-8ca8-de28e54e32d7\"><span class=\"hs-cta-node hs-cta-87c13c37-bf67-436c-8ca8-de28e54e32d7\" id=\"hs-cta-87c13c37-bf67-436c-8ca8-de28e54e32d7\"><!--[if lte IE 8]><div id=\"hs-cta-ie-element\"><\/div><![endif]--><a href=\"https:\/\/cta-redirect.hubspot.com\/cta\/redirect\/5442029\/87c13c37-bf67-436c-8ca8-de28e54e32d7\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" class=\"hs-cta-img\" id=\"hs-cta-img-87c13c37-bf67-436c-8ca8-de28e54e32d7\" style=\"border-width:0px;\" src=\"https:\/\/no-cache.hubspot.com\/cta\/default\/5442029\/87c13c37-bf67-436c-8ca8-de28e54e32d7.png\" alt=\"CTA\"><\/a><\/span><\/span><!-- end HubSpot Call-to-Action Code -->\n<\/div>\n<div class=\"call-to-action__right\"><img decoding=\"async\" style=\"max-width: 200px;\" src=\"https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2020\/05\/Archiving-to-Cold-Storage-icon.png\" alt=\"Cold Storage WP icon\" \/><\/div>\n<\/div>\n<p>Glacier storage ensures high redundancy, as an archive is stored within multiple facilities at once. The archived data is secured with AES-256 encryption on the server-side. Additional safety is ensured by <a href=\"http:\/\/docs.aws.amazon.com\/amazonglacier\/latest\/dev\/vault-lock.html\" target=\"_blank\" rel=\"noopener noreferrer\">Vault Lock<\/a> policies.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-28836\" src=\"https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2017\/10\/finding-glacier.png\" alt=\"Finding Glacier\" width=\"498\" height=\"360\" srcset=\"https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2017\/10\/finding-glacier.png 498w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2017\/10\/finding-glacier-300x217.png 300w\" sizes=\"auto, (max-width: 498px) 100vw, 498px\" \/><\/p>\n<p>The monthly storage price is fixed and varies from $0.004 to $0.013 per 1GB, depending on the region. Retrieval is free for up to 10 GB a month. The deletion of data is free if this data was stored for more than 3 months, otherwise, an early deletion fee would be applied.<\/p>\n<p><span class=\"further-reading \">Further reading<\/span> <a href=\"https:\/\/www.msp360.com\/resources\/blog\/amazon-glacier-pricing-explained\/\">Amazon S3 Glacier Pricing Explained<\/a><\/p>\n<blockquote><p><a href=\"https:\/\/www.msp360.com\/resources\/blog\/amazon-s3-glacier-deep-archive-support-in-msp360-backup-and-explorer\/\"><strong>Amazon S3 Glacier Deep Archive<\/strong><\/a> storage class is meant for deep archival data that is only needed very infrequently but can\u2019t be deleted. Storing 1Gb will cost you $0.00099 per month.<\/p>\n<!--HubSpot Call-to-Action Code --><span class=\"hs-cta-wrapper hs-cta-deferred\" id=\"hs-cta-wrapper-19273f9a-59e0-4890-a7c3-9edc4119e6f9\" data-portal=\"5442029\" data-id=\"19273f9a-59e0-4890-a7c3-9edc4119e6f9\"><span class=\"hs-cta-node hs-cta-19273f9a-59e0-4890-a7c3-9edc4119e6f9\" id=\"hs-cta-19273f9a-59e0-4890-a7c3-9edc4119e6f9\"><!--[if lte IE 8]><div id=\"hs-cta-ie-element\"><\/div><![endif]--><a href=\"https:\/\/cta-redirect.hubspot.com\/cta\/redirect\/5442029\/19273f9a-59e0-4890-a7c3-9edc4119e6f9\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" class=\"hs-cta-img\" id=\"hs-cta-img-19273f9a-59e0-4890-a7c3-9edc4119e6f9\" style=\"border-width:0px;\" src=\"https:\/\/no-cache.hubspot.com\/cta\/default\/5442029\/19273f9a-59e0-4890-a7c3-9edc4119e6f9.png\" alt=\"CTA\"><\/a><\/span><\/span><!-- end HubSpot Call-to-Action Code --><\/blockquote>\n<p>Users have to set up jobs in order to download archives or archive lists in vault snapshots. These jobs run in the background and usually take several hours to complete. There are two ways to upload data<\/p>\n<ul>\n<li style=\"font-weight: 400;\">Direct upload to Glacier from the user's instance.<\/li>\n<li style=\"font-weight: 400;\">Using Amazon S3 lifecycle policies to move data from S3 to Glacier.<\/li>\n<\/ul>\n<p>Let's explore both of them in detail.<\/p>\n<h2>Direct Upload to Glacier<\/h2>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-14720 aligncenter\" src=\"https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/scheme-of-how-amazon-glacier-upload-works-1024x576.jpg\" alt=\"scheme-of-how-amazon-glacier-upload-works\" width=\"625\" height=\"352\" srcset=\"https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/scheme-of-how-amazon-glacier-upload-works.jpg 1024w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/scheme-of-how-amazon-glacier-upload-works-300x169.jpg 300w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/scheme-of-how-amazon-glacier-upload-works-768x432.jpg 768w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/scheme-of-how-amazon-glacier-upload-works-624x351.jpg 624w\" sizes=\"auto, (max-width: 625px) 100vw, 625px\" \/><\/p>\n<p id=\"last\">There is no Wizard in the AWS console for uploading archives to Glacier vaults. Users have to do that by creating requests via Glacier REST API or use <a href=\"http:\/\/docs.aws.amazon.com\/amazonglacier\/latest\/dev\/using-aws-sdk.html\" target=\"_blank\" rel=\"noopener noreferrer\">AWS Software Development Kits (or SDKs) for their own applications. All that requires some coding and AWS provides SDKs with Glacier support for the following programming languages:<\/a><\/p>\n<div id=\"slidebox\"><!--HubSpot Call-to-Action Code --><span class=\"hs-cta-wrapper hs-cta-deferred\" id=\"hs-cta-wrapper-aa07fdb8-7776-46a5-9fa0-ec6e93f0f0a6\" data-portal=\"5442029\" data-id=\"aa07fdb8-7776-46a5-9fa0-ec6e93f0f0a6\"><span class=\"hs-cta-node hs-cta-aa07fdb8-7776-46a5-9fa0-ec6e93f0f0a6\" id=\"hs-cta-aa07fdb8-7776-46a5-9fa0-ec6e93f0f0a6\"><!--[if lte IE 8]><div id=\"hs-cta-ie-element\"><\/div><![endif]--><a href=\"https:\/\/cta-redirect.hubspot.com\/cta\/redirect\/5442029\/aa07fdb8-7776-46a5-9fa0-ec6e93f0f0a6\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" class=\"hs-cta-img\" id=\"hs-cta-img-aa07fdb8-7776-46a5-9fa0-ec6e93f0f0a6\" style=\"border-width:0px;\" src=\"https:\/\/no-cache.hubspot.com\/cta\/default\/5442029\/aa07fdb8-7776-46a5-9fa0-ec6e93f0f0a6.png\" alt=\"CTA\"><\/a><\/span><\/span><!-- end HubSpot Call-to-Action Code --><\/div>\n<ul>\n<li>C++.<\/li>\n<li>Go.<\/li>\n<li>Java.<\/li>\n<li>JavaScript in Node.js.<\/li>\n<li>.NET.<\/li>\n<li>PHP.<\/li>\n<li>Python.<\/li>\n<li>Ruby.<\/li>\n<\/ul>\n<p>This way of uploading is, therefore, most convenient for users with programming skills or for third-party providers who offer their own tools for S3 Glacier <a href=\"https:\/\/www.msp360.com\/resources\/blog\/backup-storage-management-for-msps\/\">storage management<\/a>.<\/p>\n<p>Amazon provides two alternative schemes of direct upload to Glacier:<\/p>\n<ul>\n<li>Upload in a single operation<\/li>\n<li>Upload in parts<\/li>\n<\/ul>\n<p>Single operation option is available for up to 4GB of data. Upload in parts is recommended for archives bigger than 100MB: it transfers each part in a parallel session (size of parts is specified by the user). If a session fails, only this part would be missing so a user will have to resend only it alone. No additional fees are charged for <a href=\"https:\/\/www.msp360.com\/resources\/blog\/amazon-s3-multi-part-upload\/\">multipart upload<\/a>.<\/p>\n<h2>Scheduled Upload to Glacier from S3<\/h2>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-26721 size-full\" src=\"https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2017\/11\/AWS-Management-console-lifecycle-rules-2.png\" alt=\"Adding a rule for S3-glacier lifecycle\" width=\"700\" height=\"700\" srcset=\"https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2017\/11\/AWS-Management-console-lifecycle-rules-2.png 700w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2017\/11\/AWS-Management-console-lifecycle-rules-2-150x150.png 150w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2017\/11\/AWS-Management-console-lifecycle-rules-2-300x300.png 300w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2017\/11\/AWS-Management-console-lifecycle-rules-2-624x624.png 624w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2017\/11\/AWS-Management-console-lifecycle-rules-2-250x250.png 250w\" sizes=\"auto, (max-width: 700px) 100vw, 700px\" \/><\/p>\n<p>Data that is already in AWS\u2019s cloud can be moved to Glacier storage with the help of the lifecycle policy feature. If you do not urgently need some of the files stored in an S3 bucket, it is possible to schedule their transfer to a less costly place - that is what these policies are for.<\/p>\n<p>You can create a policy via your AWS console, in the <b>Properties<\/b> page of your S3 bucket. Just make sure that the <b>Archive to the S3 Glacier Storage Class<\/b> checkbox is selected. After a new policy is created, your data will be transferred from S3 to Glacier after the time specified. It will not show up in Glacier storage, however - you still could view it from the S3 bucket. You would have to restore this archive from Glacier before any other operations would be available.<\/p>\n<p><span class=\"further-reading \">Further reading<\/span> <a href=\"https:\/\/www.msp360.com\/resources\/blog\/amazon-s3-lifecycle-rules-upload-to-glacier\/\">How to Upload Files to S3 Glacier with Lifecycle Rules<\/a><\/p>\n<p>Scheduled upload to Glacier is the best option in case the user's data is already in S3. It is also a more convenient way for companies with a great flow of electronic documentation because it allows an administrator to automate the archiving of a large number of items. On the downside, this additional tier of storage results in <a href=\"https:\/\/aws.amazon.com\/s3\/pricing\/\" target=\"_blank\" rel=\"noopener noreferrer\">extra storage fees<\/a> plus a request fee for archiving to Glacier.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-14721 aligncenter\" src=\"https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/amazon-storage-tiers-scheme-1024x576.jpg\" alt=\"amazon-storage-tiers-scheme\" width=\"625\" height=\"352\" srcset=\"https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/amazon-storage-tiers-scheme.jpg 1024w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/amazon-storage-tiers-scheme-300x169.jpg 300w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/amazon-storage-tiers-scheme-768x432.jpg 768w, https:\/\/www.msp360.com\/resources\/wp-content\/uploads\/2016\/06\/amazon-storage-tiers-scheme-624x351.jpg 624w\" sizes=\"auto, (max-width: 625px) 100vw, 625px\" \/><\/p>\n<h2>Summary<\/h2>\n<p>Both ways of transferring data to Glacier storage have certain pros and cons. Let us summarize their differences to make the comparison easier.<\/p>\n<table>\n<tbody>\n<tr>\n<td><\/td>\n<td><strong>Direct Upload to Glacier<\/strong><\/td>\n<td><strong>Archiving from S3<\/strong><\/td>\n<\/tr>\n<tr>\n<td><strong>Time consumption<\/strong><\/td>\n<td>Multipart upload allows faster archiving<\/td>\n<td>Scheduled archiving jobs automate the process and save time<\/td>\n<\/tr>\n<tr>\n<td><strong>Fees that apply<\/strong><\/td>\n<td>Glacier storage fee<\/td>\n<td>\n<ul>\n<li>S3 storage fee<\/li>\n<li>S3 archiving request fee<\/li>\n<li>Glacier storage fee<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td><strong>Preconditions<\/strong><\/td>\n<td>An interface must be set up programmatically in order to send uploading requests to AWS<\/td>\n<td>Data must be stored in S3 in order to be transferred to Glacier<\/td>\n<\/tr>\n<tr>\n<td><strong>Visibility<\/strong><\/td>\n<td>Archives are visible on the Glacier control panel<\/td>\n<td>Archives are not visible on the Glacier side and must be managed via the S3 control panel<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><a href=\"https:\/\/www.msp360.com\/backup\/\">MSP360 Backup<\/a> supports Amazon Glacier and you can perform direct uploads of the data to your Glacier storage. It's also possible to create and manage lifecycle policies and transfer archives to Glacier directly from the MSP360 Backup user interface.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Amazon S3 Glacier is a cloud service dedicated to storing archived data that is not likely to be retrieved often. In other words, it is designed for infrequently accessed data. Glacier has a high latency of data retrieval but offers low pricing and high safety for stored archives. In this article, we are going to [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":26980,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[883,877],"tags":[],"class_list":["post-14717","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-backup-and-dr-articles","category-blog-articles"],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/posts\/14717","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/comments?post=14717"}],"version-history":[{"count":3,"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/posts\/14717\/revisions"}],"predecessor-version":[{"id":56987,"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/posts\/14717\/revisions\/56987"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/media\/26980"}],"wp:attachment":[{"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/media?parent=14717"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/categories?post=14717"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.msp360.com\/resources\/wp-json\/wp\/v2\/tags?post=14717"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}