From 2ba860d28f9b94abec4edc646d7b39e827dc2158 Mon Sep 17 00:00:00 2001 From: kananinirav <30398499+kananinirav@users.noreply.github.com> Date: Sun, 7 Aug 2022 23:17:03 +0900 Subject: [PATCH] ec2_storage added --- README.md | 1 + ec2_storage.md | 136 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 137 insertions(+) create mode 100644 ec2_storage.md diff --git a/README.md b/README.md index 7755232..f0b99db 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,7 @@ - [What is Cloud Computing?](/cloud_computing.md) - [IAM: Identity Access & Management](/iam.md) - [EC2: Virtual Machines](/ec2.md) + - [EC2 Instance Storage](/ec2_storage.md) ### Contributors diff --git a/ec2_storage.md b/ec2_storage.md new file mode 100644 index 0000000..8f5ec80 --- /dev/null +++ b/ec2_storage.md @@ -0,0 +1,136 @@ +# EC2 Instance Storage + +* [EBS volumes](#ebs-volume) +* [EFS: network file system, can be attached to 100s of instances in a region](#efs-elastic-file-system) +* [EFS-IA: cost-optimized storage class for infrequent accessed files](#efs-infrequent-access-efs-ia) +* [FSx for Windows: Network File System for Windows servers](#amazon-fsx-for-windows-file-server) +* [FSx for Lustre: High Performance Computing Linux file system](#amazon-fsx-for-lustre) + +## EBS Volumes + +### What’s an EBS Volume? + +* An EBS (Elastic Block Store) Volume is a network drive you can attach to your instances while they run +* It allows your instances to persist data, even after their termination +* They can only be mounted to one instance at a time (at the CCP level) +* They are bound to a specific availability zone +* Analogy: Think of them as a “network USB stick” +* Free tier: 30 GB of free EBS storage of type General Purpose (SSD) or Magnetic per month + +### EBS Volume + +* It’s a network drive (i.e. not a physical drive) + * It uses the network to communicate the instance, which means there might be a bit of latency + * It can be detached from an EC2 instance and attached to another one quickly +* It’s locked to an Availability Zone (AZ) + * An EBS Volume in us-east-1a cannot be attached to us-east-1b + * To move a volume across, you first need to snapshot it +* Have a provisioned capacity (size in GBs, and IOPS) + * You get billed for all the provisioned capacity + * You can increase the capacity of the drive over time + +### EBS – Delete on Termination attribute + +* Controls the EBS behaviour when an EC2 instance terminates + * By default, the root EBS volume is deleted (attribute enabled) + * By default, any other attached EBS volume is not deleted (attribute disabled) +* This can be controlled by the AWS console / AWS CLI +* Use case: preserve root volume when instance is terminated + +### EBS Snapshots + +* Make a backup (snapshot) of your EBS volume at a point in time +* Not necessary to detach volume to do snapshot, but recommended +* Can copy snapshots across AZ or Region + +### EBS Snapshots Features + +* EBS Snapshot Archive + * Move a Snapshot to an ”archive tier” that is 75% cheaper + * Takes within 24 to 72 hours for restoring the archive +* Recycle Bin for EBS Snapshots + * Setup rules to retain deleted snapshots so you can recover them after an accidental deletion + * Specify retention (from 1 day to 1 year) + +## EFS: Elastic File System + +* Managed NFS (network file system) that can be mounted on 100s of EC2 +* EFS works with Linux EC2 instances in multi-AZ +* Highly available, scalable, expensive (3x gp2), pay per use, no capacity planning + +## EFS Infrequent Access (EFS-IA) + +* Storage class that is cost-optimized for files not accessed every day +* Up to 92% lower cost compared to EFS Standard +* EFS will automatically move your files to EFS-IA based on the last time they were accessed +* Enable EFS-IA with a Lifecycle Policy +* Example: move files that are not accessed for 60 days to EFS-IA +* Transparent to the applications accessing EFS + +## Amazon FSx – Overview + +* Launch 3rd party high-performance file systems on AWS +* Fully managed service + * FSx for Lustre + * FSx for Windows File Server + * FSx for NetApp ONTAP + +### Amazon FSx for Windows File Server + +* A fully managed, highly reliable, and scalable Windows native shared file system +* Built on Windows File Server +* Supports SMB protocol & Windows NTFS +* Integrated with Microsoft Active Directory +* Can be accessed from AWS or your on-premise infrastructure + +### Amazon FSx for Lustre + +* A fully managed, high-performance, scalable file storage for High Performance Computing (HPC) +* The name Lustre is derived from “Linux” and “cluster” +* Machine Learning, Analytics, Video Processing, Financial Modeling +* Scales up to 100s GB/s, millions of IOPS, sub-ms latencies + +## EC2 Instance Store + +* EBS volumes are network drives with good but “limited” performance +* If you need a high-performance hardware disk, use EC2 Instance Store +* Better I/O performance +* EC2 Instance Store lose their storage if they’re stopped (ephemeral) +* Good for buffer / cache / scratch data / temporary content +* Risk of data loss if hardware fails +* Backups and Replication are your responsibility + +## Shared Responsibility Model for EC2 Storage + +AWS | USER +---- | ---- +Infrastructure | Setting up backup / snapshot procedures +Replication for data for EBS volumes & EFS drives | Setting up data encryption +Replacing faulty hardware | Responsibility of any data on the drives +Ensuring their employees cannot access your data | Understanding the risk of using EC2 Instance Store + +## AMI Overview + +* AMI = Amazon Machine Image +* AMI are a customization of an EC2 instance + * You add your own software, configuration, operating system, monitoring… + * Faster boot / configuration time because all your software is pre-packaged +* AMI are built for a specific region (and can be copied across regions) +* You can launch EC2 instances from: + * A Public AMI: AWS provided + * Your own AMI: you make and maintain them yourself + * An AWS Marketplace AMI: an AMI someone else made (and potentially sells) + +### AMI Process (from an EC2 instance) + +* Start an EC2 instance and customize it +* Stop the instance (for data integrity) +* Build an AMI – this will also create EBS snapshots +* Launch instances from other AMIs + +## EC2 Image Builder + +* Used to automate the creation of Virtual Machines or container images +* => Automate the creation, maintain, validate and test EC2 AMIs +* Can be run on a schedule (weekly, whenever packages are updated, etc…) +* Free service (only pay for the underlying resources)