masterElmar/AWS-CCP-Notes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Modified] Table Of Contents added

Browse Source
This commit is contained in:
kananinirav
2022-08-16 10:20:01 +09:00
parent bfe63bf998
commit a2ec3e9877
6 changed files with 949 additions and 825 deletions
Download Patch File Download Diff File Expand all files Collapse all files

192
sections/ec2_storage.md
View File

@@ -1,136 +1,154 @@
# EC2 Instance Storage
* [EBS volumes](#ebs-volume)
* [EFS: network file system, can be attached to 100s of instances in a region](#efs-elastic-file-system)
* [EFS-IA: cost-optimized storage class for infrequent accessed files](#efs-infrequent-access-efs-ia)
* [FSx for Windows: Network File System for Windows servers](#amazon-fsx-for-windows-file-server)
* [FSx for Lustre: High Performance Computing Linux file system](#amazon-fsx-for-lustre)
- [EC2 Instance Storage](#ec2-instance-storage)
- [EBS Volumes](#ebs-volumes)
- [What’s an EBS Volume?](#whats-an-ebs-volume)
- [EBS Volume](#ebs-volume)
- [EBS – Delete on Termination attribute](#ebs--delete-on-termination-attribute)
- [EBS Snapshots](#ebs-snapshots)
- [EBS Snapshots Features](#ebs-snapshots-features)
- [EFS: Elastic File System](#efs-elastic-file-system)
- [EFS Infrequent Access (EFS-IA)](#efs-infrequent-access-efs-ia)
- [Amazon FSx – Overview](#amazon-fsx--overview)
- [Amazon FSx for Windows File Server](#amazon-fsx-for-windows-file-server)
- [Amazon FSx for Lustre](#amazon-fsx-for-lustre)
- [EC2 Instance Store](#ec2-instance-store)
- [Shared Responsibility Model for EC2 Storage](#shared-responsibility-model-for-ec2-storage)
- [AMI Overview](#ami-overview)
- [AMI Process (from an EC2 instance)](#ami-process-from-an-ec2-instance)
- [EC2 Image Builder](#ec2-image-builder)
- EBS: Elastic Block Store, Volume is a network drive you can attach to your instances while they run
- EFS: network file system, can be attached to 100s of instances in a region
- EFS-IA: cost-optimized storage class for infrequent accessed files
- FSx for Windows: Network File System for Windows servers
- FSx for Lustre: High Performance Computing Linux file system
## EBS Volumes
### What’s an EBS Volume?
* An EBS (Elastic Block Store) Volume is a network drive you can attach to your instances while they run
* It allows your instances to persist data, even after their termination
* They can only be mounted to one instance at a time (at the CCP level)
* They are bound to a specific availability zone
* Analogy: Think of them as a “network USB stick”
* Free tier: 30 GB of free EBS storage of type General Purpose (SSD) or Magnetic per month
- An EBS (Elastic Block Store) Volume is a network drive you can attach to your instances while they run
- It allows your instances to persist data, even after their termination
- They can only be mounted to one instance at a time (at the CCP level)
- They are bound to a specific availability zone
- Analogy: Think of them as a “network USB stick”
- Free tier: 30 GB of free EBS storage of type General Purpose (SSD) or Magnetic per month
### EBS Volume
* It’s a network drive (i.e. not a physical drive)
* It uses the network to communicate the instance, which means there might be a bit of latency
* It can be detached from an EC2 instance and attached to another one quickly
* It’s locked to an Availability Zone (AZ)
* An EBS Volume in us-east-1a cannot be attached to us-east-1b
* To move a volume across, you first need to snapshot it
* Have a provisioned capacity (size in GBs, and IOPS)
* You get billed for all the provisioned capacity
* You can increase the capacity of the drive over time
- It’s a network drive (i.e. not a physical drive)
- It uses the network to communicate the instance, which means there might be a bit of latency
- It can be detached from an EC2 instance and attached to another one quickly
- It’s locked to an Availability Zone (AZ)
- An EBS Volume in us-east-1a cannot be attached to us-east-1b
- To move a volume across, you first need to snapshot it
- Have a provisioned capacity (size in GBs, and IOPS)
- You get billed for all the provisioned capacity
- You can increase the capacity of the drive over time
### EBS – Delete on Termination attribute
* Controls the EBS behaviour when an EC2 instance terminates
* By default, the root EBS volume is deleted (attribute enabled)
* By default, any other attached EBS volume is not deleted (attribute disabled)
* This can be controlled by the AWS console / AWS CLI
* Use case: preserve root volume when instance is terminated
- Controls the EBS behaviour when an EC2 instance terminates
- By default, the root EBS volume is deleted (attribute enabled)
- By default, any other attached EBS volume is not deleted (attribute disabled)
- This can be controlled by the AWS console / AWS CLI
- Use case: preserve root volume when instance is terminated
### EBS Snapshots
* Make a backup (snapshot) of your EBS volume at a point in time
* Not necessary to detach volume to do snapshot, but recommended
* Can copy snapshots across AZ or Region
- Make a backup (snapshot) of your EBS volume at a point in time
- Not necessary to detach volume to do snapshot, but recommended
- Can copy snapshots across AZ or Region
### EBS Snapshots Features
* EBS Snapshot Archive
* Move a Snapshot to an ”archive tier” that is 75% cheaper
* Takes within 24 to 72 hours for restoring the archive
* Recycle Bin for EBS Snapshots
* Setup rules to retain deleted snapshots so you can recover them after an accidental deletion
* Specify retention (from 1 day to 1 year)
- EBS Snapshot Archive
- Move a Snapshot to an ”archive tier” that is 75% cheaper
- Takes within 24 to 72 hours for restoring the archive
- Recycle Bin for EBS Snapshots
- Setup rules to retain deleted snapshots so you can recover them after an accidental deletion
- Specify retention (from 1 day to 1 year)
## EFS: Elastic File System
* Managed NFS (network file system) that can be mounted on 100s of EC2
* EFS works with Linux EC2 instances in multi-AZ
* Highly available, scalable, expensive (3x gp2), pay per use, no capacity planning
- Managed NFS (network file system) that can be mounted on 100s of EC2
- EFS works with Linux EC2 instances in multi-AZ
- Highly available, scalable, expensive (3x gp2), pay per use, no capacity planning
## EFS Infrequent Access (EFS-IA)
* Storage class that is cost-optimized for files not accessed every day
* Up to 92% lower cost compared to EFS Standard
* EFS will automatically move your files to EFS-IA based on the last time they were accessed
* Enable EFS-IA with a Lifecycle Policy
* Example: move files that are not accessed for 60 days to EFS-IA
* Transparent to the applications accessing EFS
- Storage class that is cost-optimized for files not accessed every day
- Up to 92% lower cost compared to EFS Standard
- EFS will automatically move your files to EFS-IA based on the last time they were accessed
- Enable EFS-IA with a Lifecycle Policy
- Example: move files that are not accessed for 60 days to EFS-IA
- Transparent to the applications accessing EFS
## Amazon FSx – Overview
* Launch 3rd party high-performance file systems on AWS
* Fully managed service
* FSx for Lustre
* FSx for Windows File Server
* FSx for NetApp ONTAP
- Launch 3rd party high-performance file systems on AWS
- Fully managed service
- FSx for Lustre
- FSx for Windows File Server
- FSx for NetApp ONTAP
### Amazon FSx for Windows File Server
* A fully managed, highly reliable, and scalable Windows native shared file system
* Built on Windows File Server
* Supports SMB protocol & Windows NTFS
* Integrated with Microsoft Active Directory
* Can be accessed from AWS or your on-premise infrastructure
- A fully managed, highly reliable, and scalable Windows native shared file system
- Built on Windows File Server
- Supports SMB protocol & Windows NTFS
- Integrated with Microsoft Active Directory
- Can be accessed from AWS or your on-premise infrastructure
### Amazon FSx for Lustre
* A fully managed, high-performance, scalable file storage for High Performance Computing (HPC)
* The name Lustre is derived from “Linux” and “cluster”
* Machine Learning, Analytics, Video Processing, Financial Modeling
* Scales up to 100s GB/s, millions of IOPS, sub-ms latencies
- A fully managed, high-performance, scalable file storage for High Performance Computing (HPC)
- The name Lustre is derived from “Linux” and “cluster”
- Machine Learning, Analytics, Video Processing, Financial Modeling
- Scales up to 100s GB/s, millions of IOPS, sub-ms latencies
## EC2 Instance Store
* EBS volumes are network drives with good but “limited” performance
* If you need a high-performance hardware disk, use EC2 Instance Store
* Better I/O performance
* EC2 Instance Store lose their storage if they’re stopped (ephemeral)
* Good for buffer / cache / scratch data / temporary content
* Risk of data loss if hardware fails
* Backups and Replication are your responsibility
- EBS volumes are network drives with good but “limited” performance
- If you need a high-performance hardware disk, use EC2 Instance Store
- Better I/O performance
- EC2 Instance Store lose their storage if they’re stopped (ephemeral)
- Good for buffer / cache / scratch data / temporary content
- Risk of data loss if hardware fails
- Backups and Replication are your responsibility
## Shared Responsibility Model for EC2 Storage
AWS | USER
---- | ----
Infrastructure | Setting up backup / snapshot procedures
Replication for data for EBS volumes & EFS drives | Setting up data encryption
Replacing faulty hardware | Responsibility of any data on the drives
Ensuring their employees cannot access your data | Understanding the risk of using EC2 Instance Store
| AWS | USER |
| ------------------------------------------------- | -------------------------------------------------- |
| Infrastructure | Setting up backup / snapshot procedures |
| Replication for data for EBS volumes & EFS drives | Setting up data encryption |
| Replacing faulty hardware | Responsibility of any data on the drives |
| Ensuring their employees cannot access your data | Understanding the risk of using EC2 Instance Store |
## AMI Overview
* AMI = Amazon Machine Image
* AMI are a customization of an EC2 instance
* You add your own software, configuration, operating system, monitoring…
* Faster boot / configuration time because all your software is pre-packaged
* AMI are built for a specific region (and can be copied across regions)
* You can launch EC2 instances from:
* A Public AMI: AWS provided
* Your own AMI: you make and maintain them yourself
* An AWS Marketplace AMI: an AMI someone else made (and potentially sells)
- AMI = Amazon Machine Image
- AMI are a customization of an EC2 instance
- You add your own software, configuration, operating system, monitoring…
- Faster boot / configuration time because all your software is pre-packaged
- AMI are built for a specific region (and can be copied across regions)
- You can launch EC2 instances from:
- A Public AMI: AWS provided
- Your own AMI: you make and maintain them yourself
- An AWS Marketplace AMI: an AMI someone else made (and potentially sells)
### AMI Process (from an EC2 instance)
* Start an EC2 instance and customize it
* Stop the instance (for data integrity)
* Build an AMI – this will also create EBS snapshots
* Launch instances from other AMIs
- Start an EC2 instance and customize it
- Stop the instance (for data integrity)
- Build an AMI – this will also create EBS snapshots
- Launch instances from other AMIs
## EC2 Image Builder
* Used to automate the creation of Virtual Machines or container images
* => Automate the creation, maintain, validate and test EC2 AMIs
* Can be run on a schedule (weekly, whenever packages are updated, etc…)
* Free service (only pay for the underlying resources)
- Used to automate the creation of Virtual Machines or container images
- => Automate the creation, maintain, validate and test EC2 AMIs
- Can be run on a schedule (weekly, whenever packages are updated, etc…)
- Free service (only pay for the underlying resources)