Introduction to AWS Elastic File System(EFS)
Last Updated : 02 Aug, 2023
AWS(Amazon Web Services) offers a wide range of storage services that can be provisioned depending on your project requirements and use case. AWS storage services have different provisions for highly confidential data, frequently accessed data, and the not so frequently accessed data. You can choose from various storage types namely, object storage, file storage, block storage services, backups, and data migration options. All of which fall under the AWS Storage Services list.
What is AWS Elastic File System?
From the aforementioned list, EFS falls under the file storage category. EFS is a file-level, fully managed, storage provided by AWS (Amazon Web Services) that can be accessed by multiple EC2 instances concurrently. Just like the AWS EBS, EFS is specially designed for high throughput and low latency applications.
How Does EFS Work?
EFS can be created using the EC2-Instance where it will be created in a specific region and distributed across multiple availability zones for the purpose of high availability and durability. You can choose the EFS based on the I/Ops you are going to perform.
Once the EFS is created you need to set up mount targets which will provide the connectivity to your EFS file system. Following are some of the resources which you can mount on EFS.
Use Cases Of EFS
- Secured file sharing: You can share your files in every secured manner and in a faster and easier way and also ensures consistency across the system.
- Web Hosting: Well suited for web servers where multiple web servers can access the file system and can store the data EFS also scales whenever the data incoming is increased.
- Modernize application development: You can share the data from the AWS resources like ECS, EKS, and any serverless web applications in an efficient manner and without more management required.
- Machine Learning and AI Workloads: EFS is well suited for large data AI applications where multiple instances and containers will access the same data improving collaboration and reducing data duplication.
Above are some of the use cases of AWS EFS widely used for handling larger datasets type applications.
When To Choose Amazon EFS?
Amazon EFS is suitable for the following scenarios:
- Shared File Storage: If the multiple EC2-Instances have to access the same data. EFS management of shared data and ensures consistency across instances.
- Scalability: EFS can increase and decrease its storage capacity depending on the incoming data. If you don't have an idea how much data is going to come to the store then you can use the Amazon EFS.
- Simplified Data Sharing: If different applications want the same data to use in a collaborative manner then you can choose the Amazon EFS. EFS can share large datasets across a group of instances.
- Use with Serverless Applications: Amazon EFS is well suited for the service like serverless computing services like some of the examples AWS lambda, EFS, and so on.
- Pay-as-You-go-Model: If your application is having unpredictable storage growth then there is no need of paying upfront or no need of any prior commitments. You pay only for the storage that you are going to use.
Before you are going to use any particular service for your application you need to analyze the cost and requirements of the data that you're going to save.
How is Amazon EFS Different Than Amazon S3?
Amazon EFS (Elastic file system) and S3 (Simple Storage Service) are two different storage services provided by Amazon web services with two different purposes one storage is to store static data and another is to store dynamic data.
|
EFS can be accessed by multiple EC2-instance at the same time which can analyze the data and can use the data combined. | S3 is an object storage that is mainly used to store and retrieve static data the data is stored in the form of objects. |
If any changes are made by one instance to the data it is visible to the other instances immediately. | When you perform the read and write operations on the data you will always get the most updated version of the data only. |
Widely used for the scenarios like data sharing with multiple instances. | Commonly used for backup, restoring, and hosting static content. |
How is Amazon EFS Different Than Amazon EBS?
Amazon EFS ( Elastic file system) and Amazon EBS (Elastic Block Store) are two different services provided by Amazon web services for different use cases.
|
If the application is required to share the shared file access then you can choose the Elastic file system. | If the application requires separate storage then you can use the block store. |
Multiple instances can access the storage system at once. | A single instance only can access the storage system at once. |
Deliveries the aggregate throughputs to thousands of clients simultaneously. | Highly available with low latency. |
Different Storage Classes in AWS EFS
Standard storage class
- This is the default storage class for EFS.
- The user is only charged for the amount of storage used.
- This is recommended for storing frequently accessed files.
Infrequently Accessed storage class(One Zone)
- Cheaper storage space.
- Recommended for rarely accessed files.
- Increased latency when reading or writing files.
- The user is charged not only for the storage of files but also charged for read and write operations.
Different Performance Modes in EFS
General-purpose
- Offers low latency.
- Supports a maximum of 7000 IOPS.
- As a cloud watch metric, you can view the amount of IOPS your architecture uses and can switch to Max IOPS if required.
Max I/O
- This is recommended when EFS needs over 7000 IOPS
- Theoretically, this mode has an unlimited I/O speed.
Different Throughput Modes in EFS
- Burst Mode: Allows 100MBPS of burst speed per TB of storage.
- Provisioned Mode: Users can decide the max burst speed of the EFS but are charged more when speeds go beyond the default limit.
Steps To Configure and Connect To EFS(Elastic File Storage)
First, create an AWS account create refer to Amazon Web Services – Setting Up an AWS Account.
Step 1: Create an EFS from the AWS console. Choose the correct VPC (Virtual Private Cloud) and configuration that suits your use case.
.webp)
Step 2: Create one or more EC2 servers from the EC2 dashboard as needed for your use case.
Step 3: Allow the EC2 security group to access EFS. Connect To EFS from your EC2 servers. Primarily there are 2 methods of connecting to EFS from EC2 servers: Linux NFS Client: This is the old traditional method of connecting to file systems.EFS Mount Helper: This is the AWS-recommended and simpler solution to connect to EFS.
-(1).webp)
.webp)
- Once you have connected to AWS EFS from your EC2 instances you will have a folder of any name (say EFS-Folder) which will hold all the files in the EFS. Any file created in this directory can be seen or edited from any EC2 instances that have access to the EFS.
Features of AWS EFS
- Storage capacity: Theoretically EFS provides an infinite amount of storage capacity. This capacity grows and shrinks as required by the user.
- Fully Managed: Being an AWS-managed service, EFS takes the overhead of creating, managing, and maintaining file servers and storage.
- Multi EC2- Instance Connectivity: EFS can be shared between any number of EC2- instances by using mount targets.
- Note-: A mount target is an Access point for AWS EFS that is further attached to EC2 instances, allowing then access to the EFS.
- Availability: AWS EFS is region specific., however, can be present in multiple availability zones in a single region.
- EC2- instances across different availability zones can connect to EFS in that zone for a quicker access
- EFS LifeCycle Management: Lifecycle management moved files between storage classes. Users can select a retention period parameter (in number of days). Any file in standard storage which is not accessed for this time period is moved to the Infrequently accessed class for cost-saving.
- Note that the retention period of the file in standard storage resets each time the file is accessed
- Files once accessed in the IA EFS class are then moved to Standard storage.
- Note that file metadata and files under 128KB cannot be transferred to the IA storage class.
- LifeCycle management can be turned on and off as deemed fit by the users.
- Durability: Multi-availability zone presence accounts for the high durability of the Elastic File System.
- Transfer: Data can be transferred from on-premise to the EFS in the cloud using AWS Data Sync Service. Data Sync can also be used to transfer data between multiple EFS across regions.
The above image shows an Elastic File System shared between two instances which are further connected to their own EBS volumes. The following are some use cases of EFS:
- Multiple server architectures: In AWS only EFS provides a shared file system. So all the applications that require multiple servers to share one single file system have to use EFS.
- Big Data Analytics: Virtually infinite capacity and extremely high throughput make EFS highly suitable for storing files for Big data analysis.
- Reliable data file storage: EBS data is stored redundantly in a single Availability Zone however EFS data is stored redundantly in multiple Availability Zones. Making it more robust and reliable than EBS.
- Media Processing: High capacity and high throughput make EFS highly favorable for processing big media files.
Advantages of AWS Elastic File System(EFS)
- EFS is scalable and elastic where you can scale the EFS depending on the data that is going to be stored. The scaling is done automatically.
- Multiple instances in AWS can access the EFS simultaneously. It makes it easy to share data across instances.
- Within the same region, the EFS replicates the data to the multiple availability zones.
- EFS allows you to take a backup of data from time to time so if there is any data loss you can always have backup.
- EFS supports a wide range of POSIX file system features, making it compatible with many Linux-based applications
Limitations of AWS Elastic File System(EFS)
There are a few limitations to consider when using AWS Elastic File System (EFS).
- EFS only supports the Network File System (NFS) protocol, so it can only be mounted and accessed by devices that support NFS.
- EFS has a maximum file size of 47.9 TB.
- EFS has a maximum throughput of 1000 MB/s per file system and a maximum of 16,000 IOPS per file system.
- EFS has a maximum number of files and directories that can be created within a single file system, which is determined by the size of the file system. For example, a 1 TB file system can support up to about 20 million files and directories.
- EFS is only available in certain regions, and it is not possible to migrate data between regions.