How To Using S3 with Laravel S3

laravel s3 aws

How To Using S3 with Laravel s3 – Brillian Solution. When you store data in Amazon S3, you can easily share it for use by more than one application. However, each application may have unique data format criteria, and may require modification or processing of your data for specific usage issues. For example, the data set created by an ecommerce application might include Custom ID (PII). When the same data is processed for analytics, this PII is not required and needs to be edited. However, if the same dataset is used for a marketing campaign, you may need to carefully enrich additional data, such as info from a customer loyalty database.

With S3 Object Lambda, you can add your own code to produce the retrieved data from S3 before returning it to the application. Specifically, you can configure an AWS Lambda utility and attach it to an S3 Lambda Object access point. When an application sends a standard S3 GET request via the S3 Object Lambda access path, the specified Lambda function is called to produce any data that is retrieved from the S3 bucket via the supporting S3 access path. Then, the S3 Object Lambda access point returns the result of switching again to the application. You can authorize and generate your own custom Lambda functions, setting up S3 Object Lambda knowledge transformations for your specific usage problem, all with no changes required to your application.

S3 Amazon

Amazon Simple Storage Service (Amazon S3) is an object storage facility that offers industry-leading scalability, data availability, security, and performance. Customers of all sizes and industries can store and maintain any quantity of data for almost any usage issue, such as data lakes, cloud-native applications, and mobile applications. With cost-effective storage classes and easy-to-use management features, you can optimize costs, customize data, and configure appropriate access checks to meet specific business, organizational, and compliance needs.

Amazon S3 Compatible Filesystems

In Laravel By default, your application’s filesystems configuration file contains a disk configuration for the s3 disk. In addition to using this disk to interact with Amazon S3, you may use it to interact with any S3 compatible file storage service such as MinIO or DigitalOcean Spaces.

Typically, after updating the disk’s credentials to match the credentials of the service you are planning to use, you only need to update the value of the endpoint configuration option. This option’s value is typically defined via the AWS_ENDPOINT environment variable:

'endpoint' => env('AWS_ENDPOINT', 'https://minio:9000'),

Configuration

  • Within Laravel – usually via .env but potentially also within config/filesystem.php
  • Within your AWS account

Laravel S3 Config

If you check your config/filesystem.php file you will see that s3 is already an option. It’s set to use the environment variables in the .env file!

Unless you need to customize this, you can leave it alone and just set the value in your .env file:

# Optionally Set the default filesystem driver to S3
FILESYSTEM_DISK=s3
# Or if using Laravel < 9
FILESYSTEM_DRIVER=s3
 
# Add items needed for S3-based filesystem to work
AWS_ACCESS_KEY_ID=xxxzzz
AWS_SECRET_ACCESS_KEY=xxxyyy
AWS_DEFAULT_REGION=us-east-2
AWS_BUCKET=my-awesome-bucket
AWS_USE_PATH_STYLE_ENDPOINT=false

The config/filesystem.php file contains the following options:

return [
    'disks' => [
        // 'local' and 'public' ommitted...
 
        's3' => [
            'driver' => 's3',
            'key' => env('AWS_ACCESS_KEY_ID'),
            'secret' => env('AWS_SECRET_ACCESS_KEY'),
            'region' => env('AWS_DEFAULT_REGION'),
            'bucket' => env('AWS_BUCKET'),
            'url' => env('AWS_URL'),
            'endpoint' => env('AWS_ENDPOINT'),
            'use_path_style_endpoint' => env('AWS_USE_PATH_STYLE_ENDPOINT', false),
        ],
    ],
];

There are some options that we didn’t use in the .env file. For example, AWS_URL can be set, which is useful for using other file storage clouds with S3-compatible APIs, such as B. CloudFlare’s R2 or Digital Ocean’s Spaces.

AWS Configuration

In AWS, you need to do two things:

Create a bucket in the S3 service
Create an IAM user to get the key/secret, then attach a policy to the user to allow access to the S3 API.
As with anything in AWS, when you create a bucket in S3, you have to look at a lot of configuration options and ask yourself if you need any of them. For most use cases, this is not the case!

Go to the S3 console, create a bucket name (it needs to be unique globally, not just your AWS account), select the region you operate in, and accept all defaults (including those marked “Block public access” set. “this bucket”).

Yes, you may want to use some of these options, but you can choose them later.

After creating the bucket, we need to get permission to use it. Suppose we create a bucket named my-awesome-bucket.

We can create an IAM User, select “programmatic access”, but don’t attach any policies or setup anything else. Make sure to record the secret access key, as they’ll only show it once.

I’ve created a video showing the process of creating a bucket and setting up IAM permissions here: https://www.youtube.com/watch?v=FLIp6BLtwjk

The Access Key and Secret Access Key should be put into your .env file.

Next, click into the IAM User and add an Inline Policy. Edit it using the JSON editor, and add the following (straight from the Flysystem docs):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1420044805001",
            "Effect": "Allow",
            "Action": [
                "s3:ListBuckets",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:ReplicateObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::my-awesome-bucket",
                "arn:aws:s3:::my-awesome-bucket/*"
            ]
        }
    ]
}

Laravel Usage

Within Laravel, you can use the file storage like so:

# If you set S3 as your default:
$contents = Storage::get('path/to/file.ext');
Storage::put('path/to/file.ext', 'some-content');
 
# If you do not have S3 as your default:
$contents = Storage::disk('s3')->get('path/to/file.ext');
Storage::disk('s3')->put('path/to/file.ext', 'some-content');

The path to the file (within S3) gets appended to the bucket name, so a file named path/to/file.ext will exist in s3://my-awesome-bucket/path/to/file.ext.

Pricing

S3 is fairly cheap – most of us will spend pennies to a few dollars a month. This is especially true if you delete files from S3 after you’re done with them, or setup Lifecycle rules to delete files after a set period of time.

The pricing is (mostly) driven by 3 dimensions. The prices vary by region and usage. Here’s an example based on usage for a real application in a given month for Chipper CI (my CI for Laravel application), which stores a lot of data in S3:

  • Storage: $0.023 per GB, ~992GB ~= $22.82
  • Number of API Calls: ~7 million requests ~= $12
  • Bandwidth usage: This is super imprecise. Data transfer for this was about $23, but this excludes EC2 based bandwidth charges.

Useful Bits about S3

  • Still, and uses NAT Gateways, be sure to produce an S3 Endpoint( type of Gateway), If your AWS setup has waiters in a private network. This is done within the Endpoints section in the VPC service. This allows calls to/ from S3 to bypass the NAT Gateway and therefore get around redundant bandwidth charges. It does not bring redundant to use this.
  • Considering enabling Versioning in your S3 pail if you are upset about lines being overwritten or deleted
  • Consider enabling Intelligent Tiering in your S3 pail to help save on storehouse costs of lines you probably will not interact with again after they’re old
  • Be apprehensive that deleting large pails( lots of lines) can bring plutocrat! This is due to the number of API calls you’d have to make to cancel lines.

Source :

https://laravel-news.com/using-s3-with-laravel

https://aws.amazon.com/id/s3/

https://docs.aws.amazon.com/id_id/AmazonS3/latest/userguide/tutorial-s3-object-lambda-uppercase.html

brillian

Leave a Reply

%d bloggers like this: