External Storage - AWS S3

External Storage / AWS S3

To utilize AWS S3 as your External Storage, you must set the provider attribute to aws within the store hash and, at a minimum, provide your AWS credentials (key and secret), along with a bucket name and the AWS region where your bucket is located. Below is the full list of store hash attributes for AWS S3 storage.

Alternatively, you can use our Secure Storage Connectors. Simply save your credentials in your Optidash Account and reference them by ID. This mechanism significantly enhances the security of your cloud credentials. When Secure Storage Connectors are in use, you only need to provide the Optidash API with your Connector id instead of provider, key, and secret properties. You can add a new Connector in your Optidash Account.

Authentication

When passing AWS credentials in your request JSON, set the following authentication properties:

Attribute Type Description
provider String provider must be set to aws
key String AWS Access Key Id
secret String AWS Secret Access Key 
{
    "store": {
        "provider": "aws",
        "key": "your-aws-key",
        "secret": "your-aws-secret"
    }
}

When using Secure Storage Connectors, you only need to provide your Connector ID:

Attribute Type Description
id String Secure Storage Connector ID 
{
    "store": {
        "id": "your-connector-id"
    }
}

AWS S3 properties and settings

Attribute Type Required Description
bucket String Yes Destination bucket name in your Amazon S3 account.
region String Yes AWS Region where your S3 bucket is located.
path String No Destination path in your S3 bucket (without leading slash). Defaults to root.
acl String No Access Control List of the destination object. Defaults to public-read.
class String No Custom Storage Class for your object. Valid values are standard, reduced-redundancy, standard-ia, onezone-ia, intelligent-tiering, glacier and deep-archive. Defaults to standard.
metadata Hash No Custom S3 Metadata.
headers Hash No Custom HTTP headers.
tags Hash No Custom S3 Tags.

The Optidash API also supports setting custom headers on your objects, including Expires, Cache-Control, Content-Type, Content-Encoding, Content-Language and Content-Disposition.

{
    "store": {
        "provider": "aws",
        "key": "your-aws-key",
        "secret": "your-aws-secret",
        "bucket": "images",
        "region": "eu-central-1",
        "path": "assets/image.jpg",
        "acl": "public-read",
        "metadata": {
            "key": "value"
        },
        "headers": {
            "Cache-Control": "max-age=2592000000"
        },
        "tags": {
            "key": "value"
        }
    }
}

Here's an example of using AWS S3 as the External Storage provider in a cURL request:

curl https://api.optidash.ai/1.0/fetch -X POST -u your-api-key: \
-H "Content-Type: application/json" \
-d '{
    "url": "https://www.website.com/image.jpg",
    "resize": {
        "width": 100,
        "height": 75
    },
    "store": {
        "provider": "aws",
        "key": "your-aws-key",
        "secret": "your-aws-secret",
        "bucket": "bucket-name",
        "region": "eu-central-1",
        "path": "assets/image.jpg",
        "headers": {
            "Cache-Control": "max-age=2592000000"
        }
    }
}'

When using AWS as your External Storage, the url property within the JSON response will point to the object's location within the S3 bucket, which you can safely use in production.

HTTP/1.1 200 OK

Date: 
Status: 200 OK
Content-Type: application/json

{
    "success": true,
    "code": 200,
    "id": "9fccf4b5-cfab-4e92-9276-5d2028fcb6a0",
    "input": {
        "name": "image.jpg",
        ..
    },
    "output": {
        "url": "https://bucket-name.s3.eu-central-1.amazonaws.com/assets/image.jpg",
        ..
    }
}

Custom Bucket Policies

If you need to create an AWS User dedicated only for the Optidash API or are using a custom bucket policy, ensure to include s3:PutObject and s3:PutObjectAcl entries in the allowed actions section in your bucket policy file (and replace bucket-name in this example).

{
    "Statement": {
        "Effect": "Allow",
        "Action": [
            "s3:PutObject",
            "s3:PutObjectAcl"
        ],
        "Resource": [
            "arn:aws:s3:::bucket-name/*"
        ]
    }
}