S3 Object Storage

Amazon Simple Storage Service (S3) was one of the first AWS services, launched in 2006, and it is still one of the most used. Photo apps, backup tools, data lakes and static websites all keep their files in S3. It stores any amount of data, keeps it safe across several data centres, and lets your applications read and write it through simple API calls.

In this lesson you will learn how S3 organises data, its secure defaults, how to choose a storage class, and how to upload, protect, share and automatically archive files.

Buckets, objects and keys

• An object is a file plus metadata, up to 50 TB in size (raised from 5 TB in December 2025). A single upload request can send up to 5 GB; larger files use multipart upload, which the CLI and SDKs do automatically.
• A bucket is a container for objects. It lives in one Region you choose.
• A key is the object's full name, such as notes/semester-3.pdf. S3 has no real folders; the console shows the part before / (a prefix) as a folder for convenience.

Bucket names are usually globally unique across all AWS accounts, 3 to 63 characters, using lowercase letters, numbers, dots and hyphens. Since March 2026 you can also create buckets in your account regional namespace, where names end with your account ID, Region and -an (for example priya-notes-111122223333-ap-south-1-an), so nobody else can take your name.

S3 is designed for very high durability (eleven 9s, 99.999999999%) by storing data across multiple Availability Zones, and it offers strong read-after-write consistency: once an upload succeeds, every read sees the new version.

Secure by default

New buckets start locked down:

• Block Public Access is on, so policies or ACLs cannot make the bucket public by accident.
• Object Ownership is set to "Bucket owner enforced", which disables old-style access control lists (ACLs); access is controlled only by IAM and bucket policies.
• Server-side encryption with S3 managed keys (SSE-S3) encrypts every new object automatically. You can choose KMS keys (SSE-KMS) for extra control and audit.

Keep these defaults unless you have a clear reason. Most S3 data leaks happened when someone turned them off.

Working with the CLI

The high-level aws s3 commands feel like copying files:

aws s3 mb s3://priya-notes-2026 --region ap-south-1
aws s3 cp semester-3.pdf s3://priya-notes-2026/notes/semester-3.pdf
aws s3 ls s3://priya-notes-2026/notes/

make_bucket: priya-notes-2026
upload: ./semester-3.pdf to s3://priya-notes-2026/notes/semester-3.pdf
2026-10-30 12:40:03     482113 semester-3.pdf

aws s3 sync copies only new or changed files, which is perfect for publishing a website folder. Add --delete to remove files from the bucket that you deleted locally, and try --dryrun first:

aws s3 sync ./site s3://kochi-bakery-site/ --delete --dryrun

For settings such as versioning, policies and lifecycle rules, use the lower-level aws s3api commands, which map one-to-one to the S3 API.

Storage classes

You pay mainly for storage per GB-month, requests and data transfer, and the price depends on the storage class:

Infrequent-access and archive classes have minimum storage durations and retrieval charges, so moving data that you read every day into them costs more, not less. Check the S3 pricing page before choosing.

Versioning and lifecycle rules

Versioning keeps every version of an object, so an accidental overwrite or delete can be undone:

aws s3api put-bucket-versioning --bucket priya-notes-2026 \
  --versioning-configuration Status=Enabled

Lifecycle rules move or delete objects automatically. This rule, lifecycle.json, moves notes to Glacier Instant Retrieval after 90 days, deletes them after two years, and removes old non-current versions after 30 days:

{
  "Rules": [
    {
      "ID": "archive-old-notes",
      "Status": "Enabled",
      "Filter": { "Prefix": "notes/" },
      "Transitions": [{ "Days": 90, "StorageClass": "GLACIER_IR" }],
      "Expiration": { "Days": 730 },
      "NoncurrentVersionExpiration": { "NoncurrentDays": 30 }
    }
  ]
}

aws s3api put-bucket-lifecycle-configuration --bucket priya-notes-2026 \
  --lifecycle-configuration file://lifecycle.json

Bucket policies

A bucket policy is a resource-based policy attached to the bucket. A good first policy refuses any request that does not use HTTPS:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::priya-notes-2026",
        "arn:aws:s3:::priya-notes-2026/*"
      ],
      "Condition": { "Bool": { "aws:SecureTransport": "false" } }
    }
  ]
}

Apply it with aws s3api put-bucket-policy --bucket priya-notes-2026 --policy file://policy.json. In the project you will write a bucket policy that lets only your CloudFront distribution read a private website bucket.

Share private files with presigned URLs

To let a classmate download one private file without making anything public, create a presigned URL: a link signed with your credentials that works for a limited time.

from urllib.parse import parse_qs, urlparse

import boto3
from botocore.config import Config

s3 = boto3.client("s3", region_name="ap-south-1", config=Config(signature_version="s3v4"))
s3.upload_file("semester-3.pdf", "priya-notes-2026", "notes/semester-3.pdf")

url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "priya-notes-2026", "Key": "notes/semester-3.pdf"},
    ExpiresIn=600,
)
print("Valid for", parse_qs(urlparse(url).query)["X-Amz-Expires"][0], "seconds")

Valid for 600 seconds

The CLI equivalent is aws s3 presign s3://priya-notes-2026/notes/semester-3.pdf --expires-in 600 (the default is one hour). A presigned URL stops working early if the credentials that signed it expire, so links signed with temporary credentials last at most as long as that session.

Common mistakes

• Turning off Block Public Access to "fix" an access error. Fix the policy instead.
• Using a name that reveals private information, such as a customer's name. Bucket names can appear in DNS and logs.
• Archiving hot data into Glacier classes and paying retrieval charges every day.
• Forgetting old versions. With versioning on, deleted and overwritten objects still use storage until a lifecycle rule removes them.
• Hard-coding public URLs to private objects instead of generating presigned URLs.

Key points

• S3 stores objects (files plus metadata) in buckets; keys with / look like folders but are just names.
• Objects can be up to 50 TB; S3 is highly durable and strongly consistent.
• New buckets block public access, disable ACLs and encrypt objects by default; keep it that way.
• Choose storage classes by access pattern, and use lifecycle rules to move and expire data automatically.
• Versioning protects against overwrites and deletes; bucket policies control access at the bucket level.
• Presigned URLs share private objects safely for a limited time.

aws s3 mb s3://jaipur-photo-contest-2026 --region ap-south-1
aws s3api put-bucket-versioning --bucket jaipur-photo-contest-2026 \
  --versioning-configuration Status=Enabled
aws s3 cp entry-001.jpg s3://jaipur-photo-contest-2026/entries/entry-001.jpg

{
  "Rules": [
    {
      "ID": "deep-archive-entries",
      "Status": "Enabled",
      "Filter": { "Prefix": "entries/" },
      "Transitions": [{ "Days": 180, "StorageClass": "DEEP_ARCHIVE" }]
    }
  ]
}