S3 Optimization

Jetscale provides AI-powered cost optimization for Amazon S3 (Simple Storage Service) buckets. Our specialized agents analyze your storage workloads to identify tiering opportunities, lifecycle policy optimizations, encryption cost reductions, and network routing savings.

Overview

Jetscale optimizes S3 buckets by analyzing:

• Storage class distribution: How data is spread across Standard, Infrequent Access, Glacier, and other tiers
• Access patterns: Request frequency and data transfer volumes
• Configuration state: Intelligent-Tiering, lifecycle rules, encryption settings
• Network routing: Whether S3 traffic is flowing through costly NAT Gateways

What We Optimize

Unlike compute services where Jetscale selects a single best alternative, S3 optimizations are non-exclusive. Jetscale can recommend any combination of the following:

• Intelligent-Tiering: Automatically move objects between access tiers based on usage
• Lifecycle rules: Transition objects to cheaper storage classes over time
• Bucket Key (KMS): Reduce KMS API call costs by up to 99%
• S3 Gateway Endpoints: Eliminate NAT Gateway processing fees for S3 traffic

Optimization Types

Enable Intelligent-Tiering

Best for: Buckets with unknown or changing access patterns

Intelligent-Tiering automatically moves objects between access tiers without performance impact or retrieval fees.

• Monitoring fee: $0.0025 per 1,000 objects
• No retrieval fees (unlike Standard-IA or Glacier)
• Objects accessed frequently stay in Standard tier
• Objects not accessed for 30 days move to Infrequent Access (45% cheaper)
• Objects not accessed for 90 days move to Archive Instant Access (68% cheaper)

When Jetscale recommends this:

• Bucket does not already have Intelligent-Tiering enabled
• Bucket is a General Purpose bucket (not Directory)
• Bucket contains objects with mixed or unknown access patterns
• Typical savings: 20-40% on infrequently accessed data

Add Lifecycle Rules

Best for: Buckets with known data aging patterns, logs, backups, and archival data

Lifecycle rules automatically transition objects to cheaper storage classes based on age.

Example transition flow:

• Standard (Day 0) → Standard-IA at 30 days (45% cheaper)
• Standard-IA → Glacier at 90 days (68% cheaper)
• Glacier → Deep Archive at 180 days (95% cheaper)

When Jetscale recommends this:

• Bucket does not already have lifecycle rules
• S3 Analytics data (if available) shows significant cold data
• AWS-recommended transition age is identified
• Typical savings: 45-95% on aged data depending on transition target

Enable Bucket Key

Best for: KMS-encrypted buckets with high request volumes

Bucket Key reduces the number of KMS API calls by using a bucket-level key for encryption, cutting KMS costs by up to 99%.

• Savings: ~$0.03 per 10,000 requests saved
• Zero performance impact
• Only applies to buckets using aws:kms encryption

When Jetscale recommends this:

• Bucket uses KMS encryption (aws:kms)
• Bucket Key is not already enabled
• Bucket has measurable PUT/GET request volume

Add S3 Gateway Endpoint

Best for: VPCs routing S3 traffic through NAT Gateways

This is often the highest-impact S3 optimization. S3 Gateway Endpoints are completely free and eliminate NAT Gateway data processing fees of $0.045/GB.

• Gateway Endpoints: Free
• NAT Gateway processing: $0.045/GB ($45/TB)
• Savings: $45 per TB of S3 traffic routed through NAT

When Jetscale recommends this:

• VPCs in the bucket's region have NAT Gateways
• Those VPCs do not have S3 Gateway Endpoints configured
• The bucket has measurable data transfer volume
• Typical savings: $45/TB of S3 traffic

How Jetscale Optimizes S3

1. Data Collection

Jetscale analyzes multiple data sources:

CloudWatch Metrics (30-day lookback):

• Storage size by storage class (Standard, Glacier, etc.)
• Total object count
• Request metrics (PUT, GET, LIST, HEAD, DELETE)
• Data transfer (bytes downloaded/uploaded)
• Activity history and peak usage

S3 Configuration:

• Intelligent-Tiering status
• Lifecycle rule configuration
• Encryption algorithm and Bucket Key status
• Versioning, replication, public access settings
• Bucket type (General Purpose vs. Directory)

S3 Analytics (if enabled):

• Object age distribution
• Access frequency by age bracket
• AWS-recommended lifecycle transition age

Network Analysis (cross-resource):

• NAT Gateways in the bucket's region
• VPCs with and without S3 Gateway Endpoints
• Data transfer volumes through NAT

AWS Pricing Data:

• Storage class pricing per GB/month
• Request pricing (PUT, GET, LIST)
• Data transfer pricing
• Intelligent-Tiering monitoring fees

2. Analysis

Our AI agents perform deep analysis:

Storage Patterns:

• Distribution of data across storage classes
• Object age distribution (via S3 Analytics when available)
• Access frequency trends
• Growth rate and total volume

Cost Modeling:

• Current cost breakdown (storage + requests + transfer + KMS)
• Projected cost for each optimization action
• Combined savings when applying multiple actions

Safety Checks:

• Empty buckets return no recommendation (AWS security best practice to prevent bucket name hijacking)
• Directory buckets excluded from storage optimizations (unsupported)
• Already-enabled features skipped (no duplicate recommendations)
• KMS encryption required for Bucket Key recommendations

3. Recommendations

Jetscale can select any combination of optimizations for a single bucket, unlike compute services which pick one alternative.

Selection priority:

1. S3 Gateway Endpoint (free, immediate, highest dollar impact)
2. Intelligent-Tiering (automatic, low risk)
3. Lifecycle Rules (highest percentage savings on aged data)
4. Bucket Key (KMS cost reduction)

Combined Recommendation Example

Resource: my-application-logs (us-east-1)
Current: Standard, 500 GB, KMS encrypted, no lifecycle rules
Monthly Cost: $11.50 (storage) + $45 (NAT transfer on 1TB) = $56.50

Selected Actions:
1. Add S3 Gateway Endpoint
   - Savings: $45/month (eliminates NAT processing)
   - Effort: Low | Risk: Low

2. Enable Intelligent-Tiering
   - Savings: ~$3.50/month (cold data moves to IA tier)
   - Effort: Low | Risk: Low

3. Enable Bucket Key
   - Savings: ~$0.30/month (99% fewer KMS calls)
   - Effort: Low | Risk: Low

Combined Savings: $48.80/month, $585.60/year
Combined Risk: Low
Implementation Effort: Low

Enable Intelligent-Tiering Example

Resource: user-uploads-bucket (us-west-2)
Current: Standard, 2 TB, 5M objects
Monthly Cost: $47.10/month (storage)

S3 Analytics shows:
- 800 GB accessed in last 30 days
- 1.2 TB not accessed in 30+ days

Recommended: Enable Intelligent-Tiering
- 1.2 TB moves to IA tier automatically
- Monitoring fee: $12.50/month (5M objects)
- IA storage savings: $13.80/month

Cost Impact:
- Current: $47.10/month
- Projected: $45.80/month (storage) + monitoring ($12.50) - IA savings ($13.80)
- Net Savings: ~$2.60/month initially, increasing as more data ages

Risk: Low - No retrieval fees, automatic tier management
Note: Savings increase over time as more data becomes infrequent

Lifecycle Rule Example

Resource: application-logs (us-east-1)
Current: Standard, 1 TB, all in Standard class
Monthly Cost: $23.55/month

S3 Analytics recommendation:
- Transition data older than 45 days
- 600 GB is older than 45 days

Recommended: Add Lifecycle Rule
- Standard → Standard-IA at 45 days
- Standard-IA → Glacier at 90 days

Cost Impact:
- Current: $23.55/month
- Projected: ~$12.00/month
- Savings: ~$11.55/month (49%), $138.60/year

Risk: Low - Glacier data still retrievable (higher latency)
Note: Review retrieval needs before enabling Glacier transitions

S3 Gateway Endpoint Example

Resource: data-pipeline-bucket (us-east-1)
Current: 5 TB/month data transfer through NAT Gateway

Network Analysis:
- VPC vpc-abc123 has NAT Gateway, no S3 endpoint
- VPC vpc-def456 has NAT Gateway, no S3 endpoint
- Monthly S3 transfer: 5 TB

Recommended: Add S3 Gateway Endpoints to both VPCs
- Gateway Endpoints are FREE
- Eliminates: $0.045/GB × 5,000 GB = $225/month

Cost Impact:
- Current NAT processing: $225/month
- Projected: $0/month
- Savings: $225/month (100%), $2,700/year

Risk: Low - Gateway Endpoints are transparent to applications
VPCs to update: vpc-abc123, vpc-def456

4. Terraform Generation

For each recommendation, Jetscale generates production-ready Terraform code:

Example: Intelligent-Tiering Configuration

# S3 Intelligent-Tiering Configuration
# Generated by Jetscale on 2024-01-15
# Recommendation ID: rec_s3_001

resource "aws_s3_bucket_intelligent_tiering_configuration" "optimization" {
  bucket = "user-uploads-bucket"
  name   = "EntireBucket"

  tiering {
    access_tier = "DEEP_ARCHIVE_ACCESS"
    days        = 180
  }

  tiering {
    access_tier = "ARCHIVE_ACCESS"
    days        = 90
  }
}

Example: Lifecycle Rule

# S3 Lifecycle Rule Configuration
# Generated by Jetscale on 2024-01-15

resource "aws_s3_bucket_lifecycle_configuration" "optimization" {
  bucket = "application-logs"

  rule {
    id     = "jetscale-cost-optimization"
    status = "Enabled"

    transition {
      days          = 45
      storage_class = "STANDARD_IA"
    }

    transition {
      days          = 90
      storage_class = "GLACIER"
    }

    transition {
      days          = 180
      storage_class = "DEEP_ARCHIVE"
    }
  }
}

Example: S3 Gateway Endpoint

# VPC S3 Gateway Endpoint
# Generated by Jetscale on 2024-01-15
# Eliminates NAT Gateway processing fees for S3 traffic

resource "aws_vpc_endpoint" "s3" {
  vpc_id       = "vpc-abc123"
  service_name = "com.amazonaws.us-east-1.s3"

  tags = {
    Name                       = "s3-gateway-endpoint"
    "jetscale:optimized"       = "true"
    "jetscale:recommendation"  = "rec_s3_002"
  }
}

resource "aws_vpc_endpoint_route_table_association" "s3" {
  route_table_id  = var.route_table_id
  vpc_endpoint_id = aws_vpc_endpoint.s3.id
}

Best Practices

Testing Strategy

Pre-Production Testing:

1. Enable Intelligent-Tiering on non-critical buckets first
2. Test lifecycle rules with a subset prefix before applying bucket-wide
3. Verify application compatibility with storage class transitions
4. Monitor retrieval patterns after enabling Glacier transitions

Production Rollout:

1. Intelligent-Tiering and Bucket Key are safe to enable immediately (no data movement)
2. Lifecycle rules take effect over time as objects age past thresholds
3. Gateway Endpoints are transparent to applications (no code changes needed)
4. Monitor CloudWatch S3 metrics after enabling changes

Storage Class Selection

Standard-IA considerations:

• Minimum storage duration: 30 days (charged even if deleted earlier)
• Minimum object size: 128 KB (smaller objects charged as 128 KB)
• Retrieval fee applies ($0.01/GB)

Glacier considerations:

• Retrieval time: 1-5 minutes (Expedited), 3-5 hours (Standard), 5-12 hours (Bulk)
• Not suitable for data that needs frequent or immediate access
• Minimum storage duration: 90 days

Deep Archive considerations:

• Retrieval time: 12-48 hours
• Minimum storage duration: 180 days
• Best for compliance, archival, and rarely accessed data

Gateway Endpoint Deployment

Before deploying S3 Gateway Endpoints:

1. Verify route table associations for all relevant subnets
2. Ensure S3 bucket policies don't restrict by source IP (Gateway Endpoints use private IPs)
3. Test with a single VPC before rolling out to all VPCs
4. No application changes required

Common Optimization Patterns

Pattern 1: Logs Buckets Without Lifecycle

Symptoms:

• Large buckets growing continuously
• All data in Standard storage class
• Data rarely accessed after initial analysis

Jetscale Recommendation:

• Add lifecycle rules (Standard → IA → Glacier → Deep Archive)
• Typical savings: 45-95% on aged data
• Risk: Low (verify retrieval needs first)

Pattern 2: NAT Gateway S3 Traffic

Symptoms:

• High NAT Gateway data processing charges
• S3 is a top data transfer destination
• No S3 Gateway Endpoints in VPCs

Jetscale Recommendation:

• Add S3 Gateway Endpoints to all relevant VPCs
• Typical savings: $45/TB of S3 traffic
• Risk: Low (transparent to applications)

Pattern 3: KMS-Encrypted Buckets Without Bucket Key

Symptoms:

• Buckets using aws:kms encryption
• High KMS API request costs
• Bucket Key not enabled

Jetscale Recommendation:

• Enable Bucket Key
• Typical savings: Up to 99% reduction in KMS API costs
• Risk: Low (transparent to applications)

Pattern 4: Mixed Access Patterns Without Tiering

Symptoms:

• All data in Standard class
• Mix of frequently and rarely accessed objects
• Access patterns are unpredictable or changing

Jetscale Recommendation:

• Enable Intelligent-Tiering
• Typical savings: 20-40% on infrequently accessed data
• Risk: Low (no retrieval fees, automatic management)

Troubleshooting

Recommendation Concerns

Q: Will enabling Intelligent-Tiering cause any downtime?

A: No. Intelligent-Tiering is a storage class configuration change that takes effect immediately. Objects are moved between tiers automatically without any impact to access. There are no retrieval fees when accessing objects in any tier.

Q: What if I need to access data that was transitioned to Glacier?

A: You can still access Glacier data, but retrieval takes longer (1-12 hours depending on retrieval tier). If you need faster access, consider using Intelligent-Tiering instead of lifecycle rules with Glacier transitions, as it provides automatic access tier management without retrieval delays.

Q: Can I undo a lifecycle rule?

A: Yes. You can modify or delete lifecycle rules at any time. However, objects already transitioned to a different storage class will remain in that class until accessed or manually moved. New objects and objects that haven't been transitioned yet will follow the updated rules.

Q: Will S3 Gateway Endpoints affect my existing application?

A: No. S3 Gateway Endpoints are transparent to applications. They route S3 traffic through the AWS private network instead of the public internet/NAT Gateway. Your application code, S3 URLs, and SDK calls all work identically. The only change is the network path.

Performance Concerns

Q: Does Intelligent-Tiering have a monitoring overhead?

A: The only cost is the monitoring fee of $0.0025 per 1,000 objects/month. There is no performance overhead. For buckets with millions of objects, evaluate whether the monitoring fee is offset by the tiering savings.

Q: What happens to S3 Analytics data after enabling optimizations?

A: S3 Analytics continues to collect data independently. You can use it to validate that your lifecycle rules and tiering configurations are performing as expected.

Limitations

Not Currently Supported:

• Cross-region replication optimization
• S3 Inventory cost optimization
• Object Lock configuration changes
• Bucket policy or ACL modifications
• Multipart upload cleanup
• Versioning configuration changes

S3-Specific Constraints:

• Empty buckets receive no recommendation (security best practice)
• Directory buckets are excluded from storage optimizations
• Lifecycle rules and Intelligent-Tiering are independent features (both can be enabled)
• Gateway Endpoints are per-VPC and per-region

API Integration

Jetscale provides API access for programmatic optimization:

# List S3 recommendations
GET /api/v1/recommendations?resource_type=s3

# Get specific recommendation details
GET /api/v1/recommendations/{recommendation_id}

# Approve recommendation (generates Terraform)
POST /api/v1/recommendations/{recommendation_id}/approve

# Retrieve generated Terraform
GET /api/v1/recommendations/{recommendation_id}/terraform

See our API Documentation for complete reference.

Support

Need help with S3 optimization?

• Email: support@jetscale.ai
• Documentation: FAQ
• GitHub Issues: Report a problem

Related Documentation:

• EC2 Optimization
• RDS Optimization
• EBS Optimization
• ElastiCache Optimization
• EKS Optimization