API Gateway access logs cost: how to estimate ingestion and retention
API Gateway access logging is easy to enable, but at high request volume it can become a real bill line item. A good estimate needs three parts: ingestion (GB/day), retention (GB-month), and queries (if you run searches/analytics over the logs).
Where the cost shows up
- Ingestion: GB ingested into your logging destination (often CloudWatch Logs).
- Storage: retained GB-month (retention policy creates the steady-state storage baseline).
- Queries/scans: log searches and dashboards (often the surprise during incidents).
If you already have a log pipeline, use measured ingestion and query scan volumes instead of estimating from first principles.
Step 1: estimate average bytes per log line
- Pick a representative window (normal traffic + a small slice of peak).
- Sample a few thousand log lines and compute average bytes/line (and p95 if sizes vary a lot).
- Watch the big fields: long headers, JWT claims, user agents, and referrers.
Step 2: convert requests into GB/day
Approximation: GB/day ~= requests/day x bytes/log / 1e9.
If you only have requests/month, use: requests/day ~= requests/month / 30.4.
Step 3: model retention (GB-month)
Retention creates a steady storage baseline. If you ingest 10 GB/day and retain 30 days, steady-state stored volume is roughly 300 GB.
Model retention separately from ingestion: they are different rate cards in many tools.
Step 4: add query/scan costs (often the surprise)
- Incident response: repeated broad searches can scan huge volumes.
- Dashboards: scheduled queries add predictable recurring scans.
- Bad filters: scanning unpartitioned data is expensive.
Tool: Log search/scan cost calculator.
How to reduce access log cost (without losing signal)
- Make the log format intentional: drop large headers; log stable identifiers; avoid duplicating payload fields.
- Separate “always-on” vs “debug”: keep a slim always-on format and enable verbose logging only during investigations.
- Retention by value: short retention for noisy success logs, longer retention for security signals where required.
- Prevent retry storms: timeouts and retries multiply both requests and log volume.
Validation checklist
- Validate requests/day from a representative window (include weekday/weekend patterns).
- Validate bytes/log using a real sample after finalizing the log format.
- Confirm retention policy (days) and whether logs are replicated/forwarded elsewhere.
- Estimate query frequency and scan size for dashboards and incident workflows.
Related links
Sources
- API Gateway logging: docs.aws.amazon.com
- CloudWatch Logs pricing: aws.amazon.com/cloudwatch/pricing