Bigtable cost estimation: nodes, storage growth, and transfer (practical model)
Start with a calculator if you need a first-pass estimate, then use this guide to validate the assumptions and catch the billing traps.
Bigtable-like systems are capacity planned. Your bill is driven by provisioned throughput capacity plus stored data, and it becomes expensive when you are forced to over-provision for peaks or hotspots. This guide gives a simple model you can validate.
0) Identify the workload shape (what forces capacity)
- Read/write throughput: baseline vs peak (batch jobs and backfills are peak).
- Hot keys / hotspots: uneven key distribution forces more capacity.
- Latency targets: stricter tail latency often implies more headroom.
1) Provisioned capacity (node-hours)
Model baseline nodes and peak nodes separately. If peak is rare, do not pay for it 24/7; if peak is frequent, you should treat peak as the real baseline.
Tool: Compute cost model (generic monthly capacity pricing).
- Keep baseline and peak node counts as two explicit scenarios.
- If one job drives peak, model that job separately (duration per day/week).
2) Storage (GB-month) and growth
Storage estimation is about average GB across the month, including growth. If you retain multiple versions or keep long TTLs, storage grows faster than most teams expect.
Tool: Storage growth model.
- Model data retention and any versioning explicitly (retention is a multiplier).
- Validate compaction/GC behavior (retention settings and churn affect stored size).
3) Network transfer (billable boundaries)
If clients are outside the region, or you replicate/serve across regions, include outbound transfer as a separate line. Transfer often becomes meaningful when you add multi-region patterns.
Tools: Egress, Cross-region transfer.
Worked estimate template (copy/paste)
- Baseline node-hours = baseline nodes × hours/month
- Peak node-hours = (peak nodes - baseline nodes) × peak hours/month
- Stored GB-month = average stored GB across the month (include growth)
- Transfer GB/month = cross-region/internet bytes (if applicable)
Common pitfalls
- Ignoring hotspots (hot keys force capacity above average throughput).
- Paying for peak capacity 24/7 even if peak is a short batch window.
- Not modeling retention/versioning (stored GB grows quietly).
- Missing cross-region transfer when adding DR or global clients.
- Not validating with real workload traces (good-day averages hide tail risk).
How to validate
- Validate peak read/write throughput and identify peak drivers (batch jobs, backfills).
- Validate hotspot risk (key distribution, top partitions) and fix the schema before scaling blindly.
- Validate storage growth and retention (including churn and compaction behavior).
- Validate where clients are (same-region vs cross-region) and which legs are billable.