Cockroach Labs Estate economics What idle nodes, provisioned bytes and peak-sized bills cost, worked out from your estate.
  1. Nodes 3 baseline
  2. Avg CPU utilization 15.0% idle hardware
  3. Storage provisioned 150.0 TB 25.0 TB of data
  4. Est. monthly saving 0%

Step 4 of 8 - Storage

Scaling without copying the data

One action, both lanes. The only difference is where the bytes already are.

Fire an action Add a node Remove a node Scale to zero

Stateful nodes on EBS

75.0 TB to move

0% after 0 s, 27 d 19 h still to goStateful nodes on EBS progress75.0 TB to move. 0% after 0 s, 27 d 19 h still to go.

Rebalancing ranges: 0 GB of 75.0 TB copied in 0 s.

Data to move150000 ranges x 512 MB75.0 TB
Copy raterebalance snapshots are rate-limited per store32 MiB/s
Time to complete75.0 TB at 32 MiB/s, and the docs give a band of minutes to hours27 d 19 h
Consequencecan't shrink quickly, so provision for peakpeak-provisioned

Stateless KV nodes, disaggregated storage

9375 KB of metadata

0% after 0 s, under a second still to goStateless KV nodes, disaggregated storage progress9375 KB of metadata. 0% after 0 s, under a second still to go.

Creating hard links: the ranges are already in the storage layer.

Hard links created150000 ranges shared, not copied150000 links
Metadata written150000 x 64 bytes9375 KB
Time to complete9375 KB at 32 MiB/sunder a second
Bulk data movednone: the range's SSTables are shared, not copied0 GB
Still sentthe range's store-local state, which no hard link sharessmall
How this is worked out
  • Both bars run in real time. A second on screen is a second of the actual operation: the hard links are written in under a second and the copy needs 27 d 19 h, so the left-hand bar is still going when you leave the page. There's nothing to wait for.
  • Moving a range is rate-limited per store, so the duration is arithmetic: 75.0 TB at 32 MiB/s, which is CockroachDB's default kv.snapshot_rebalance.max_rate. It's the optimistic end - a real rebalance shares the link with your queries - and it lands inside the "minutes to hours" the design docs give.