I can't speak to CrateDB's tests, but the article I linked to said it took them 3 days to load ~3 billion metrics using 20 clients, which is on-par with our findings too.
For our tests, we used TSBS and used the "cpu-only" use case to simulate 100 hosts. That test creates 1,000 time-series across 10 metrics every 10 seconds. On Wednesday, November 25, 2020 at 11:36:36 AM UTC-5 Stuart Clark wrote: > On 25/11/2020 16:27, Ryan Booz wrote: > > As the makers of Promscale, we're very attuned to the needs of > > effective Prometheus deployments. With that in mind, one thing to > > consider with Timestream is that ingest performance from a single > > client seems to be a current limitation. The creator of this adaptor > > doesn't mention his his setup or how many metrics he was trying to > > ingest per minute or second. > > > > In recent benchmarks by CrateDB > > (https://crate.io/a/amazon-timestream-first-impressions/) and > > Timescale (not yet published), it appears that Timestream only > > achieves a consistent ingest rate in the range of 500-800 > > metrics/second from a single client, especially when lots of > > attributes are involved.. Higher throughput is achieved using a > > streaming service (ie. Kinesis) or adding more clients. In our tests > > using the open-source Time-series Benchmarking Suite > > (https://github.com/timescale/tsbs), we ended up using 10 EC2 clients > > to import data for about 36 hours and were only able to achieve > > (effectively) 5,000 metrics/sec, meaning clients averaged ~550 > > metrics/sec. So, definitely test your throughput and make sure the > > system can keep up with ingesting data. > > Was that metrics per second or time series per second? If metrics, how > many labels were there & how many time series did that equate to? > > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/b6fff442-3445-48d3-b6ec-d2a54c0d603fn%40googlegroups.com.
