Sorry - just realized I mistyped that second sentence in the midst of trying to spell it out and crossed terminology. It should have read:
For our tests, we used TSBS and used the "cpu-only" use case to simulate 100 hosts. That test creates 10 CPU time-series for 100 hosts, every 10 seconds - essentially 1,000 samples every 10 seconds (6,000/minute). On Wednesday, November 25, 2020 at 12:04:52 PM UTC-5 Ryan Booz wrote: > I can't speak to CrateDB's tests, but the article I linked to said it took > them 3 days to load ~3 billion metrics using 20 clients, which is on-par > with our findings too. > > For our tests, we used TSBS and used the "cpu-only" use case to simulate > 100 hosts. That test creates 1,000 time-series across 10 metrics every 10 > seconds. > On Wednesday, November 25, 2020 at 11:36:36 AM UTC-5 Stuart Clark wrote: > >> On 25/11/2020 16:27, Ryan Booz wrote: >> > As the makers of Promscale, we're very attuned to the needs of >> > effective Prometheus deployments. With that in mind, one thing to >> > consider with Timestream is that ingest performance from a single >> > client seems to be a current limitation. The creator of this adaptor >> > doesn't mention his his setup or how many metrics he was trying to >> > ingest per minute or second. >> > >> > In recent benchmarks by CrateDB >> > (https://crate.io/a/amazon-timestream-first-impressions/) and >> > Timescale (not yet published), it appears that Timestream only >> > achieves a consistent ingest rate in the range of 500-800 >> > metrics/second from a single client, especially when lots of >> > attributes are involved.. Higher throughput is achieved using a >> > streaming service (ie. Kinesis) or adding more clients. In our tests >> > using the open-source Time-series Benchmarking Suite >> > (https://github.com/timescale/tsbs), we ended up using 10 EC2 clients >> > to import data for about 36 hours and were only able to achieve >> > (effectively) 5,000 metrics/sec, meaning clients averaged ~550 >> > metrics/sec. So, definitely test your throughput and make sure the >> > system can keep up with ingesting data. >> >> Was that metrics per second or time series per second? If metrics, how >> many labels were there & how many time series did that equate to? >> >> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/0d6cbebb-ff39-4f54-8bd4-e3f8e80210cfn%40googlegroups.com.
