scwhittle commented on pull request #13782: URL: https://github.com/apache/beam/pull/13782#issuecomment-770755221
Not sure if there are dashboards but the perf results from console output were: 16:21:21 org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInStreaming STANDARD_OUT 16:21:21 Load test results for test (ID): 539adecb-21ea-4aaf-935c-95c9bb9b91c5 and timestamp: 2021-01-28T15:02:34.094000000Z: 16:21:21 Metric: Value: 16:21:21 read_time 1.385 16:21:21 write_time 9.316 16:21:21 run_time 10.701 The subsequent run https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/1871/console had results: 20:34:52 org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInStreaming STANDARD_OUT 20:34:52 Load test results for test (ID): cb47f5d5-0102-49ce-8fd1-73eb3e6bbe40 and timestamp: 2021-01-28T19:16:14.211000000Z: 20:34:52 Metric: Value: 20:34:52 read_time 2.873 20:34:52 write_time 14.97 20:34:52 run_time 17.843 I'm not sure how stable these are, write_time shouldn't directly be effected, but if the pipeline is doing both in parallel the CPU waste on reading could impact write performance as well. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
