kfaraz commented on pull request #11950: URL: https://github.com/apache/druid/pull/11950#issuecomment-978796502
Thanks for the changes, @gianm . On running ingestion of sample "wikipedia" data, I got the following persisted stats in the indexing logs. Before the changes (total 38 persistence cycles, 652 rows persisted in each cycle): ``` ... 2021-11-24T15:55:26,139 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [1304], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [1], persisted fireHydrants (across sinks): [1] 2021-11-24T15:55:26,244 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [1956], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [2], persisted fireHydrants (across sinks): [2] 2021-11-24T15:55:26,353 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [2608], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [3], persisted fireHydrants (across sinks): [2] 2021-11-24T15:55:26,436 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [3260], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [4], persisted fireHydrants (across sinks): [2] 2021-11-24T15:55:26,509 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [3912], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [5], persisted fireHydrants (across sinks): [2] 2021-11-24T15:55:26,578 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] ... ``` After the changes (total 1 persistence cycle, ~24k rows persisted in one cycle): ``` 2021-11-25T03:16:30,200 INFO [[index_parallel_test_sketchy_ghobnhcj_2021-11-25T03:15:57.793Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [24433], persisted rows[24433], sinks: [1], total fireHydrants (across sinks): [1], persisted fireHydrants (across sinks): [1] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
