dd-annarose commented on code in PR #25337: URL: https://github.com/apache/datafusion/pull/25337#discussion_r4182803048
########## datafusion/sqllogictest/test_files/aggregate_memory_spill.slt: ########## @@ -227,11 +227,18 @@ FROM ( GROUP BY (v * 7) % 100000 ) ---- -<slt:ignore> -06)----------AggregateExec: mode=FinalPartitioned, gby=[t.v * Int64(7) % Int64(100000)@0 as t.v * Int64(7) % Int64(100000)], aggr=[sum(t.v)], metrics=[<slt:ignore>spilled_rows=<slt:ignore>K,<slt:ignore>] -<slt:ignore> -08)--------------AggregateExec: mode=Partial, gby=[v@0 * 7 % 100000 as t.v * Int64(7) % Int64(100000)], aggr=[sum(t.v)], metrics=[<slt:ignore>early_emit_count=<slt:ignore>] -<slt:ignore> +Plan with Metrics +01)ProjectionExec: expr=[count(Int64(1))@0 as count(*), sum(total)@1 as sum(total)], metrics=[output_rows=1, elapsed_compute=<slt:ignore>, output_bytes=16.0 B, output_batches=1, expr_0_eval_time=<slt:ignore>, expr_1_eval_time=<slt:ignore>] +02)--AggregateExec: mode=Final, gby=[], aggr=[count(Int64(1)), sum(total)], metrics=[output_rows=1, elapsed_compute=<slt:ignore>, output_bytes=16.0 B, output_batches=1, agg_expr_0_arguments_time=<slt:ignore>, agg_expr_0_evaluate_time=<slt:ignore>, agg_expr_0_merge_time=<slt:ignore>, agg_expr_1_arguments_time=<slt:ignore>, agg_expr_1_evaluate_time=<slt:ignore>, agg_expr_1_merge_time=<slt:ignore>] +03)----CoalescePartitionsExec, metrics=[output_rows=4, elapsed_compute=<slt:ignore>, output_bytes=64.0 B, output_batches=4] +04)------AggregateExec: mode=Partial, gby=[], aggr=[count(Int64(1)), sum(total)], metrics=[output_rows=4, elapsed_compute=<slt:ignore>, output_bytes=64.0 B, output_batches=4, agg_expr_0_arguments_time=<slt:ignore>, agg_expr_0_state_time=<slt:ignore>, agg_expr_0_update_time=<slt:ignore>, agg_expr_1_arguments_time=<slt:ignore>, agg_expr_1_state_time=<slt:ignore>, agg_expr_1_update_time=<slt:ignore>] +05)--------ProjectionExec: expr=[sum(t.v)@1 as total], metrics=[output_rows=100.0 K, elapsed_compute=<slt:ignore>, output_bytes=787.4 KB, output_batches=788, expr_0_eval_time=<slt:ignore>] +06)----------AggregateExec: mode=FinalPartitioned, gby=[t.v * Int64(7) % Int64(100000)@0 as t.v * Int64(7) % Int64(100000)], aggr=[sum(t.v)], metrics=[output_rows=100.0 K, elapsed_compute=<slt:ignore>, output_bytes=<slt:ignore> MB, output_batches=788, spill_count=<slt:ignore>, spilled_bytes=<slt:ignore> MB, spilled_rows=<slt:ignore> K, agg_expr_0_arguments_time=<slt:ignore>, agg_expr_0_evaluate_time=<slt:ignore>, agg_expr_0_merge_time=<slt:ignore>, agg_expr_0_state_time=<slt:ignore>, aggregate_arguments_time=<slt:ignore>, aggregation_time=<slt:ignore>, emitting_time=<slt:ignore>, time_calculating_group_ids=<slt:ignore>] +07)------------RepartitionExec: partitioning=Hash([t.v * Int64(7) % Int64(100000)@0], 4), input_partitions=4, metrics=[output_rows=100.0 K, elapsed_compute=<slt:ignore>, output_bytes=<slt:ignore> KB, output_batches=784, spill_count=<slt:ignore>, spilled_bytes=<slt:ignore> KB, spilled_rows=<slt:ignore>, fetch_time=<slt:ignore>, repartition_time=<slt:ignore>, send_time=<slt:ignore>] +08)--------------AggregateExec: mode=Partial, gby=[v@0 * 7 % 100000 as t.v * Int64(7) % Int64(100000)], aggr=[sum(t.v)], metrics=[output_rows=100.0 K, elapsed_compute=<slt:ignore>, output_bytes=<slt:ignore> MB, output_batches=782, spill_count=0, spilled_bytes=0.0 B, spilled_rows=0, early_emit_count=<slt:ignore>, skipped_aggregation_rows=0, agg_expr_0_arguments_time=<slt:ignore>, agg_expr_0_convert_to_state_time=<slt:ignore>, agg_expr_0_state_time=<slt:ignore>, agg_expr_0_update_time=<slt:ignore>, aggregate_arguments_time=<slt:ignore>, aggregation_time=<slt:ignore>, emitting_time=<slt:ignore>, time_calculating_group_ids=<slt:ignore>, reduction_factor=100% (100.0 K/100.0 K)] +09)----------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1, maintains_sort_order=true, metrics=[output_rows=100.0 K, elapsed_compute=<slt:ignore>, output_bytes=<slt:ignore> KB, output_batches=782, spill_count=<slt:ignore>, spilled_bytes=<slt:ignore> KB, spilled_rows=<slt:ignore> K, fetch_time=<slt:ignore>, repartition_time=<slt:ignore>, send_time=<slt:ignore>] +10)------------------ProjectionExec: expr=[value@0 as v], metrics=[output_rows=100.0 K, elapsed_compute=<slt:ignore>, output_bytes=782.0 KB, output_batches=782, expr_0_eval_time=<slt:ignore>] +11)--------------------LazyMemoryExec: partitions=1, batch_generators=[generate_series: start=1, end=100000, batch_size=128], metrics=[output_rows=100.0 K, elapsed_compute=<slt:ignore>, output_bytes=782.0 KB, output_batches=782] Review Comment: main changed since I pushed this code so my "fix" is not needed anymore. I'll revert this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
