GitHub user ParyshevSergey added a comment to the discussion: [GLUTEN][FLINK] Nexmark q0 performance
@lgbo-ustc Flink config above, results: <details><summary>Gluten w/ Kafka, 3 of 4 native operators (StreamRecordTimestampInserter is java operator)</summary> <p> -------------------------------- Nexmark Results -------------------------------- +------+-----------------+--------+----------+-----------------+--------------+-----------------+ | Query| Events Num | Cores | Time(s) | Cores * Time(s) | Throughput | Throughput/Cores| +------+-----------------+--------+----------+-----------------+--------------+-----------------+ |q0 |100,000,000 |0 |2995.692 |0.000 |33.38 K/s |9.22 E/s | -- metric monitor is off |q0 |100,000,000 |1 |2985.706 |2983.679 |33.49 K/s |33.52 K/s | </p> </details> <details><summary>Gluten w/ Kafka, 3 of 3 native operators (StreamRecordTimestampInserter is deleted in code)</summary> <p> -------------------------------- Nexmark Results -------------------------------- +------+-----------------+--------+----------+-----------------+--------------+-----------------+ | Query| Events Num | Cores | Time(s) | Cores * Time(s) | Throughput | Throughput/Cores| +------+-----------------+--------+----------+-----------------+--------------+-----------------+ |q0 |100,000,000 |0.36 |525.402 |187.356 |190.33 K/s |533.74 K/s | |Total |100,000,000 |0.357 |525.402 |187.356 |190.33 K/s |533.74 K/s | +------+-----------------+--------+----------+-----------------+--------------+-----------------+ 2026-01-28 16:42:33,100 INFO org.apache.gluten.client.OffloadedJobGraphGenerator [] - OperatorChainSliceGraph: Slice ID: 7, offloadable: false Inputs: [] Outputs: [] Operator Configs: nexmark_q0[3]: Writer(7) 2026-01-28 16:42:33,108 INFO org.apache.gluten.client.OffloadedJobGraphGenerator [] - OperatorChainSliceGraph: Slice ID: 2, offloadable: true Inputs: [] Outputs: [3] Operator Configs: Source: kafka[1](2) Slice ID: 3, offloadable: true Inputs: [2] Outputs: [5] Operator Configs: gluten-calc(3) Slice ID: 5, offloadable: true Inputs: [3] Outputs: [] Operator Configs: Writer(5) </p> </details> <details><summary>Flink 1.20</summary> <p> -------------------------------- Nexmark Results -------------------------------- +------+-----------------+--------+----------+-----------------+--------------+-----------------+ | Query| Events Num | Cores | Time(s) | Cores * Time(s) | Throughput | Throughput/Cores| +------+-----------------+--------+----------+-----------------+--------------+-----------------+ |q0 |100,000,000 |0.88 |275.188 |242.630 |363.39 K/s |412.15 K/s | |Total |100,000,000 |0.882 |275.188 |242.630 |363.39 K/s |412.15 K/s | +------+-----------------+--------+----------+-----------------+--------------+-----------------+ </p> </details> GitHub link: https://github.com/apache/incubator-gluten/discussions/11508#discussioncomment-15638439 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
