GitHub user lgbo-ustc edited a comment on the discussion: [GLUTEN][FLINK] Nexmark q0 performance
@ParyshevSergey Can you provide more details about the test settings. It is expected that the new version of the code may experience a certain degree of performance regression, with a potential decrease of around 10%. Currently, our priority is the improvement of functionalities, and performance optimization in this aspect will be considered later. However, as you mentioned, we have not encountered such significant slowdowns before. We have re-run the stress tests, and the results are as follows (without `StreamRecordTimestampInserter ` which is not supported at present). #### datagen source ##### gluten ``` -------------------------------- Nexmark Results -------------------------------- +------+-----------------+--------+----------+-----------------+--------------+-----------------+ | Query| Events Num | Cores | Time(s) | Cores * Time(s) | Throughput | Throughput/Cores| +------+-----------------+--------+----------+-----------------+--------------+-----------------+ |q0 |100,000,000 |1.01 |40.124 |40.460 |2.49 M/s |2.47 M/s | |Total |100,000,000 |1.008 |40.124 |40.460 |2.49 M/s |2.47 M/s | +------+-----------------+--------+----------+-----------------+--------------+-----------------+ ``` ##### flink ``` -------------------------------- Nexmark Results -------------------------------- +------+-----------------+--------+----------+-----------------+--------------+-----------------+ | Query| Events Num | Cores | Time(s) | Cores * Time(s) | Throughput | Throughput/Cores| +------+-----------------+--------+----------+-----------------+--------------+-----------------+ |q0 |100,000,000 |1.03 |115.690 |119.576 |864.38 K/s |836.29 K/s | |Total |100,000,000 |1.034 |115.690 |119.576 |864.38 K/s |836.29 K/s | +------+-----------------+--------+----------+-----------------+--------------+-----------------+ ``` #### kafka source ##### gluten <img width="715" height="239" alt="企业微信截图_11538e53-0693-4a0c-b423-62ec7c41d044" src="https://github.com/user-attachments/assets/0198a4e3-3c6f-4685-a77d-9d66c54c5cc9"; /> ##### flink <img width="709" height="176" alt="企业微信截图_6a69335c-573b-4ed6-b7a3-ade33c3a081f" src="https://github.com/user-attachments/assets/e5ee37ef-6cc3-44ab-b63d-74f225f27f24"; /> GitHub link: https://github.com/apache/incubator-gluten/discussions/11508#discussioncomment-15638130 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
