[I] [VL] gluten reads slower than native Spark [incubator-gluten]

Thu, 12 Jun 2025 04:03:00 -0700 


wqyzstql opened a new issue, #9956:
URL: https://github.com/apache/incubator-gluten/issues/9956

   ### Backend
   
   VL (Velox)
   
   ### Bug description
   
   In the same SQL query job, native Spark only generated 9,000 tasks, but 
Gluten generated 26,475 tasks. Native Spark only took 43 minutes, but Gluten 
took 3 hours.
   
   
![Image](https://github.com/user-attachments/assets/618e660f-2645-4a29-b618-74601de0ded4)
   
   
![Image](https://github.com/user-attachments/assets/25e8590e-108a-4c5a-b618-9648ed140e43)
   
   And you can see that Gluten's time of scan and filter total takes up most of 
the time
   
   
![Image](https://github.com/user-attachments/assets/e833012c-b757-4345-8f4c-b5f9cd6a7a6d)
   
   while Spark use much less time than gluten
   
   
![Image](https://github.com/user-attachments/assets/56a72c21-d28c-4701-9ffc-366bf4295211)
   
   ### Gluten version
   
   Gluten-1.3
   
   ### Spark version
   
   Spark-3.2.x
   
   ### Spark configurations
   
   _No response_
   
   ### System information
   
   _No response_
   
   ### Relevant logs
   
   ```bash
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to