Re: [I] Investigate options for improving performance when reading decimals from Parquet [datafusion-comet]

[via GitHub]

kazuyukitanimura commented on issue #679:
URL: 
https://github.com/apache/datafusion-comet/issues/679#issuecomment-2240962098

   It looks like pure scan is not a problem based on profiling
   ### Baseline
   ![Screenshot 2024-07-19 at 10 49 16 
PM](https://github.com/user-attachments/assets/1f2bd34b-fe3c-453c-a023-ed087f6368de)
   ### Scan-only enabled
   ![Screenshot 2024-07-19 at 10 48 59 
PM](https://github.com/user-attachments/assets/059a529d-4aae-4d3a-a69b-a23c5aef45c2)
   
   ### Scan comparison (the name says add_many_decimals, but I was just lazy to 
change the name)
   ![Screenshot 2024-07-19 at 11 28 02 
PM](https://github.com/user-attachments/assets/8e389460-4c07-4c8b-a7fa-e5ffc157e8a9)
   
   Now I remember that we need to increase the minimum number of iteration to 
get a stable results. We used to use a remote Linux machine that was stable so 
2-3 iterations were fine. But for local testing, we need to increase it. For 
the third screen shot, I used 33 iterations.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org