Re: [PR] Enable reading `StringViewArray` by default from Parquet [datafusion]

[via GitHub]

alamb commented on PR #12092:
URL: https://github.com/apache/datafusion/pull/12092#issuecomment-2354023193

   I figured out what is going on (different than I thought). I believe 
`StringView::slice()` is quite a bit slower than `StringArray::slice` due to 
the fact it has a buffers field
   
   The query is
   ```sql
   SELECT REGEXP_REPLACE("Referer", '^https?://(?:www\\.)?([^/]+)/.*$', '\\1') 
AS k, AVG(length("Referer")) AS l, COUNT(*) AS c, MIN("Referer")
   FROM hits_partitioned
   WHERE "Referer" <> '' GROUP BY k HAVING COUNT(*) > 100000 ORDER BY l DESC 
LIMIT 25;
   ```
   
   Some flamegraphs:
   
![flamegraph-main](https://github.com/user-attachments/assets/740c4dcc-5d0f-4ef1-8dd7-f61ad5459133)
   
![flamegraph-string-view](https://github.com/user-attachments/assets/1dc311d8-55e9-46d3-bdd6-12890fdfd6cd)
   
   ![Screenshot 2024-09-16 at 4 44 50 
PM](https://github.com/user-attachments/assets/551582fa-0744-4c7a-9075-b5b4102c6861)
   
   I will think about the best way to proceed here


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org