Omega359 commented on PR #20604: URL: https://github.com/apache/datafusion/pull/20604#issuecomment-3991462205
Parquet files @Rafferty97. I think I found the issue yesterday - a combination of writing out using duckdb and datafusion and one being in utf8 and the other in utf8view. Still, at the end of the codepath for datafusion I had to put in casts for utf8 -> ut8view for all utf8 fields. I'm rather curious if all of this utf8view work will have any measurable impact or if overhead of S3 and zstd decompression will just overshadow it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
