Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21070
Simple scan benchmarks:
code:
https://github.com/apache/spark/compare/master...maropu:DataSourceReadBenchmark
master: https://gist.github.com/maropu/a767d21ed1dd047ec2bdca92915dc5c5
this pr: https://gist.github.com/maropu/b9312488ec791caec9711ba64027e3e2
Pushdown benchmarks:
code:
https://github.com/apache/spark/compare/master...maropu:UpdateParquetBenchmark
this pr: https://gist.github.com/maropu/e7cbfb8dc73a91806088e159f64e113f
Obviously, the pushdown for strings didn't work and I don't look into why
now.
Can you check this? @rdblue--- --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
