emaynardigs edited a comment on issue #27155: [SPARK-17636][SPARK-25557][SQL] 
Parquet and ORC predicate pushdown in nested fields
URL: https://github.com/apache/spark/pull/27155#issuecomment-586352041
 
 
   @dbtsai 
   
   >so I'm thinking to submit a PR based on our internal version (a modified 
version of #22535) which is proven to be stable and in production for awhile.
   
   My main reservation is that #22535 relies on a dot in the name of the field, 
and so cannot support ORC. The key difference in this PR is that it actually 
inspects the type of the field and extends the same functionality to Parquet 
and ORC. It's also rebased for the current master already and so merges 
cleanly. I also added more tests. If we merge #22535 we'll need another PR to 
un-do this logic and implement the same for ORC again.
   
   No, I intended to cherry pick this back after it merges, but if it doesn't 
get merge we'll probably end up using it on our fork much like you've done. 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to