Alexey Kudinkin created HUDI-3896:
-------------------------------------
Summary: Support Spark optimizations for `HadoopFsRelation`
Key: HUDI-3896
URL: https://issues.apache.org/jira/browse/HUDI-3896
Project: Apache Hudi
Issue Type: Bug
Reporter: Alexey Kudinkin
Assignee: Alexey Kudinkin
Fix For: 0.12.0
After migrating to Hudi's own Relation impls, we unfortunately broke off some
of the optimizations that Spark apply exclusively for `HadoopFsRelation`.
While these optimizations could be perfectly implemented for any
`FileRelation`, Spark is unfortunately predicating them on usage of
HadoopFsRelation, therefore making them non-applicable to any of the Hudi's
relations.
Proper longterm solutions would be fixing this in Spark and could be either of:
# Generalizing such optimizations to any `FileRelation`
# Making `HadoopFsRelation` extensible (making it non-case class)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
