Alexey Kudinkin created HUDI-4036:
-------------------------------------
Summary: Investigate whether meta fields could be omitted
completely
Key: HUDI-4036
URL: https://issues.apache.org/jira/browse/HUDI-4036
Project: Apache Hudi
Issue Type: Bug
Reporter: Alexey Kudinkin
Currently, even when meta fields are not populated, we still insert
empty-string columns to adhere to the expected schema.
This has a non-trivial overhead of ~20% (relative to just writing dataset as
is), since Spark had to essentially "re-write" the original row with prepended
new fields.
We should investigate whether it's feasible to avoid adding empty-string
columns completely if meta-fields are disabled.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
