alexeykudinkin commented on code in PR #6358:
URL: https://github.com/apache/hudi/pull/6358#discussion_r986271533
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/HoodieMergeHelper.java:
##########
@@ -74,90 +75,103 @@ public static HoodieMergeHelper newInstance() {
@Override
public void runMerge(HoodieTable<T, HoodieData<HoodieRecord<T>>,
HoodieData<HoodieKey>, HoodieData<WriteStatus>> table,
HoodieMergeHandle<T, HoodieData<HoodieRecord<T>>,
HoodieData<HoodieKey>, HoodieData<WriteStatus>> mergeHandle) throws IOException
{
- final boolean externalSchemaTransformation =
table.getConfig().shouldUseExternalSchemaTransformation();
- Configuration cfgForHoodieFile = new Configuration(table.getHadoopConf());
+ HoodieWriteConfig writeConfig = table.getConfig();
Review Comment:
This piece had to change b/c we're now setting up a projection schema when
reading from Parquet and that fails unless projection schema is matching the
file-schema. This unfortunately has not been the case as this code was evolving
over the years, so it forced me to untangle flows here to fix it.
Changes:
- Abstracted and extracted Advanced Schema Evolution into standalone method
(this code doesn't change, except it was abstracted to return a function
rewriting the record instead of returning multiple values)
- Simplified logic pertaining to record rewriting to be shared by all flows
(bootstrapping, etc)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
