Davis Zhang created HUDI-9597:
---------------------------------
Summary: Schema reconcilation issue
Key: HUDI-9597
URL: https://issues.apache.org/jira/browse/HUDI-9597
Project: Apache Hudi
Issue Type: Bug
Reporter: Davis Zhang
on branch 0.14, we have
{code:java}
// Save internal schema
private void saveInternalSchema(HoodieTable table, String instantTime,
HoodieCommitMetadata metadata) {
TableSchemaResolver schemaUtil = new
TableSchemaResolver(table.getMetaClient());
String historySchemaStr =
schemaUtil.getTableHistorySchemaStrFromCommitMetadata().orElse("");
FileBasedInternalSchemaStorageManager schemasManager = new
FileBasedInternalSchemaStorageManager(table.getMetaClient());
if (!historySchemaStr.isEmpty() ||
Boolean.parseBoolean(config.getString(HoodieCommonConfig.RECONCILE_SCHEMA.key())))
{
InternalSchema internalSchema;
Schema avroSchema =
HoodieAvroUtils.createHoodieWriteSchema(config.getSchema(),
config.allowOperationMetadataField());
if (historySchemaStr.isEmpty()) {
internalSchema =
SerDeHelper.fromJson(config.getInternalSchema()).orElse(AvroInternalSchemaConverter.convert(avroSchema));
internalSchema.setSchemaId(Long.parseLong(instantTime));
} else {
internalSchema =
InternalSchemaUtils.searchSchema(Long.parseLong(instantTime),
SerDeHelper.parseSchemas(historySchemaStr));
}
InternalSchema evolvedSchema =
AvroSchemaEvolutionUtils.reconcileSchema(avroSchema, internalSchema); {code}
the AvroSchemaEvolutionUtils.reconcileSchema take 2 inputs
avroSchema - which is the writer schema populated in string format from the
writer config
internalSchema - which is read from the branch
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
