[
https://issues.apache.org/jira/browse/HUDI-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Vexler updated HUDI-5740:
----------------------------------
Description:
The configs in the following classes are implemented not using HoodieConfig,
making it impossible to be surfaced on the Configurations page. We need to
refactor the code so that each config property is implemented using
ConfigProperty in a corresponding new HoodieConfig class. Refer to
HoodieArchivalConfig for existing implementation of configs.
InitialCheckPointProvider
HoodieDeltaStreamer
HoodieMultiTableDeltaStreamer
FilebasedSchemaProvider
HiveSchemaProvider
JdbcbasedSchemaProvider
ProtoClassBasedSchemaProvider
SchemaPostProcessor
SchemaRegistryProvider
SparkAvroPostProcessor
DropColumnSchemaPostProcessor
BaseSchemaPostProcessorConfig
KafkaOffsetPostProcessor
SanitizationUtils
Also 'hoodie.deltastreamer.multiwriter.source.checkpoint.id' in
HoodieWriteConfig
was:
The configs in the following classes are implemented not using HoodieConfig,
making it impossible to be surfaced on the Configurations page. We need to
refactor the code so that each config property is implemented using
ConfigProperty in a corresponding new HoodieConfig class. Refer to
HoodieArchivalConfig for existing implementation of configs.
InitialCheckPointProvider
HoodieDeltaStreamer
HoodieMultiTableDeltaStreamer
FilebasedSchemaProvider
HiveSchemaProvider
JdbcbasedSchemaProvider
ProtoClassBasedSchemaProvider
SchemaPostProcessor
SchemaRegistryProvider
SparkAvroPostProcessor
DropColumnSchemaPostProcessor
BaseSchemaPostProcessorConfig
> Refactor Deltastreamer and schema providers to use HoodieConfig/ConfigProperty
> ------------------------------------------------------------------------------
>
> Key: HUDI-5740
> URL: https://issues.apache.org/jira/browse/HUDI-5740
> Project: Apache Hudi
> Issue Type: Improvement
> Components: configs, deltastreamer
> Reporter: Jonathan Vexler
> Assignee: Lokesh Jain
> Priority: Blocker
> Fix For: 0.14.0
>
>
> The configs in the following classes are implemented not using HoodieConfig,
> making it impossible to be surfaced on the Configurations page. We need to
> refactor the code so that each config property is implemented using
> ConfigProperty in a corresponding new HoodieConfig class. Refer to
> HoodieArchivalConfig for existing implementation of configs.
>
> InitialCheckPointProvider
> HoodieDeltaStreamer
> HoodieMultiTableDeltaStreamer
> FilebasedSchemaProvider
> HiveSchemaProvider
> JdbcbasedSchemaProvider
> ProtoClassBasedSchemaProvider
> SchemaPostProcessor
> SchemaRegistryProvider
> SparkAvroPostProcessor
> DropColumnSchemaPostProcessor
> BaseSchemaPostProcessorConfig
> KafkaOffsetPostProcessor
> SanitizationUtils
> Also 'hoodie.deltastreamer.multiwriter.source.checkpoint.id' in
> HoodieWriteConfig
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
