[
https://issues.apache.org/jira/browse/HUDI-5761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17708012#comment-17708012
]
Jonathan Vexler commented on HUDI-5761:
---------------------------------------
Ok so keygenerator.type is now not such a good example because
https://issues.apache.org/jira/browse/HUDI-5929 . So I will give a different
example. In HoodieBootstrapConfig.java we have config
"hoodie.bootstrap.mode.selector". We have selectors that are already created:
org.apache.hudi.client.bootstrap.selector.BootstrapRegexModeSelector
org.apache.hudi.client.bootstrap.selector.FullRecordBootstrapModeSelector
org.apache.hudi.client.bootstrap.selector.MetadataOnlyBootstrapModeSelector
But you need to read the code to figure that out and it is long to type out. We
want to create a new enum
{code:java}
public enum BootstrapSelectorType {
REGEX,
FULL_RECORD,
METADATA_ONLY,
CUSTOM
}
{code}
And create a new config "hoodie.bootstrap.mode.type" that takes in those config
values
Then, in HoodieWriteConfig.java there is a method
{code:java}
public String getBootstrapModeSelectorClass() {
return getString(HoodieBootstrapConfig.MODE_SELECTOR_CLASS_NAME);
} {code}
We want to replace it with something like
{code:java}
public String getBootstrapModeSelectorClass() {
switch (getString(HoodieBootstrapConfig.MODE_SELECTOR_TYPE_NAME)) {
case REGEX: return
"org.apache.hudi.client.bootstrap.selector.BootstrapRegexModeSelector";
case FULL_RECORD: return
"org.apache.hudi.client.bootstrap.selector.FullRecordBootstrapModeSelector";
case METADATA_ONLY: return "
org.apache.hudi.client.bootstrap.selector.FullRecordBootstrapModeSelector";
case CUSTOM: return
getString(HoodieBootstrapConfig.MODE_SELECTOR_CLASS_NAME);
}
}
{code}
Then repeat this with all other configs that take in class:
hoodie.datasource.write.payload.class
hoodie.datasource.write.record.merger.impls
hoodie.writestatus.class
hoodie.fileid.prefix.provider.class
for example. There are probably more than that.
To get assigned tickets, you need to email
[[email protected]|mailto:[email protected]] and ask to be added as a
contributor.
> Create "Type" configs for current configs that take in classes
> --------------------------------------------------------------
>
> Key: HUDI-5761
> URL: https://issues.apache.org/jira/browse/HUDI-5761
> Project: Apache Hudi
> Issue Type: Improvement
> Components: configs
> Reporter: Jonathan Vexler
> Priority: Major
> Fix For: 0.14.0
>
>
> Take for example `hoodie.datasource.write.keygenerator.type` and its
> counterpart `hoodie.datasource.write.keygenerator.class`. We should make
> configs like keygenerator.type for configs currently like keygenerator.class.
> Take for example `hoodie.datasource.write.record.merger.impls`. If you want
> HoodieAvroRecordMerger you need to set the value to
> org.apache.hudi.common.model.HoodieAvroRecordMerger. However, if you want
> HoodieSparkRecordMerger, the full class path is
> org.apache.hudi.HoodieSparkRecordMerger. Differences like this are an easy
> cause for mistakes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
