yanghua commented on a change in pull request #3170:
URL: https://github.com/apache/hudi/pull/3170#discussion_r662256578
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/factory/HoodieSparkKeyGeneratorFactory.java
##########
@@ -54,7 +59,12 @@ public static KeyGenerator
createKeyGenerator(TypedProperties props) throws IOEx
private static BuiltinKeyGenerator createKeyGeneratorByType(TypedProperties
props) throws IOException {
// Use KeyGeneratorType.SIMPLE as default keyGeneratorType
String keyGeneratorType =
- props.getString(HoodieWriteConfig.KEYGENERATOR_TYPE_PROP.key(),
KeyGeneratorType.SIMPLE.name());
Review comment:
Why do we need to refactor this? For adding a log message?
##########
File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SparkMain.java
##########
@@ -353,14 +354,20 @@ private static int
deduplicatePartitionPath(JavaSparkContext jsc, String duplica
private static int doBootstrap(JavaSparkContext jsc, String tableName,
String tableType, String basePath,
String sourcePath, String recordKeyCols, String partitionFields, String
parallelism, String schemaProviderClass,
- String bootstrapIndexClass, String selectorClass, String
keyGeneratorClass, String fullBootstrapInputProvider,
+ String bootstrapIndexClass, String selectorClass, String keyGenerator,
String fullBootstrapInputProvider,
String payloadClassName, String enableHiveSync, String propsFilePath,
List<String> configs) throws IOException {
TypedProperties properties = propsFilePath == null ?
UtilHelpers.buildProperties(configs)
: UtilHelpers.readConfig(FSUtils.getFs(propsFilePath,
jsc.hadoopConfiguration()), new Path(propsFilePath), configs).getConfig();
properties.setProperty(HoodieBootstrapConfig.BOOTSTRAP_BASE_PATH_PROP.key(),
sourcePath);
- properties.setProperty(HoodieBootstrapConfig.BOOTSTRAP_KEYGEN_CLASS.key(),
keyGeneratorClass);
+
+ if (KeyGeneratorType.getNames().contains(keyGenerator)) {
Review comment:
it should be `contains` or `equals` or `equals ignore case sensitive`？
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/factory/HoodieSparkKeyGeneratorFactory.java
##########
@@ -31,6 +32,8 @@
import org.apache.hudi.keygen.SimpleKeyGenerator;
import org.apache.hudi.keygen.TimestampBasedKeyGenerator;
import org.apache.hudi.keygen.constant.KeyGeneratorType;
+import org.slf4j.Logger;
Review comment:
follow the rule of import orders
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/bootstrap/SparkBootstrapCommitActionExecutor.java
##########
@@ -390,8 +390,15 @@ private BootstrapWriteStatus
handleMetadataBootstrap(String srcPartitionPath, St
TypedProperties properties = new TypedProperties();
properties.putAll(config.getProps());
- KeyGeneratorInterface keyGenerator = (KeyGeneratorInterface)
ReflectionUtils.loadClass(config.getBootstrapKeyGeneratorClass(),
- properties);
+
+ KeyGeneratorInterface keyGenerator;
+ try {
+ keyGenerator =
HoodieSparkKeyGeneratorFactory.createKeyGenerator(properties);
+ } catch (IOException e) {
+ LOG.error("Init keyGenerator failed ", e);
Review comment:
Since we have thrown an exception, we may not need to add a log.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
