xushiyan commented on code in PR #7722:
URL: https://github.com/apache/hudi/pull/7722#discussion_r1085452892
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala:
##########
@@ -110,7 +110,8 @@ class DefaultSource extends RelationProvider
}
log.info("Obtained hudi table path: " + tablePath)
- val metaClient =
HoodieTableMetaClient.builder().setConf(fs.getConf).setBasePath(tablePath).build()
+ val metaClient =
HoodieTableMetaClient.builder().setProperties(optParams.asJava)
+ .setConf(fs.getConf).setBasePath(tablePath).build()
Review Comment:
@minihippo this change is too broad. this should be fixed in
`org.apache.hudi.DataSourceReadOptions` to target just for metaserver enabling
config. `optParams` is already processed in the lines above into `parameters`,
so we should follow it there.
##########
hudi-platform-service/hudi-metaserver/hudi-metaserver-server/src/main/java/org/apache/hudi/metaserver/store/RelationalDBBasedStorage.java:
##########
@@ -105,8 +106,7 @@ public Long getTableId(String db, String tb) throws
MetaserverStorageException {
@Override
public String createNewTimestamp(long tableId) throws
MetaserverStorageException {
- // todo: support SSS
- SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmss");
+ SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmss:SSS");
Review Comment:
can we reuse
`org.apache.hudi.common.table.timeline.HoodieInstantTimeGenerator#MILLIS_INSTANT_TIME_FORMATTER`
here? so to make formats aligned
##########
packaging/bundle-validation/validate.sh:
##########
@@ -185,6 +186,42 @@ test_kafka_connect_bundle() {
kill $ZOOKEEPER_PID $KAFKA_SERVER_PID $SCHEMA_REG_PID
}
+##
+# Function to test the hudi metaserver bundles.
+#
+# env vars (defined in container):
+# SPARK_HOME: path to the spark directory
+##
+test_hudi_metaserver_bundles () {
+ echo "::warning::validate.sh setting up hudi metaserver bundles validation"
+
+ echo "::warning::validate.sh Start hudi metaserver"
+ java -jar $JARS_DIR/metaserver.jar & local METASEVER=$!
+
+ echo "::warning::validate.sh Start hive server"
+ $DERBY_HOME/bin/startNetworkServer -h 0.0.0.0 &
+ local DERBY_PID=$!
+ $HIVE_HOME/bin/hiveserver2 --hiveconf
hive.aux.jars.path=$JARS_DIR/hadoop-mr.jar &
+ local HIVE_PID=$!
Review Comment:
ok we can fix this later to decouple hive from this testcase's setup
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
