Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
jonvex commented on code in PR #10998:
URL: https://github.com/apache/hudi/pull/10998#discussion_r1562940754
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##
@@ -964,6 +964,11 @@ object DataSourceOptionsHelper {
def translateConfigurations(optParams: Map[String, String]): Map[String,
String] = {
val translatedOpt = scala.collection.mutable.Map[String, String]() ++=
optParams
+if (!translatedOpt.contains(HoodieTableConfig.NAME.key()) &&
Review Comment:
I don't think this makes things worse than they already are with spark
configs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
yihua commented on code in PR #10998:
URL: https://github.com/apache/hudi/pull/10998#discussion_r1562929292
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##
@@ -964,6 +964,11 @@ object DataSourceOptionsHelper {
def translateConfigurations(optParams: Map[String, String]): Map[String,
String] = {
val translatedOpt = scala.collection.mutable.Map[String, String]() ++=
optParams
+if (!translatedOpt.contains(HoodieTableConfig.NAME.key()) &&
Review Comment:
Understood. Should we deprioritize this for now and look for a better way?
I think it's better to have such logic in a common place instead of being
scattered in a separate method, which is going to be harder to maintain.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
hudi-bot commented on PR #10998: URL: https://github.com/apache/hudi/pull/10998#issuecomment-2050666813 ## CI report: * e7e51394cc39b914503b7e1e3608cdb3ff690a30 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23197) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
jonvex commented on code in PR #10998:
URL: https://github.com/apache/hudi/pull/10998#discussion_r1561821789
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##
@@ -964,6 +964,11 @@ object DataSourceOptionsHelper {
def translateConfigurations(optParams: Map[String, String]): Map[String,
String] = {
val translatedOpt = scala.collection.mutable.Map[String, String]() ++=
optParams
+if (!translatedOpt.contains(HoodieTableConfig.NAME.key()) &&
Review Comment:
I tried adding, that, but it removes the configs with the new ones, so it
was making other things fail if I used that instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
yihua commented on code in PR #10998:
URL: https://github.com/apache/hudi/pull/10998#discussion_r1561742623
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##
@@ -964,6 +964,11 @@ object DataSourceOptionsHelper {
def translateConfigurations(optParams: Map[String, String]): Map[String,
String] = {
val translatedOpt = scala.collection.mutable.Map[String, String]() ++=
optParams
+if (!translatedOpt.contains(HoodieTableConfig.NAME.key()) &&
Review Comment:
`HoodieWriterUtils` has `sparkDatasourceConfigsToTableConfigsMap` which
contains `hoodie.datasource.write.table.name` to `hoodie.table.name` mapping.
Is it not working?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
yihua commented on code in PR #10998:
URL: https://github.com/apache/hudi/pull/10998#discussion_r1561716214
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##
@@ -964,6 +964,11 @@ object DataSourceOptionsHelper {
def translateConfigurations(optParams: Map[String, String]): Map[String,
String] = {
val translatedOpt = scala.collection.mutable.Map[String, String]() ++=
optParams
+if (!translatedOpt.contains(HoodieTableConfig.NAME.key()) &&
Review Comment:
Should this logic be incorporated into L972?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
hudi-bot commented on PR #10998: URL: https://github.com/apache/hudi/pull/10998#issuecomment-2050520984 ## CI report: * ea501ae87f61ff965f558360bb703bfad595c2a0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23196) * e7e51394cc39b914503b7e1e3608cdb3ff690a30 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23197) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
hudi-bot commented on PR #10998: URL: https://github.com/apache/hudi/pull/10998#issuecomment-2050508252 ## CI report: * ea501ae87f61ff965f558360bb703bfad595c2a0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23196) * e7e51394cc39b914503b7e1e3608cdb3ff690a30 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
nsivabalan commented on code in PR #10998:
URL: https://github.com/apache/hudi/pull/10998#discussion_r1561644552
##
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:
##
@@ -964,6 +964,11 @@ object DataSourceOptionsHelper {
def translateConfigurations(optParams: Map[String, String]): Map[String,
String] = {
val translatedOpt = scala.collection.mutable.Map[String, String]() ++=
optParams
+if (!translatedOpt.contains(HoodieTableConfig.HOODIE_TABLE_NAME_KEY) &&
+ translatedOpt.contains(HoodieTableConfig.HOODIE_WRITE_TABLE_NAME_KEY)) {
Review Comment:
can we replace DataSourceWriteOptions.TABLE_NAME.key() instead of
HoodieTableConfig.HOODIE_WRITE_TABLE_NAME_KEY
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
hudi-bot commented on PR #10998: URL: https://github.com/apache/hudi/pull/10998#issuecomment-2050413788 ## CI report: * ea501ae87f61ff965f558360bb703bfad595c2a0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23196) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7604] Make table name config work properly [hudi]
hudi-bot commented on PR #10998: URL: https://github.com/apache/hudi/pull/10998#issuecomment-2050403305 ## CI report: * ea501ae87f61ff965f558360bb703bfad595c2a0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
