[
https://issues.apache.org/jira/browse/HUDI-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo reassigned HUDI-4735:
-------------------------------
Assignee: Raymond Xu
> Spark2 bundles made from master after 2022-07-23 failed to stop
> ---------------------------------------------------------------
>
> Key: HUDI-4735
> URL: https://issues.apache.org/jira/browse/HUDI-4735
> Project: Apache Hudi
> Issue Type: Bug
> Components: hive, spark, spark-sql
> Reporter: Vitali Obukh
> Assignee: Raymond Xu
> Priority: Major
>
> Daily Spark2 Structured streaming application built against the latest change
> on master started failing on 2022-07-23 with the error:
> {quote}
> 22/07/28 10:52:50 ERROR org.apache.hudi.HoodieStreamingSink: Micro batch id=2
> threw following exception:
> org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database
> 'test' not found;
> {quote}
> Actually the database exists (created by the Hudi). Spark3 application with
> the same code works fine.
> Hudi options used for writing:
> {code}
> Map("hoodie.datasource.write.operation" -> "upsert",
> "hoodie.datasource.write.table.type" -> "MERGE_ON_READ",
> "hoodie.datasource.write.recordkey.field" -> s"$Meta.$Key",
> "hoodie.datasource.write.precombine.field" -> s"$Meta.$Version",
> "hoodie.datasource.write.payload.class" ->
> "com.walmart.udps.hudi.integration.HudiTestAvroPayload",
> "hoodie.compaction.payload.class" ->
> "com.walmart.udps.hudi.integration.HudiTestAvroPayload",
> "hoodie.datasource.write.partitionpath.field" -> TopLevelPartition,
> "hoodie.datasource.write.hive_style_partitioning" -> "true",
> "hoodie.datasource.hive_sync.enable" -> "true",
> "hoodie.datasource.hive_sync.mode" -> "hms",
> "hoodie.datasource.hive_sync.database" -> dbName(),
> "hoodie.datasource.hive_sync.partition_fields" -> TopLevelPartition,
> "hoodie.datasource.compaction.async.enable" -> "false"
> )
> {code}
> Cloud platform - Google Dataproc.
> Spark version - 3.1.3.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
