[
https://issues.apache.org/jira/browse/HUDI-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kate Huber reassigned HUDI-8559:
--------------------------------
Assignee: Lin Liu (was: Y Ethan Guo)
> Spark SQL commands fail on AWS EMR
> ----------------------------------
>
> Key: HUDI-8559
> URL: https://issues.apache.org/jira/browse/HUDI-8559
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Y Ethan Guo
> Assignee: Lin Liu
> Priority: Blocker
> Fix For: 1.0.0
>
>
> With EMR Spark the below error is thrown for any SQL commands with master.
> {code:java}
> java.lang.IllegalStateException: Loop detected; file
> file:/etc/hudi/conf/hudi-defaults.conf already referenced
> at
> org.apache.hudi.common.config.DFSPropertiesConfiguration.addPropsFromFile(DFSPropertiesConfiguration.java:147)
> at
> org.apache.hudi.common.config.DFSPropertiesConfiguration.loadGlobalProps(DFSPropertiesConfiguration.java:127)
> at
> org.apache.hudi.common.config.DFSPropertiesConfiguration.<clinit>(DFSPropertiesConfiguration.java:71)
> at
> org.apache.spark.sql.hudi.ProvidesHoodieConfig$.isSchemaEvolutionEnabled(ProvidesHoodieConfig.scala:594)
> at
> org.apache.spark.sql.hudi.Spark34ResolveHudiAlterTableCommand.apply(Spark34ResolveHudiAlterTableCommand.scala:35)
> at
> org.apache.spark.sql.hudi.Spark34ResolveHudiAlterTableCommand.apply(Spark34ResolveHudiAlterTableCommand.scala:32)
> at
> org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1(RuleExecutor.scala:239)
> at
> scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
> at
> scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
> at scala.collection.immutable.List.foldLeft(List.scala:91)
> at
> org.apache.spark.sql.catalyst.rules.RuleExecutor.executeBatch$1(RuleExecutor.scala:236)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
