Yann Byron created HUDI-2455:
--------------------------------
Summary: Fail to build Spark caused by spark_avro dependency
Key: HUDI-2455
URL: https://issues.apache.org/jira/browse/HUDI-2455
Project: Apache Hudi
Issue Type: Bug
Components: Spark Integration
Reporter: Yann Byron
Fix For: 0.10.0
Fail to build hudi using this command:
{code:java}
mvn clean package -DskipTests
{code}
The key error infomation:
{code:java}
[ERROR]
/hudi/hudi-integ-test/src/main/scala/org/apache/hudi/integ/testsuite/utils/SparkSqlUtils.scala:32:
error: object avro is not a member of package org.apache.spark.sql [ERROR]
import org.apache.spark.sql.avro.SchemaConverters
[ERROR] ^
[ERROR]
/hudi/hudi-integ-test/src/main/scala/org/apache/hudi/integ/testsuite/utils/SparkSqlUtils.scala:142:
error: not found: value SchemaConverters [ERROR] val structType =
SchemaConverters.toSqlType(schema).dataType.asInstanceOf[StructType] [ERROR] ^
[ERROR] two errors found
{code}
Adding the spark_avro dependency to hudi-integ-test module works.
{code:java}
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-avro_${scala.binary.version}</artifactId>
</dependency>
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
