HariprasadAllaka1612 opened a new issue #898: Caused by: org.apache.hudi.exception.HoodieUpsertException: Failed to upsert for commit time URL: https://github.com/apache/incubator-hudi/issues/898 I am trying to write data to s3 as Hudi file. Code: inputDF .write.format("org.apache.hudi") .option(HoodieWriteConfig.TABLE_NAME, tablename) .option(DataSourceWriteOptions.RECORDKEY_FIELD_OPT_KEY, "GameId") .option(DataSourceWriteOptions.PARTITIONPATH_FIELD_OPT_KEY,"OperatorShortName") .option(DataSourceWriteOptions.PRECOMBINE_FIELD_OPT_KEY, "HandledTimestamp") .option(DataSourceWriteOptions.OPERATION_OPT_KEY, DataSourceWriteOptions.UPSERT_OPERATION_OPT_VAL) .option(DataSourceWriteOptions.STORAGE_TYPE_OPT_KEY, DataSourceWriteOptions.MOR_STORAGE_TYPE_OPT_VAL) .mode(SaveMode.Append) .save("s3a://" + s3RawDataLakeBucket + "/Games2" ) coming across below exception Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:66) Caused by: org.apache.hudi.exception.HoodieUpsertException: Failed to upsert for commit time 20190917110809 at org.apache.hudi.HoodieWriteClient.upsert(HoodieWriteClient.java:177) at org.apache.hudi.DataSourceUtils.doWriteOperation(DataSourceUtils.java:181) at org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:143) at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:91) at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45) at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70) at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68) at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86) at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131) at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127) at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152) at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127) at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80) at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80) at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:668) at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:668) at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78) at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125) at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73) at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:668) at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:276) at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:270) at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:228) at com.playngoplatform.scala.dao.DataAccessS3.writeDataToRefinedS3(DataAccessS3.scala:30) at com.playngoplatform.scala.controller.GameAndProviderDataTransform.processData(GameAndProviderDataTransform.scala:27) at com.playngoplatform.scala.action.GameAndProviderData$.main(GameAndProviderData.scala:10) at com.playngoplatform.scala.action.GameAndProviderData.main(GameAndProviderData.scala) ... 5 more Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.fs.FSDataOutputStream: method <init>(Ljava/io/OutputStream;)V not found at org.apache.hudi.common.io.storage.SizeAwareFSDataOutputStream.<init>(SizeAwareFSDataOutputStream.java:46) at org.apache.hudi.common.io.storage.HoodieWrapperFileSystem.wrapOutputStream(HoodieWrapperFileSystem.java:160) at org.apache.hudi.common.io.storage.HoodieWrapperFileSystem.create(HoodieWrapperFileSystem.java:168) at org.apache.hudi.common.table.timeline.HoodieActiveTimeline.createFileInPath(HoodieActiveTimeline.java:386) at org.apache.hudi.common.table.timeline.HoodieActiveTimeline.createFileInMetaPath(HoodieActiveTimeline.java:371) at org.apache.hudi.common.table.timeline.HoodieActiveTimeline.saveToInflight(HoodieActiveTimeline.java:359) at org.apache.hudi.HoodieWriteClient.saveWorkloadProfileMetadataToInflight(HoodieWriteClient.java:417) at org.apache.hudi.HoodieWriteClient.upsertRecordsInternal(HoodieWriteClient.java:440) at org.apache.hudi.HoodieWriteClient.upsert(HoodieWriteClient.java:172) ... 33 more
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
