tprelle opened a new issue #2541: URL: https://github.com/apache/iceberg/issues/2541
insert into from hive tez it's not working for simple insert query because we do not have forcommit file because it's only created for reduce task https://github.com/apache/iceberg/blob/master/mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java#L82 Just an insert into export_table select * from import_table; reproduce the bug. ` ---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ---------------------------------------------------------------------------------------------- Map 1 .......... container SUCCEEDED 1 1 0 0 0 0 ---------------------------------------------------------------------------------------------- VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 28.92 s ---------------------------------------------------------------------------------------------- ERROR : Commit failed for output: outputName:out_Map 1 of vertex/vertexGroup:Map 1 isVertexGroupOutput:false, org.apache.iceberg.exceptions.NotFoundException: Failed to open input stream for file: hdfs://.../table/temp/hive_20210428203410_e66e7c0d-64d8-4266-9d96-8512a097ade2-job_16191695629360_106820/task-1.forCommit at org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:177) at org.apache.iceberg.mr.hive.HiveIcebergOutputCommitter.readFileForCommit(HiveIcebergOutputCommitter.java:439) at org.apache.iceberg.mr.hive.HiveIcebergOutputCommitter.lambda$dataFiles$9(HiveIcebergOutputCommitter.java:394) at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:404) at org.apache.iceberg.util.Tasks$Builder.access$300(Tasks.java:70) at org.apache.iceberg.util.Tasks$Builder$1.run(Tasks.java:310) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
