The query looks like this: INSERT overwrite TABLE our_schema.our_table
SELECT wh.* FROM our_schema.our_table wh LEFT JOIN our_schema.our_table p ON p.product_id = wh.product_id WHERE p.is_deleted = FALSE OR p.is_deleted IS NULL it is a non transactional table. could it be the reason a combination between DbTransactionManager and the fact that the query performs an overwrite of the table with itself ? We found similar issues: https://issues.apache.org/jira/browse/HIVE-18922 or issues.apache.org/jira/browse/HIVE-13850<https://issues.apache.org/jira/browse/HIVE-13850> Thank you for any hints on how to address this issue. Julien Julien Tane Big Data Engineer [Tel.] +49 721 98993-393 [Fax] +49 721 98993-66 [E-Mail] j...@solute.de<mailto:j...@solute.de> solute GmbH Zeppelinstraße 15 76185 Karlsruhe Germany [Logo Solute] Marken der solute GmbH | brands of solute GmbH [Marken] Geschäftsführer | Managing Director: Dr. Thilo Gans, Bernd Vermaaten Webseite | www.solute.de <http://www.solute.de/> Sitz | Registered Office: Karlsruhe Registergericht | Register Court: Amtsgericht Mannheim Registernummer | Register No.: HRB 110579 USt-ID | VAT ID: DE234663798 Informationen zum Datenschutz | Information about privacy policy https://www.solute.de/ger/datenschutz/grundsaetze-der-datenverarbeitung.php ________________________________ Von: Julien Tane <j...@solute.de> Gesendet: Dienstag, 30. Juni 2020 17:39:45 An: user@hive.apache.org Betreff: Error with hive-staging.staging Hello, we are having trouble with a query which exits infrequently: org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:180)\n\t... 17 more\nCaused by: java.io.IOException: hdfs://ourserver:8020/apps/hive/warehouse/our_schema.db/our_table/.hive-staging_hive_2020-06-30_14-58-28_289_2174922658316991535-9493/_task_tmp.-ext-10000/_tmp.000002_0 to hdfs://ourserver:8020/apps/hive/warehouse/our_schema.db/our_table/.hive-staging_hive_2020-06-30_14-58-28_289_2174922658316991535-9493/_tmp.-ext-10000/000002_0 We are not completely sure how to solve this issue... or even why it happens. Julien Tane Big Data Engineer [Tel.] +49 721 98993-393 [Fax] +49 721 98993-66 [E-Mail] j...@solute.de<mailto:j...@solute.de> solute GmbH Zeppelinstraße 15 76185 Karlsruhe Germany [Logo Solute] Marken der solute GmbH | brands of solute GmbH [Marken] Geschäftsführer | Managing Director: Dr. Thilo Gans, Bernd Vermaaten Webseite | www.solute.de <http://www.solute.de/> Sitz | Registered Office: Karlsruhe Registergericht | Register Court: Amtsgericht Mannheim Registernummer | Register No.: HRB 110579 USt-ID | VAT ID: DE234663798 Informationen zum Datenschutz | Information about privacy policy https://www.solute.de/ger/datenschutz/grundsaetze-der-datenverarbeitung.php