Peter Vary created HIVE-23883:
---------------------------------

             Summary: Streaming does not flush the side file
                 Key: HIVE-23883
                 URL: https://issues.apache.org/jira/browse/HIVE-23883
             Project: Hive
          Issue Type: Bug
          Components: Streaming, Transactions
            Reporter: Peter Vary


When a streaming write commits a mid-batch write with 
{{connection.commitTransaction()}} then it tries to flush the sideFile with 
{{OrcInputFormat.SHIMS.hflush(flushLengths)}}. This uses FSOutputSummer.flush, 
which does not flush the buffer data to the disk so the actual data is not 
written.

Had to remove the check from the end of the streaming tests in 
{{TestCrudCompactorOnTez.java}}
{code:java}
      CompactorTestUtilities.checkAcidVersion(fs.listFiles(new 
Path(table.getSd().getLocation()), true), fs,
          conf.getBoolVar(HiveConf.ConfVars.HIVE_WRITE_ACID_VERSION_FILE),
          new String[] { AcidUtils.DELTA_PREFIX });
{code}
These checks verifies the {{_flush_length}} files, and they would fail 
otherwise.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to