[
https://issues.apache.org/jira/browse/HIVE-29225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18033692#comment-18033692
]
Dmitriy Fingerman commented on HIVE-29225:
------------------------------------------
Hi [~sercan.tekin], please set the affected version(s).
> Premature deletion of scratch directories during output streaming
> -----------------------------------------------------------------
>
> Key: HIVE-29225
> URL: https://issues.apache.org/jira/browse/HIVE-29225
> Project: Hive
> Issue Type: Bug
> Reporter: Sercan Tekin
> Assignee: Sercan Tekin
> Priority: Critical
> Labels: pull-request-available
>
> Once a job or application finishes, the corresponding lock file is released,
> and YARN no longer reports any active jobs or applications. At this point,
> Hive assumes the associated scratch directory is no longer needed and
> proceeds to delete it upon *ClearDanglingScratchDir* service is invoked.
> However, in some cases, Hive may still be streaming output to the client
> after the application is marked as finished. This causes the scratch
> directory to be deleted prematurely, even though it is still required for
> ongoing output.
> As a result, queries can fail with *IOException* errors because the scratch
> directory is removed while Hive is still using it.
> {code:java}
> org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
> java.io.IOException: 2049.323.265264
> /user/mapr/tmp/hive/mapr/ecfdf7a4-91ba-4832-a408-8d459f90ac4b/hive_2025-07-23_12-57-08_793_7535333864129536266-1/-mr-10001/.hive-staging_hive_2025-07-23_12-57-08_793_7535333864129536266-1/-ext-10002/000008_0
> (Input/output error)
> {code}
> ---
> *STEPS TO REPRODUCE:*
> 1. Add the following properties into hive-site.xml and restart Hive services:
> {code:java}
> <property>
> <name>hive.scratchdir.lock</name>
> <value>true</value>
> </property>
> <property>
> <name>tez.session.am.dag.submit.timeout.secs</name>
> <value>10</value>
> </property>
> {code}
> 2. Generate the data using the following commands:
> {code:java}
> for i in {1..1000000}; do echo "" >> /tmp/file1.txt; done
> for i in {1..8}; do cat /tmp/file1.txt >> /tmp/file2.txt; cat /tmp/file2.txt
> >> /tmp/file1.txt; done
> {code}
> 3. Create Hive table and load data into it:
> {code:java}
> CREATE TABLE i (id INT);
> LOAD DATA LOCAL INPATH '/tmp/file2.txt' INTO TABLE i;
> {code}
> 4. Connect to HiveServer2 using Beeline and execute the following queries:
> {code:java}
> SET hive.fetch.task.conversion=none;
> SELECT * FROM i;
> {code}
> 5. Open a new session to the same host. After the query from step 4 starts
> returning the results, wait 30 seconds and execute the command below:
> {code:java}
> hive --service cleardanglingscratchdir -v
> {code}
> 6. ”hive --service cleardanglingscratchdir -v” deletes the scratchdir used by
> the query and the query fails during the next 20-30 seconds with the
> following error:
> {code:java}
> ...
> | NULL |
> org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
> java.io.IOException: 2049.323.265264
> /user/mapr/tmp/hive/mapr/ecfdf7a4-91ba-4832-a408-8d459f90ac4b/hive_2025-07-23_12-57-08_793_7535333864129536266-1/-mr-10001/.hive-staging_hive_2025-07-23_12-57-08_793_7535333864129536266-1/-ext-10002/000008_0
> (Input/output error)
> 0: jdbc:hive2://node1:10000/>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
