[
https://issues.apache.org/jira/browse/IMPALA-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yida Wu updated IMPALA-12720:
-----------------------------
Description:
This is a following jira for IMPALA-12698. While running the test
TestScratchDir.test_scratch_dirs_batch_reading, it was observed that certain
hdfs files in /proc/xxxx/fd appeared as deleted. This suggests a delay in the
immediate release of file handles. However, in local tests, it was noticed that
the files are eventually deleted from the fd path after a few minutes. Further
investigation may be worthwhile to understand the underlying cause of this
symptom.
{code:java}
lr-x------ 1 impala impala 64 Jan 9 23:40 750 ->
'/impala/Impala/testdata/cluster/cdh7/node-2/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920999_180368.meta
(deleted)'
lr-x------ 1 impala impala 64 Jan 9 23:40 751 ->
'/impala/Impala/testdata/cluster/cdh7/node-1/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920938
(deleted)'
{code}
was:
This is following jira for IMPALA-12698. While running the test
TestScratchDir.test_scratch_dirs_batch_reading, it was observed that certain
hdfs files in /proc/xxxx/fd appeared as deleted. This suggests a delay in the
immediate release of file handles. However, in local tests, it was noticed that
the files are eventually deleted from the fd path after a few minutes. Further
investigation may be worthwhile to understand the underlying cause of this
symptom.
{code:java}
lr-x------ 1 impala impala 64 Jan 9 23:40 750 ->
'/impala/Impala/testdata/cluster/cdh7/node-2/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920999_180368.meta
(deleted)'
lr-x------ 1 impala impala 64 Jan 9 23:40 751 ->
'/impala/Impala/testdata/cluster/cdh7/node-1/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920938
(deleted)'
{code}
> Investigate reference deleted hdfs files in spilling tests
> ----------------------------------------------------------
>
> Key: IMPALA-12720
> URL: https://issues.apache.org/jira/browse/IMPALA-12720
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.3.0
> Reporter: Yida Wu
> Assignee: Yida Wu
> Priority: Minor
>
> This is a following jira for IMPALA-12698. While running the test
> TestScratchDir.test_scratch_dirs_batch_reading, it was observed that certain
> hdfs files in /proc/xxxx/fd appeared as deleted. This suggests a delay in the
> immediate release of file handles. However, in local tests, it was noticed
> that the files are eventually deleted from the fd path after a few minutes.
> Further investigation may be worthwhile to understand the underlying cause of
> this symptom.
> {code:java}
> lr-x------ 1 impala impala 64 Jan 9 23:40 750 ->
> '/impala/Impala/testdata/cluster/cdh7/node-2/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920999_180368.meta
> (deleted)'
> lr-x------ 1 impala impala 64 Jan 9 23:40 751 ->
> '/impala/Impala/testdata/cluster/cdh7/node-1/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920938
> (deleted)'
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
