[
https://issues.apache.org/jira/browse/TEZ-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14014331#comment-14014331
]
Hitesh Shah commented on TEZ-1106:
----------------------------------
bq. Q1: Where/when to log the message? in createDir() or getDir() or both?
- dir creation and setting perms should only happen in createDir() hence
messages logged only in createDir()
bq. Q2: Will getDir() checks the existence of the path and creates if needed?
OR it will just return the path assuming it was already created.
- getDir() should not create. It should likely do resolvePath but no
creation.
bq. Q3: Important: path permission check.
- this is something which can let it remain as previous behaviour. It needs to
be addressed as we should not be changing permissions of the base dir at all
but only setting restrictive permissions on the appId subdir. We can address
this in a separate jira.
bq. Q4: How to test the whole thing? In addition to unit test,
Given that the code has been broken down into a commonly used set of functions
- this should be unit-tested using a combination of a DFSCluster/localFs and
test users with different perms. A umask related test can be added too for Q3.
It would be good if you can run a few jobs manually with diff users to see what
the end behavior is.
> Tez framework should use a unique subdir when creating new files in staging
> -----------------------------------------------------------------------------
>
> Key: TEZ-1106
> URL: https://issues.apache.org/jira/browse/TEZ-1106
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Mohammad Kamrul Islam
> Assignee: Mohammad Kamrul Islam
> Attachments: TEZ-1106.1.patch, TEZ-1106.2.patch, TEZ-1106.3.patch,
> TEZ-1106.4.patch
>
>
> Currently the files are created in different sub-directories. It is hard to
> manage and cleanup at the end.
> The proposal is to create a new subdir : $STAGE_DIR/<APP_ID>/
> All recovery files will go under : $STAGE_DIR/<APP_ID>/recovery/<attemp_num>/
> All confs will go under: $STAGE_DIR/<APP_ID>/conf/
> All dagplans will go: $STAGE_DIR/<APP_ID>/dag_id/plan/
--
This message was sent by Atlassian JIRA
(v6.2#6252)
