[
https://issues.apache.org/jira/browse/HADOOP-13951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905846#comment-15905846
]
Andrew Wang commented on HADOOP-13951:
--------------------------------------
Thanks Sean. I think this is probably HDFS-only, since it's some dfs metadata
directory.
I also see this at the end of some builds:
{noformat}
# See HADOOP-13951
find "${WORKSPACE}" -name target | xargs chmod -R u+w
[PreCommit-HDFS-Build] $ /bin/bash /tmp/hudson1505512985405565527.sh
find:
`/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/hadoop-hdfs-project/hadoop-hdfs/target/test/data/1/dfs/data/data1/current':
Permission denied
chmod: cannot read directory
‘/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/hadoop-hdfs-project/hadoop-hdfs/target/test/data/1/dfs/data/data1/current’:
Permission denied
{noformat}
I was recursively chmod'ing the entire directory in my clean script, I'm going
to modify the HDFS precommit to do that.
> Precommit builds do not adequately protect against test malformed fs
> permissions.
> ---------------------------------------------------------------------------------
>
> Key: HADOOP-13951
> URL: https://issues.apache.org/jira/browse/HADOOP-13951
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Sean Busbey
> Priority: Critical
>
> Right now this is expressed as failed Precommit-YARN-build jobs when they run
> on H5 / H6 (see INFRA-13148), but the problem exists for all of the hadoop
> related precommit jobs.
> The issue is that we have some tests in Common (and maybe HDFS) that
> purposefully set permissions within the {{target/}} directory to simulate a
> failure to interact with underlying fs data. The test sets some
> subdirectories to have permissions such that we can no longer delete their
> contents.
> Right now our precommit jobs include a step post-yetus-test-patch that
> traverses the target directories and ensures that all subdirectories are
> modifiable:
> {code}
> find ${WORKSPACE} -name target | xargs chmod -R u+w
> {code}
> Unfortunately, if we don't get to that line (say due to an aborted build, or
> if the call to yetus test-patch exceeds the job timeout), then we are left in
> a state where there are still subdirectories that can't be modified
> (including deleted).
> Our builds also currently attempt to run a {{git clean}} at the very start of
> the build after the repo is updated. If we have one of the aforementioned
> timeouts that leaves a can't-be-deleted test directory, then all future
> builds on that machine will fail attempting to run the {{git clean}} command.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
