[
https://issues.apache.org/jira/browse/SQOOP-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111759#comment-14111759
]
Richard commented on SQOOP-1392:
--------------------------------
I agree to use {{java.io.tmpdir}} as temp work output path. Shall we add some
suffix? For the *FileOutputFormat.getWorkOutputPath(context)*,
{{APPLICATION_ATTEMPT_ID}} and {{TaskAttemptID}} are used to generate the final
temp work output path, which could avoid conflict if multi-tasks are running in
the same time.
{code}
Path path = new
Path(context.getConfiguration().get(FileOutputFormat.OUTDIR));
String[] subPathes = {
FileOutputCommitter.PENDING_DIR_NAME,
String.valueOf(context.getConfiguration().getInt(MRJobConfig.APPLICATION_ATTEMPT_ID,
0)),
FileOutputCommitter.PENDING_DIR_NAME,
String.valueOf(context.getTaskAttemptID())
};
for (String subPath : subPathes){
path = new Path(path, subPath);
}
return path;
{code}
I prefer to use
{quote}
{{java.io.tmpdir}}/SQOOP-PARQUET/{{APPLICATION_ATTEMPT_ID}}/{{TaskAttemptID}}
{quote}
as final temp work output path.
> Create the temporary directory inside task working dir rather then in tmp
> -------------------------------------------------------------------------
>
> Key: SQOOP-1392
> URL: https://issues.apache.org/jira/browse/SQOOP-1392
> Project: Sqoop
> Issue Type: Improvement
> Components: tools
> Reporter: Qian Xu
> Priority: Minor
> Attachments: patch.diff
>
>
> Several mappers will create a temporary directory in /tmp ranther then inside
> the task working directory. We should change the pattern.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
