[
https://issues.apache.org/jira/browse/HADOOP-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601591#comment-13601591
]
Chris Nauroth commented on HADOOP-9397:
---------------------------------------
Thanks, Jason.
{quote}
Curious though, why does hadoop-dist invoke tar and gzip separately, while
other projects pipe the output of tar to gzip (e.g.: hadoop-mapreduce-project,
hadoop-yarn-project)?
{quote}
The changes to the distribution scripts were originally submitted in
HADOOP-9271. I left detailed comments explaining all of the changes there.
I'm pasting the most relevant part here:
{code}
- run tar czf hadoop-${project.version}.tar.gz
hadoop-${project.version}
+ run tar cf hadoop-${project.version}.tar
hadoop-${project.version}
+ run gzip hadoop-${project.version}.tar
{code}
The 'z' flag for compression causes tar to fork a separate process for gzip.
GnuWin32 tar has a limitation in that fork was never implemented, so this would
fail on Windows with "Cannot fork: Function not implemented". Splitting this
into separate tar and gzip commands works cross-platform.
Another option here would have been to control the pipeline explicitly using a
shell pipeline (tar | gzip), but the "run" helper function used here isn't
compatible with passing a command that has a pipe.
{quote}
Do we really need the intermediate .tar file kept around?
{quote}
No, and gzip actually replaces the original file, so we don't have this
problem. I just ran it again and confirmed that the end result was a .tar.gz
file (and no separate .tar file).
> Incremental dist tar build fails
> --------------------------------
>
> Key: HADOOP-9397
> URL: https://issues.apache.org/jira/browse/HADOOP-9397
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 3.0.0
> Reporter: Jason Lowe
> Assignee: Chris Nauroth
> Attachments: HADOOP-9397.1.patch
>
>
> Building a dist tar build when the dist tarball already exists from a
> previous build fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
