[
https://issues.apache.org/jira/browse/HADOOP-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564799#action_12564799
]
Ankur commented on HADOOP-1824:
-------------------------------
Small correction:
=============
The parsing code works for zip archives UPTO 4 GB (Not 2 GB)
It fails to process zip files of SIZE > 4 GB correctly.
After a little more research I figured that the Zip64 format support (for files
> 4 GB)
is not implemented presently in the minizip code.
So looks like if we need support for files > 4GB, then minizip parsing and
reading code
would definitely require re-work. In other words the minizip code would need to
be "extended"
to support Zip64 format.
This in turn further increases the scope of work.
Any suggestion or recommendations ?
> want InputFormat for zip files
> ------------------------------
>
> Key: HADOOP-1824
> URL: https://issues.apache.org/jira/browse/HADOOP-1824
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.2
> Reporter: Doug Cutting
> Attachments: ZipInputFormat_fixed.patch
>
>
> HDFS is inefficient with large numbers of small files. Thus one might pack
> many small files into large, compressed, archives. But, for efficient
> map-reduce operation, it is desireable to be able to split inputs into
> smaller chunks, with one or more small original file per split. The zip
> format, unlike tar, permits enumeration of files in the archive without
> scanning the entire archive. Thus a zip InputFormat could efficiently permit
> splitting large archives into splits that contain one or more archived files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
