[
https://issues.apache.org/jira/browse/TIKA-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18054982#comment-18054982
]
Iachimoe commented on TIKA-4630:
--------------------------------
I believe that in fact the current setup contains all of the necessary
information. That being said, my feeling was that the way it is presented could
be more internally consistent, although this is probably just a result of my
own overly simple mental model of what the different fields are supposed to
represent. So overall, it's probably best to leave things as they are, but I
will make the case for why I think my changes make sense, in case the approach
turns out to be valuable:
Just to reiterate the context regarding gzip files discussed above. GZIP files
contain optional metadata with the name of the file being compressed (a fact
that I was unaware of until this JIRA). The two principal tools used to process
gzip files on the command line, {{gzip}} itself and {{tar}} (I believe that the
behaviour is consistent across the GNU and BSD versions) have a peculiar
approach to this metadata. In the case of {{{}gzip{}}}, when creating an
archive it will, by default, set the metadata field to be the name of the file
being compressed. However, when uncompressing, unless the -N flag is used, the
resulting output file will be based on the name on the filesystem of the gzip
archive, and the name in the metadata will be ignored. Likewise {{tar}} seems
to not set the metadata when creating a tarball, and ignores it when
uncompressing one. Therefore the de-facto semantics of gzip files are that most
of the time the name of the file being ungzipped is derived from the name of
the gzip file itself.
One of the changes in my PR was to make RESOURCE_NAME consistent with the
INTERNAL_PATH. IMHO it would be confusing if a file on the filesystem is called
{{{}alice{}}}.gz but the metadata contains bob, that INTERNAL_PATH would be
{{bob}} while RESOURCE_NAME would be {{alice}} . Though as I pointed out in my
PR, this would be a change in existing behaviour, so even if logically correct
may not be ideal for a minor release.
The other part of the change relates to an idea that every file within an
archive should have an INTERNAL_PATH. This would make it easier to use the
metadata to construct consistent paths that can be accessed using other tools.
E.g. (let's assume for now that the tarball does NOT contain the file name
metadata in the gzip file), if a file hello.txt is in an archive called
archive.tgz, it would be useful to be able to construct
"archive.tgz/archive.tar/hello.txt" from the metadata. That said, this can
probably already be accomplished using EMBEDDED_ID/EMBEDDED_ID_PATH , but it
still feels like better usability if we can just depend on every item within an
archive having an INTERNAL_PATH . Given my understanding of the real-world
semantics of tar/gz files as illustrated above, I think it makes perfect sense
to use the name of the file on the filesystem (i.e. RESOURCE_NAME).
In the grand scheme of things, maybe this is a largely irrelevant detail. I'm
probably somewhat biased because I was easily able to make my client code work
when working against a build of tika using my change, whereas I'll have to do
some further rework to make it work with what's currently in tika (admittedly
probably nothing inordinately difficult).
> embeddedRelationshipId is missing from tar files that are children of gzip
> files (i.e. tarballs)
> ------------------------------------------------------------------------------------------------
>
> Key: TIKA-4630
> URL: https://issues.apache.org/jira/browse/TIKA-4630
> Project: Tika
> Issue Type: Improvement
> Components: core
> Affects Versions: 3.2.3
> Reporter: Iachimoe
> Assignee: Tim Allison
> Priority: Major
> Attachments: tika-embedded-reln-jira.PATCH
>
>
> For some context, my usecase requires me to be able to precisely locate
> individual files within nested archives, using only metadata from tika.
> I have made this work for zip files using a combination of
> EMBEDDED_RELATIONSHIP_ID and EMBEDDED_RESOURCE_PATH. EMBEDDED_RESOURCE_PATH
> on its own does not work because it omits information about which directory a
> file is stored in within an archive. When trying to make it work for tgz
> files, the approach breaks because, while the parser quite reasonably
> considers "archive.tar" to be a child of "archive.tar.gz", the corresponding
> entry metadata lacks an EMBEDDED_RELATIONSHIP_ID. This makes it difficult to
> construct a path of the form
> "archive.tar.gz/archive.tar/my_folder/my_file.txt".
> I attach a rough solution (patch is against branch_3x), with unit test, that
> appears to work for my use case. The approach is for the
> RecursiveParserWrapper to set EMBEDDED_RELATIONSHIP_ID once parsing has
> completed if it has not been set during the parsing process. It sets it to
> the value of RESOURCE_NAME_KEY , which is probably not ideal, but in practice
> it seems to contain the filename and parent folders within the archive.
> Although the aim was to make this work for tar files, the unit test
> demonstrates that it has a similar effect for chidren of ODT files (this was
> an accident as the test archive that was already in the codebase happened to
> have ODT files).
> I could not find descriptions of exactly what EMBEDDED_RELATIONSHIP_ID and
> EMBEDDED_RESOURCE_PATH are meant to represent, so I'm not sure whether this
> approach is heading in the right direction, though it seems to work for my
> usecase. A future enhancement could be for the recursive parser to include a
> new field with the complete path to each file (again, imagining nested
> archives it might look something like
> "archive.tar.gz/archive.tar/folder1/nested.zip/folder2/file.txt").
> Looking forward to everybody's thoughts, and happy to make refinements to the
> proof of concept attached as needed.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
