Take a look at the DigestingParser, which wraps another parser. new DigestingParser(p, new CommonsDigester(100, CommonsDigester.DigestAlgorithm.MD5))
If you need modifications, please open a ticket. From: Wshrdryr Corp [mailto:[email protected]] Sent: Wednesday, February 15, 2017 7:49 PM To: [email protected] Subject: Re: CRC ContentHandler Hello Markus, Thanks again for taking the time to reply. I guess I should be more specific: I am extending a Nifi component to add this CRC calculation, specifically here: https://github.com/apache/nifi/blob/0.x/nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/main/java/org/apache/nifi/processors/media/ExtractMediaMetadata.java#L213 Nifi uses Tika to parse files, but doesn't do anything with the content. My plan is to write a TIka content handler which calculates a CRC form the content as it parses the file in order to get a fingerprint of the data segment so the tags can be modified and my system can still prove it's the same underlying data. This is why I asked the original question in the way I did. I see the definition of a Sax ContentHandler, but I was hoping to find a way to get the underlying stream. Or, if I am going about this the wrong way I'd appreciate any advice. Cheers. On Wed, Feb 15, 2017 at 4:03 PM, Markus Jelsma <[email protected]<mailto:[email protected]>> wrote: Hello - streaming hash functions are, in general, from a cryptographic point of view a bad idea, but if you are just interested in checking data integrity it might work for you. You will either have to collect all bits of data and hash it in the end, or feed it to a hashing function that allows for streaming data. The algorithm is up to you. But, on the other hand, are the files you receive that large? Does your process at some point buffer the entire file? If so, hashing is it easy. I don't know if Tika supports ingesting streaming data but in Apache Nutch we buffer the entire file at some point before sending it to Apache Tika, hashing the data is, in this case, not a problem. Markus -----Original message----- > From:Wshrdryr Corp <[email protected]<mailto:[email protected]>> > Sent: Thursday 16th February 2017 0:43 > To: [email protected]<mailto:[email protected]> > Subject: Re: CRC ContentHandler > > Hello Markus, > > Thanks for replying. > > I was hoping not to have to buffer entire media files due to size. Is there a > way to get the content segment as a stream? The internal buffering of a > stream might be more efficient and less prone to spikes. > > Java is not my native tongue. Ive been able to hack through other API > challenges while doing this project. Googling has given me some suspicions > but not a clear answer. > > Cheers. > > On Wed, Feb 15, 2017 at 3:26 PM, Markus Jelsma > <[email protected]<mailto:[email protected]> > <mailto:[email protected]<mailto:[email protected]>>> wrote: > Hello - i dont know if media files even produce SAX events, but if they do > you can catch them in your startElement, charachters, and endElement methods. > I would start collecting element names (qName and/or attribute values) and > stuff in the character method, and append those to a StringBuilder. > > In the endDocument method you have collected every piece of information the > ContentHandler method receives. From thereon you just call > toString().hashCode() or whatever hashing algorithm you like on the contents > accumulated in your StringBuilder. > > Regards, > Markus > > > > -----Original message----- > > From:Wshrdryr Corp <[email protected]<mailto:[email protected]> > > <mailto:[email protected]<mailto:[email protected]>>> > > Sent: Wednesday 15th February 2017 23:22 > > To: [email protected]<mailto:[email protected]> > > <mailto:[email protected]<mailto:[email protected]>> > > Subject: CRC ContentHandler > > > > Hello all, > > > > I need to write a Tika ContentHandler which will return a CRC and/or hash > > of the non-metadata part of media files. > > > > Can anyone point me in the right direction? > > > > Im new to Tika so please forgive me if this is an obvious question. > > > > TIA for any help. >
