Matt Gilman created NIFI-71:
-------------------------------
Summary: Persistent Prov Repo should compress in blocks
Key: NIFI-71
URL: https://issues.apache.org/jira/browse/NIFI-71
Project: Apache NiFi
Issue Type: Improvement
Reporter: Matt Gilman
Priority: Minor
Currently we write a bunch of events to a file and then compress the file. We
then index the file offset of the uncompressed version of the file.
We should instead compress in chunks of X number of events of X number of
bytes. Then index the offset of the chunk in the compressed version. This way,
we can use FileInputStream.skip to seek to the appropriate offset and then wrap
the stream in GZIPInputStream. This allwos us to avoid reading a lot of
compressed data to get to the desired offset.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
