Hi, I have a Nifi design question. In order to process extremely large files (any size), we intend to create a processor that reads the file in "chunks" and sends as a multi-part FlowFile series, which will avoid using up all available content repository and/or JVM space.
One way would be to create our own state file that contains the latest job information (per thread/job), but that seems very clunky. The question is, with long-running processes like this that need to be restartable (without starting from the beginning on big files), are there any standard Nifi design patterns we should consider? Thanks in advance. Rick
