There are 2 approached NiFi processors follow today. E.g. a new TailFile saves it in a local state file (relative to NiFi root, in the ./conf/ directory).
Others leverage a DistributedCache service (it's actually more like a RemoteCache, not a distributed/replicated), which they declare as a requirement in the properties. This also has a benefit of being able to share the state between nodes and, optionally, persist the cache to the disk to be able to re-hydrate it on restart. Andrew On 11/13/15, 10:09 AM, "Sean Busbey" <[email protected]> wrote: >Sam, > >For now I would store this information in HDFS. Alternatively, if you have >it ZooKeeper would also work. > >On Thu, Nov 12, 2015 at 10:45 PM, Sam Kamau <[email protected]> wrote: > >> I am working on a processor that loads data to HDFS for an RDBMS using >> ExecuteSQL processor. The data has a last update time that I would like to >> use to load records that have changed since the last load. >> >> How best can I go about saving the last execution time so I can use it as a >> filter in the query. >> >> Thanks >> > > > >-- >Sean
