Hi Manee, It depends on the Client API on how to tell what the next response data set should be. That may be an additional query parameter such as last fetch timestamp, or something like HTTP etag header in most APIs. You can pass FlowFiles to InvokeHTTP to tell such parameters.
Also, I recommend using Record processors over splitting dataset to process. Because Record processors work more efficiently and the data set unit will be more meaningful in your case. I think your flow would be something like: InvokeHTTP (Assuming the API result contains value to make next incremental request) -> Do some JSON transformation (QueryRecord, UpdateRecord or JoltTransformRecord) -> PutDatabaseRecord -> Then connect success back to InvokeHTTP to fetch next dataset Hope this helps. Thanks, Koji On Mon, Nov 12, 2018 at 9:06 PM Manee <[email protected]> wrote: > > Hi Team, > > I am new to NiFi .I have a task like .We need to fetch a data from Client > API and stored into Postgresql > My Flow ; > > InvokeHTTP -->>--SplitJson-->>--EvaluateJsonPath > -->>ConvertJSONTosql--->>PutSql > > This is my flow is working fine but i need to make this as a incremental > fetch from API ..whenever client API made changes it should reflect in our > databases .How i can process for incremental fetch in API call ?.Please > guide me to fix this problem . > > > Thanks in Advance, > Manikandan K > > > > > > > ----- > Thanks, > Manee > -- > Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
