exceptionfactory commented on PR #10102: URL: https://github.com/apache/nifi/pull/10102#issuecomment-3103454508
> @exceptionfactory > > > ... at the moment it is not clear why it is necessary to create more than one session. Is it not possible to set the attribute on the last of Result Set FlowFiles before transferring? > > It is possible, but there is 'Output Batch Size' property. When this is set to a value, the created Flow Files need to be transferred, and the session is also need to be committed before we finish reading the records. So at the point, when we know there is no more, we might have already committed the session, so we cannot modify the attributes of the last Flow File. > > My solution here attempts to address this by always holding back the last added Flow File, and excluding it from the mid-session commits. But this is only possible, if you temporarily remove it from the session to be committed. This guarantees that the very last Flow File will be only committed, when reading records have completed. So we can add the EOF property, because we already know there will be no more. > > That's the basic concept of this PR. I'm open to alternative solutions, if you see anything better. Thanks for clarifying the issues @takraj, that makes sense in terms of the `Output Batch Size` property. Under the circumstances, I think the best option is to go with the FlowFile attribute approach (although I would adjust the name and not include the word `executesql`). However, if the `Output Batch Size` property is configured, then I would note in the description that this disables writing the FlowFile attribute. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
