Cody, ExecuteSQL outputs data in Avro format and embeds the schema in the content of the flow file (which Avro supports), leading to a non-zero flowfile size. As of NiFi 1.8.0, you can instead use ExecuteSQLRecord to output the data in any supported format (via a RecordSetWriter controller service), including an AvroRecordSetWriter (if you need the output in Avro) configured to write the schema as an attribute (or not at all) versus embedding the schema in the content. If you are converting the format downstream, you can instead choose a RecordSetWriter for your desired format, and choose how the schema is written (as an attribute or not, e.g.). Please let me know if you have any questions, happy to help!
Regards, Matt On Wed, Mar 25, 2020 at 3:04 PM Cody Ferguson <cody.fergu...@gm.com> wrote: > > Hello, > > We have a process group that queries data from our database and then passes > the data on to a RouteOnAttribute processor which determines whether we > actually have data to process or not. > > The RouteOnAttribute processor determines that by checking the file size. If > the file size is less than so many bytes it will update a run record in the > database and end, if it is more than so many bytes it will continue on > because it believes we retrieved records that need to be processed. > > One thing we noticed recently in preprod is the size of an empty file was > larger than the file size we were checking against. So somehow the size of > the file increased (maybe additional meta data was added?) and because of > that our check was no longer working. > > My question is is there a better way to determine if our ExecuteSQL 1.8.0 > processor retrieved data or not and fork the process based on that? It seems > like checking the file size can lead to issues down the road because the file > itself can grow regardless of having data or not. > > Thanks, > > Cody > > > Nothing in this message is intended to constitute an electronic signature > unless a specific statement to the contrary is included in this message. > > Confidentiality Note: This message is intended only for the person or entity > to which it is addressed. It may contain confidential and/or privileged > material. Any review, transmission, dissemination or other use, or taking of > any action in reliance upon this message by persons or entities other than > the intended recipient is prohibited and may be unlawful. If you received > this message in error, please contact the sender and delete it from your > computer.
