Ian, You can effectively turn off the NiFi provenance reporter by switching to an in-memory repository. So for example edit ./conf/nif.properties
Change this line nifi.provenance.repository.implementation=org.apache.nifi.provenance.PersistentProvenanceRepository To nifi.provenance.repository.implementation=org.apache.nifi.provenance.VolatileProvenanceRepository You can also change this line to some smaller or other number to match your needs. nifi.provenance.repository.buffer.size=100000 Thanks Joe On Mon, Nov 30, 2015 at 2:21 PM, ianwork <[email protected]> wrote: > My instance is running on a VM with 8cpu's. The repositories are all > stored on the same virtual disks. > > a snapshot of io when the throttling kicks in > Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn > vda 133.40 0.00 56982.40 0 284912 > > On Mon, Nov 30, 2015 at 12:21 PM, Aldrin Piri [via Apache NiFi Developer > List] <[email protected]> wrote: > >> Ian, >> >> Could you provide information as to how your repositories are configured? >> Specifically, what the layout is to physical volume(s)/disks? >> >> On Monday, November 30, 2015, ianwork <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=5524&i=0>> wrote: >> >> > I am also experiencing a slowdown with the provenance reporter and would >> > like >> > information on tuning. My application is performing log processing of >> 20k >> > flowfiles a second. As a workaround batch processing was implemented but >> I >> > would also like to see if it can be done without batching. I would >> prefer >> > to turn off the reporter, but since that is not possible, do you suggest >> > that I move it to memory? And if so what setting would that be? >> > >> > >> > >> > -- >> > View this message in context: >> > >> http://apache-nifi-developer-list.39713.n7.nabble.com/Provenance-tuning-tp1427p5522.html >> > Sent from the Apache NiFi Developer List mailing list archive at >> > Nabble.com. >> > >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the discussion >> below: >> >> http://apache-nifi-developer-list.39713.n7.nabble.com/Provenance-tuning-tp1427p5524.html >> To unsubscribe from Provenance tuning, click here >> <http://apache-nifi-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1427&code=aWFud29yazAwMUBnbWFpbC5jb218MTQyN3wxMjM1MTk1NzEz> >> . >> NAML >> <http://apache-nifi-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> > > > > > -- > View this message in context: > http://apache-nifi-developer-list.39713.n7.nabble.com/Provenance-tuning-tp1427p5525.html > Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
