Tim and Matt, Did the changes that facilitate these settings ever make it into somewhere public to be pulled into NiFi? I’m needing to set these BasicDataSource settings for my own needs [1] and would love to reuse someone else’s code instead of doing it up myself if possible.
[1]: https://stackoverflow.com/questions/53110163/how-can-i-configure-nifis-dbcpconnectionpool-not-to-keep-idle-connections-open On 2018/02/23 22:45:13, Matt Burgess <[email protected]<mailto:[email protected]>> wrote: > Tim,> > > We can certainly expose some DBCP properties as processor properties,> > we'd have to enumerate them explicitly since the user-defined ones are> > used for the connection. Please feel free to write up an Improvement> > Jira [1] to cover whichever properties you'd like to see added to> > DBCPConnectionPool.> > > Also, we've had some issues with Kerberos ticket expiration that were> > addressed in NiFi 1.5.0 [2], not sure if that would help you out or> > not.> > > Regards,> > Matt> > > [1] https://issues.apache.org/jira/projects/NIFI> > [2] https://issues.apache.org/jira/browse/NIFI-3472> > > On Fri, Feb 23, 2018 at 5:38 PM, Tim Zimmerman > <[email protected]<mailto:[email protected]>> wrote:> > > Matt,> > > Thank you for the quick response. I do not think these are connection> > > properties but properties of the connection pool. The properties I am> > > looking to set are the BasicDataSource Configuration parameters such as> > > minIdle and minEvictableIdleTimeMillis.> > >> > > I do use the validation query but the problem is the connection to the> > > database is still valid while, I may get this wrong but in layman's terms,> > > the kerberos authentication for the user who has connected is "stale". The> > > result is, NiFi is able to query the database, create temp tables etc but> > > when it attempts to perform the bulk insert of a file on a third (NAS)> > > server it fails. I suppose my validation query could be an attempt at some> > > bulk insert but that seems a little inefficient.> > >> > > Right now I have a test set up with four connection pools and four flows.> > > Two ran for some time, maybe a week but are now continually failing. The> > > other two continue to work. The difference is that the two that continue to> > > work have either had the pool reduced to zero at some point due to idle> > > connections being closed or the connections were never allowed to idle i.e.> > > executing the bulk insert every 10 seconds.> > >> > > The two that are failing were allowed to maintain a minimum of one> > > connection and ran less frequently. One runs every three hours the other> > > once a week. While the bulk insert for these fails, querying the database> > > and creating/dropping tables continues to succeed.> > >> > >> > >> > > --> > > Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/> >
