That could make it a little better, but I can't remember, can we terminate on a controller service?
The issue here would be on first time enabling the the HBase client service, so before even getting to a processor. On Tue, Feb 18, 2020 at 2:00 PM Pierre Villard <[email protected]> wrote: > > Bryan, > > I didn't follow the whole discussion so I apologize if I'm saying something > stupid here. Now that we have the possibility to terminate threads in a > processor, would that solve the issue? > > Pierre > > Le mar. 18 févr. 2020 à 10:52, Bryan Bende <[email protected]> a écrit : > > > Hi Josh, > > > > The problem isn't so much about the retries within the flow, its more > > about setting up the service for the first time. > > > > A common scenario for users was the following: > > > > - Create a new HBase client service > > - Enter some config that wasn't quite correct, possibly hostnames that > > weren't reachable from nifi as one example > > - Enable service and enter retry loop > > - Attempt to disable service to fix config, but have to wait 5+ mins > > for the retries to finish > > > > Maybe a lazy initialization of the connection on our side would help > > here, although it would just be moving the problem until later (i.e. > > service immediately enables because nothing is happening, then they > > find out about config problems later when a flow file hits an hbase > > processor). > > > > I guess the ideal scenario would be to have different logic for > > initializing the connection vs. using it, so that there wouldn't be > > retries during initialization. > > > > -Bryan > > > > > > > > On Tue, Feb 18, 2020 at 1:21 PM Josh Elser <[email protected]> wrote: > > > > > > Hiya! > > > > > > LarsF brought this up in the apache-hbase slack account and it caught my > > > eye. Sending a note here since the PR is closed where this was being > > > discussed before[1]. > > > > > > I understand Bryan's concerns that misconfiguration of an HBase > > > processor with a high number of retries and back-off can create a > > > situation in which the processing of a single FlowFile will take a very > > > long time to hit the onFailure state. > > > > > > However, as an HBase developer, I can confidently state that > > > hbase.client.retries=1 will create scenarios in which you'll be pushing > > > a FlowFile through a retry loop inside of NiFi for things which should > > > be implicitly retried inside of the HBase client. > > > > > > For example, if a Region is being moved between two RegionServers and an > > > HBase processor is trying to read/write to that Region, the client will > > > see an exception. This is a "retriable" exception in HBase-parlance > > > which means that HBase client code would automatically re-process that > > > request (looking for the new location of that Region first). In most > > > cases, the subsequent RPC would succeed and the caller is non-the-wiser > > > and the whole retry logic took 1's of milliseconds. > > > > > > My first idea was also what Lars' had suggested -- can we come up with a > > > sanity check to validate "correct" configuration for the processor > > > before we throw the waterfall of data at it? I can respect if processors > > > don't have a "good" hook to do such a check. > > > > > > What _would_ be the ideal semantics from NiFi's? perspective? We have > > > the ability to implicitly retry operations and also control the retry > > > backoff values. Is there something more we could do from the HBase side, > > > given what y'all have seen from the battlefield? > > > > > > Thanks! > > > > > > - Josh > > > > > > [1] https://github.com/apache/nifi/pull/3425 > >
