Re: Error handling in @OnScheduled

2018-08-23 Thread James Srinivasan
Ah, hadn't spotted that. It's close, but the Throwable I get is a java.lang.AssertionError (Could not invoke methods annotated with @OnScheduled annotation due to: java.lang.reflect.InvocationTargetException) and there doesn't seem to be any way to get the actual underlying exception my code threw

Re: Please review patch for NIFI-830 jira

2018-08-23 Thread Andy LoPresto
Hi Taha, As you can see from the discussion on that issue, this will break backward compatibility. It can only be introduced at a major version change (i.e. 1.x -> 2.0). In addition, attaching a patch to the Jira or submitting a PR to the GitHub repository [1] are both acceptable processes

Please review patch for NIFI-830 jira

2018-08-23 Thread Taha Naqvi
I was not able to find Nifi on https://reviews.apache.org/r/new/ as a Repository. So writing here. JIRA link -https://issues.apache.org/jira/browse/NIFI-830 Thanks, Taha Naqvi -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Error handling in @OnScheduled

2018-08-23 Thread Mike Thomsen
James try it with a throwable like in my example On Thu, Aug 23, 2018 at 10:51 AM Mark Payne wrote: > James, > > If you are expecting the method to throw an Exception and want to verify > that, you should > just call the method directly from your unit test and catch the Exception > there. The

Re: Re: Pheonix client jar required for HBase conn

2018-08-23 Thread Bryan Bende
Adam, One thought... the phoenix client jar is a shaded jar which I think bundles hbase-common so maybe the end result was two versions of ClassSize being on the classpath and it not being clear which one was being used. Also, a NoClassDefFoundError can be thrown if some static code in the class

Re: Re: Pheonix client jar required for HBase conn

2018-08-23 Thread Martini, Adam
Bryan, Yes, an HBase client upgrade makes sense for the Java 10 upgrade path. However, the NoClassDefFoundError is more mysterious and does concern me. Thanks, Adam On 8/23/18, 12:12 PM, "Bryan Bende" wrote: Adam, Yes for now Java 8 is what is fully supported.

Re: Re: Pheonix client jar required for HBase conn

2018-08-23 Thread Martini, Adam
Bryan, We are running HBase 1.2. I am confused as well about the missing ClassSize class. Does seem completely unrelated to Pheonix and I do not understand why adding the client would help resolve the class. Would you advise reverting to Java 8 until Java 10 is fully supported? Thanks,

Re: Re: Pheonix client jar required for HBase conn

2018-08-23 Thread Bryan Bende
Adam, Yes for now Java 8 is what is fully supported. Hopefully the remaining issues can be resolved in the not too distant future to become compatible with Java 9+. I know in the HBase case it requires changing to a newer version of the client because the 1.1.2 client has the code that caused

Re: Reading all flowfiles queued for a processor (>20000 flowfiles)

2018-08-23 Thread Mark Payne
Sam, Yes, that's right. The session provides a transaction. So each "bin" has its own session. This way, once a Bin is ready to be combined, you can do so in a single transaction/session and then the session is complete. No need to try to manage which bins contain which sessions or vice versa.

Re: Pheonix client jar required for HBase conn

2018-08-23 Thread Bryan Bende
Adam, The ClassSize class comes from hbase-common [1] so I'm not sure how that would related to the Phoenix client JAR. What version of HBase was this against? The only case I know of that needs the Phoenix client jar is when Phoenix has been installed which then modifies the HBase config files

Re: Error handling in @OnScheduled

2018-08-23 Thread Mark Payne
James, If you are expecting the method to throw an Exception and want to verify that, you should just call the method directly from your unit test and catch the Exception there. The TestRunner expects to run the full lifecycle of the Processor. Thanks -Mark > On Aug 23, 2018, at 10:49 AM,

Re: Error handling in @OnScheduled

2018-08-23 Thread James Srinivasan
I tried that, but the problem is the exception is caught and the test fails due to this: try { ReflectionUtils.invokeMethodsWithAnnotation(OnScheduled.class, processor, context); } catch (final Exception e) { e.printStackTrace(); Assert.fail("Could not invoke methods annotated with

Re: Error handling in @OnScheduled

2018-08-23 Thread Mike Thomsen
For unit tests, if you're doing this to catch a failure scenario, you should be able to wrap the failing call in something like this: final def msg = "Lorem ipsum..." def error = null try { runner.run() } catch (Throwable t) { error = t } finally { assertNotNull(error)

Error handling in @OnScheduled

2018-08-23 Thread James Srinivasan
What is the best way to handle exceptions which might be thrown in my @OnScheduled method? Right now, I'm logging and propagating the exception which has the desired behaviour in NiFi (bulletin in GUI and processor cannot be started) but when trying to add a unit test, the (expected) exception is

Re: Reading all flowfiles queued for a processor (>20000 flowfiles)

2018-08-23 Thread TLdZPYSamI4WHPl1 TLdZPYSamI4WHPl1
Neat! So the BinFiles processor doesn't quite have the control over the binning/pairing that I wanted, but it got me on the right track. I think I've got a pretty lightweight custom processor that does what I'm looking for now. Made a processor which extends AbstractSessionFactoryProcessor that