The island grammar sounds very appealing for an MVP. Simple to implement, yet it covers a very common EL use case (dynamically inserting attr vals into property values). If we have general consensus I would love to see a MINIFI JIRA ticket added for this.
-Andy ________________________________________ From: Matt Burgess <[email protected]> Sent: Thursday, May 4, 2017 11:09:46 AM To: [email protected] Subject: Re: MiNiFi C++ Expression Language What you have sounds good to me. IMO minimum viable product would be an island grammar (meaning you can have any characters outside a ${} expression) and inside would support an attribute name. Next steps could be nested expressions and/or support for functions, added piecemeal as the contributor sees fit. I wasn't involved in the early development of the EL grammar, so maybe someone who was has some thoughts on a natural evolution. Regards, Matt > On May 4, 2017, at 11:00 AM, Andrew Christianson > <[email protected]> wrote: > > My bad, what does the sketch of the plan *look like*? > > -Andy > ________________________________________ > From: Andrew Christianson > Sent: Thursday, May 4, 2017 10:59:07 AM > To: [email protected] > Subject: Re: MiNiFi C++ Expression Language > > What does the sketch of the plan to do the separate implementation? Write a > flex/bison grammar, hook it into the cmake build, and start using it? Any > constraints on features or syntax that this separate implementation must > support? > > -Andy > ________________________________________ > From: Matt Burgess <[email protected]> > Sent: Thursday, May 4, 2017 9:37:34 AM > To: [email protected] > Subject: Re: MiNiFi C++ Expression Language > > No plans that I know of. In the meantime, EL support for MiNiFi is kind of > held hostage, so maybe the separate implementation is more viable in the > nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever > exists by then with the cross-platform ANTLR target generation, and test the > whole kit and caboodle. > > Anyone out there familiar with the aforementioned tools (or willing to > hand-roll one)? What do you think about this "EL bootstrapping" approach? > > Thanks, > Matt > >> On May 4, 2017, at 9:22 AM, Andrew Christianson >> <[email protected]> wrote: >> >> Got it. So the crux of the problem is porting from v3 to v4, plus the added >> uncertainty of the C++ v4 target. >> >> I'm assuming that NiFi wants to eventually get onto v4 anyway. If that's the >> case, then porting to v4 is probably the ticket. Are there any concrete >> plans to do so in the NiFi mother project yet? >> >> -Andy >> ________________________________________ >> From: Matt Burgess <[email protected]> >> Sent: Thursday, May 4, 2017 9:18:00 AM >> To: [email protected] >> Subject: Re: MiNiFi C++ Expression Language >> >> Correct, the current NiFi EL grammar is ANTLR3. >> >> >>> On May 4, 2017, at 9:12 AM, Andrew Christianson >>> <[email protected]> wrote: >>> >>> Do I understand correctly that NiFi is currently using ANTLRv3? >>> ________________________________________ >>> From: Matt Burgess <[email protected]> >>> Sent: Thursday, May 4, 2017 9:05:35 AM >>> To: [email protected] >>> Subject: Re: MiNiFi C++ Expression Language >>> >>> I haven't used Flex/Bison since a trivial example in college, so I'm not >>> sure about the LOE for getting that set up, maybe there's a Maven-built >>> project out there that we could look at for inspiration, but that seems >>> unlikely :) >>> >>> An ANTLR4 refactor (assuming the C++ target is in good shape) would give us >>> NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), >>> but we'd have to accept the risks of introducing bugs, regressions, etc. as >>> a result of the refactor. Basically we'd just need to test the heck out of >>> it on all platforms, which isn't a bad thing but adds to the LOE for the >>> ANTLR4 upgrade, versus a smaller testing "surface" for incremental >>> development of a C/C++ based grammar. >>> >>> >>> On May 4, 2017, at 8:51 AM, Andrew Christianson >>> <[email protected]> wrote: >>> >>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor >>>>> the existing grammar. Since the source and target for MiNiFi C++ is, >>>>> well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], >>>>> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but >>>>> we are doing that with all the MiNiFi components already. The upside is >>>>> being able to support EL incrementally as the grammar is developed. What >>>>> do you think? >>>> >>>> This seems like a pragmatic approach. What's the level-of-effort required >>>> to do the initial grammar port and set up the build tooling? Less than >>>> refactoring for ANTLR4? I'm not as familiar with the EL grammar situation. >>>> >>>> -Andy >>>> ________________________________________ >>>> From: Matt Burgess <[email protected]> >>>> Sent: Thursday, May 4, 2017 8:46:20 AM >>>> To: [email protected] >>>> Subject: Re: MiNiFi C++ Expression Language >>>> >>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor >>>> the existing grammar. Since the source and target for MiNiFi C++ is, well, >>>> C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], >>>> Ragel [3], etc. The downside is maintaining two grammars, but we are doing >>>> that with all the MiNiFi components already. The upside is being able to >>>> support EL incrementally as the grammar is developed. What do you think? >>>> >>>> Regards, >>>> Matt >>>> >>>> [1] http://dinosaur.compilertools.net/ >>>> [2] http://www.hwaci.com/sw/lemon/ >>>> [3] http://www.colm.net/open-source/ragel/ >>>> >>>> >>>> >>>> Sent from my iPhone >>>>> On May 4, 2017, at 8:13 AM, Marc P. <[email protected]> wrote: >>>>> >>>>> Andrew, >>>>> I am not aware of it being actively worked [1]. This would require using >>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can >>>>> correct me if I'm wrong, but there would have to be changes to both sides. >>>>> I attempted a quick straw man with grammars, but didn't take it very far >>>>> after making initial changes to the grammar. It generated code, but I'm >>>>> uncertain of cross platform compatibility with the expression language. If >>>>> that's not expected or required that will remove some limitations as a >>>>> result of moving to ANTLR4. >>>>> >>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140 >>>>> [2] >>>>> http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here >>>>> >>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson < >>>>> [email protected]> wrote: >>>>> >>>>>> All, >>>>>> >>>>>> I see that we do not have support for the expression language yet in >>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an >>>>>> ETA? >>>>>> If no one is working on it, is there a general plan for how it should be >>>>>> implemented? I think I recall seeing references to ANTLR
