Good point, we should select a parser generator that produces tight code. Having said that, the MVP grammar is so simple that it could easily be put into a parser generator DSL later, and the MVP implementation could be hand-written for the first iteration (I'm picturing a couple dozen lines of code, but that's just a knee-jerk estimation).
> On May 4, 2017, at 11:25 AM, Marc <[email protected]> wrote: > > Andy, > Depending on the type of parser chosen you may run into memory > limitations on devices that run MiNiFi-C++. Since that's a deployment > concern -- and one that few will face, I would add that we can make this a > conditional build and only including the necessary components within the > binary if a build option is not specified to exclude these components. > > On Thu, May 4, 2017 at 11:14 AM, Andrew Christianson < > [email protected]> wrote: > >> The island grammar sounds very appealing for an MVP. Simple to implement, >> yet it covers a very common EL use case (dynamically inserting attr vals >> into property values). If we have general consensus I would love to see a >> MINIFI JIRA ticket added for this. >> >> -Andy >> ________________________________________ >> From: Matt Burgess <[email protected]> >> Sent: Thursday, May 4, 2017 11:09:46 AM >> To: [email protected] >> Subject: Re: MiNiFi C++ Expression Language >> >> What you have sounds good to me. IMO minimum viable product would be an >> island grammar (meaning you can have any characters outside a ${} >> expression) and inside would support an attribute name. Next steps could be >> nested expressions and/or support for functions, added piecemeal as the >> contributor sees fit. I wasn't involved in the early development of the EL >> grammar, so maybe someone who was has some thoughts on a natural evolution. >> >> Regards, >> Matt >> >>> On May 4, 2017, at 11:00 AM, Andrew Christianson <andrew.christianson@ >> nextcentury.com> wrote: >>> >>> My bad, what does the sketch of the plan *look like*? >>> >>> -Andy >>> ________________________________________ >>> From: Andrew Christianson >>> Sent: Thursday, May 4, 2017 10:59:07 AM >>> To: [email protected] >>> Subject: Re: MiNiFi C++ Expression Language >>> >>> What does the sketch of the plan to do the separate implementation? >> Write a flex/bison grammar, hook it into the cmake build, and start using >> it? Any constraints on features or syntax that this separate implementation >> must support? >>> >>> -Andy >>> ________________________________________ >>> From: Matt Burgess <[email protected]> >>> Sent: Thursday, May 4, 2017 9:37:34 AM >>> To: [email protected] >>> Subject: Re: MiNiFi C++ Expression Language >>> >>> No plans that I know of. In the meantime, EL support for MiNiFi is kind >> of held hostage, so maybe the separate implementation is more viable in the >> nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever >> exists by then with the cross-platform ANTLR target generation, and test >> the whole kit and caboodle. >>> >>> Anyone out there familiar with the aforementioned tools (or willing to >> hand-roll one)? What do you think about this "EL bootstrapping" approach? >>> >>> Thanks, >>> Matt >>> >>>> On May 4, 2017, at 9:22 AM, Andrew Christianson <andrew.christianson@ >> nextcentury.com> wrote: >>>> >>>> Got it. So the crux of the problem is porting from v3 to v4, plus the >> added uncertainty of the C++ v4 target. >>>> >>>> I'm assuming that NiFi wants to eventually get onto v4 anyway. If >> that's the case, then porting to v4 is probably the ticket. Are there any >> concrete plans to do so in the NiFi mother project yet? >>>> >>>> -Andy >>>> ________________________________________ >>>> From: Matt Burgess <[email protected]> >>>> Sent: Thursday, May 4, 2017 9:18:00 AM >>>> To: [email protected] >>>> Subject: Re: MiNiFi C++ Expression Language >>>> >>>> Correct, the current NiFi EL grammar is ANTLR3. >>>> >>>> >>>>> On May 4, 2017, at 9:12 AM, Andrew Christianson <andrew.christianson@ >> nextcentury.com> wrote: >>>>> >>>>> Do I understand correctly that NiFi is currently using ANTLRv3? >>>>> ________________________________________ >>>>> From: Matt Burgess <[email protected]> >>>>> Sent: Thursday, May 4, 2017 9:05:35 AM >>>>> To: [email protected] >>>>> Subject: Re: MiNiFi C++ Expression Language >>>>> >>>>> I haven't used Flex/Bison since a trivial example in college, so I'm >> not sure about the LOE for getting that set up, maybe there's a Maven-built >> project out there that we could look at for inspiration, but that seems >> unlikely :) >>>>> >>>>> An ANTLR4 refactor (assuming the C++ target is in good shape) would >> give us NiFi/MiNiFi EL compatibility (and full-featured EL support in >> MiNiFi C++), but we'd have to accept the risks of introducing bugs, >> regressions, etc. as a result of the refactor. Basically we'd just need to >> test the heck out of it on all platforms, which isn't a bad thing but adds >> to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for >> incremental development of a C/C++ based grammar. >>>>> >>>>> >>>>> On May 4, 2017, at 8:51 AM, Andrew Christianson <andrew.christianson@ >> nextcentury.com> wrote: >>>>> >>>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to >> refactor the existing grammar. Since the source and target for MiNiFi C++ >> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], >> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we >> are doing that with all the MiNiFi components already. The upside is being >> able to support EL incrementally as the grammar is developed. What do you >> think? >>>>>> >>>>>> This seems like a pragmatic approach. What's the level-of-effort >> required to do the initial grammar port and set up the build tooling? Less >> than refactoring for ANTLR4? I'm not as familiar with the EL grammar >> situation. >>>>>> >>>>>> -Andy >>>>>> ________________________________________ >>>>>> From: Matt Burgess <[email protected]> >>>>>> Sent: Thursday, May 4, 2017 8:46:20 AM >>>>>> To: [email protected] >>>>>> Subject: Re: MiNiFi C++ Expression Language >>>>>> >>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to >> refactor the existing grammar. Since the source and target for MiNiFi C++ >> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], >> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we >> are doing that with all the MiNiFi components already. The upside is being >> able to support EL incrementally as the grammar is developed. What do you >> think? >>>>>> >>>>>> Regards, >>>>>> Matt >>>>>> >>>>>> [1] http://dinosaur.compilertools.net/ >>>>>> [2] http://www.hwaci.com/sw/lemon/ >>>>>> [3] http://www.colm.net/open-source/ragel/ >>>>>> >>>>>> >>>>>> >>>>>> Sent from my iPhone >>>>>>> On May 4, 2017, at 8:13 AM, Marc P. <[email protected]> wrote: >>>>>>> >>>>>>> Andrew, >>>>>>> I am not aware of it being actively worked [1]. This would require >> using >>>>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone >> can >>>>>>> correct me if I'm wrong, but there would have to be changes to both >> sides. >>>>>>> I attempted a quick straw man with grammars, but didn't take it very >> far >>>>>>> after making initial changes to the grammar. It generated code, but >> I'm >>>>>>> uncertain of cross platform compatibility with the expression >> language. If >>>>>>> that's not expected or required that will remove some limitations as >> a >>>>>>> result of moving to ANTLR4. >>>>>>> >>>>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140 >>>>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4- >> c-target-is-here >>>>>>> >>>>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> All, >>>>>>>> >>>>>>>> I see that we do not have support for the expression language yet in >>>>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there >> an ETA? >>>>>>>> If no one is working on it, is there a general plan for how it >> should be >>>>>>>> implemented? I think I recall seeing references to ANTLR >>
