Thanks for the quick reply Casey and for filing the bug. I've been able to confirm that I can use the expression (in this case on a field transformation in the parsing stage) but I have to type 4x \, so \\\\ to get a single backslash on the join field (I've got an expression like "new_field": "JOIN([field1,field2],'\\\\')". If I type 2 i get a similar exception to the one metioned on my initial message. Not sure if the expected behaviour on the parsing/enrichment should be to have to type it 4 times. It surely is something I can live with, but I'm just mentioning in case it is not the expected behaviour and something else needs fixing. Maybe it's also related to the fact that the config files sent to zookeeper are in JSON and there's some escaping on the submission process (although dumping the zookeeper config shows 4x \ - again that may be an unescaping done by the zookeeper dump).
Let me know if you need anything from my side. Unfortunately I cannot help much on the REPL side as I've got limited knowledge on that area. Thanks! 2017-08-02 11:25 GMT+02:00 Casey Stella <[email protected]>: > Ok, I think what you've found here is a bug in the REPL. I take it that > what you're looking for is JOIN( ['a', 'b'], '\\') == 'a\b' right? That is > a valid stellar expression, BUT because the REPL seems to be trying to > interpret the \\ before it gets to stellar, it's borking something. When I > run it in the REPL, I get: > [Stellar]>>> JOIN( [ 'a', 'b'], '\\') > > > > Notice that > at the end. That's the REPL asking for more input. This is > behaving like that because we based the REPL off of a shell interpreter. > OF course, in a shell interpreter \ indicates you want to continue the > input on the next line. We need to turn this behavior off in the REPL, > likely. > > > The actual functionality does work in Stellar outside of the REPL. We > actually even have unit tests for this case: https://github.com/ > apache/metron/blob/e206f2508ef7e7d798510df76ccfeb > 38b9530e89/metron-stellar/stellar-common/src/test/java/ > org/apache/metron/stellar/dsl/functions/BasicStellarTest.java#L120 > > I also just wrote a quick testcase and validated the very specific issue > that you had (note I use \\\\ because it's java. For you, it'd be \\): > @Test > public void testEscapedLiterals_test() { > Assert.assertEquals("a\\b", run("JOIN(['a', 'b'], '\\\\')", new > HashMap<>())); > } > > tl;dr: > If you used the expression as an enrichment, it should work. There's a > bug, however, in the REPL (filed @ https://issues.apache.org/ > jira/browse/METRON-1074) > > Thanks for your careful testing! > > On Tue, Aug 1, 2017 at 8:38 PM, Guillem Mateos <[email protected]> > wrote: > >> Hi, >> >> I'm trying to do something very simple with stellar and i'm running into >> issues, which i'd say point to a bug in stellar. I'm not very familiar with >> Antlr, so can't really tell exactly what is wrong, but based on the tests >> i've done, I'd say something it's not working properly. >> >> Example: >> >> JOIN(['a','b'],'\'') >> >> ouputs: a'b >> >> as one would expect. But then, try the following: >> >> JOIN(['a','b'],'\\') >> >> which simply fails and does not output anything when typed on the stellar >> cli (if you type another ' it throws a ParseException - no viable >> alternative). >> >> funny thing though, if you add another character, it does work. So: >> >> JOIN(['a','b'],'\\a') >> >> will output: a\ab >> >> Am I doing something wrong on how I write the escaping sequence or is >> this something that should be fixed on stellar? >> >> Thanks >> > >
