Shell service? --
Sent with Airmail On September 25, 2016 at 10:54:47, Casey Stella ([email protected]) wrote: > The thought has crossed my mind, in fact I made sure that you can execute > sets of stellar statements by cat script | $METRON_HOME/bin/stellar -z > $ZK_QUORUM > > In fact that is how I generated the examples in the docs for the > management functions PR at > > https://github.com/cestella/incubator-metron/blob/00a46745f316c21f8ea2c24caeda4d2c2239da97/metron-platform/metron-management/README.md > > The challenge at the moment is the startup time to search the class path > for stellar functions at shell start. I have some ideas to make that go > away, but until that happens you will have to eat the cost per script run. > > In general just like the python command is nice as a repl, it also works > for executing scripts. We could do something similar with stellar. > > Casey > > On Sun, Sep 25, 2016 at 10:40 Otto Fowler <[email protected]> wrote: > >> Casey, >> >> Would things implemented this way be a better way to automate these flows >> ( built in error handling and checking as standards ) and not just for a >> user at the terminal? >> >> On September 24, 2016 at 19:02:59, Casey Stella ([email protected]) >> wrote: >> >> Sorry, I meant to ask if anyone had feedback or thoughts. >> On Sat, Sep 24, 2016 at 18:35 Casey Stella <[email protected]> wrote: >> >> > Grok-based parsers are an important part of Metron and Grok is an >> > extremely adaptable, if complex, language for specifying structure in >> data. >> > >> > >> > Right now, the flow is generally: >> > >> >> > 1. Get some sample data >> > 2. Use an online utility to test grok statements >> > 3. upload grok statement to HDFS >> > 4. restart parser topology >> >> > >> > Given the demo around managing stellar transformations inside of the >> REPL, >> > I began to think that it might be a more coherent and integrated >> experience >> > to manage grok inside of the REPL. >> > >> > This would involve stellar functions to >> > >> >> > - test grok statements >> > - read grok statements from HDFS >> > - push grok statements to HDFS >> >> > >> > I made a few JIRAs to outline it here >> >> > <https://issues.apache.org/jira/browse/METRON-454>. >> > >> >>
