Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
Do we have any async methods in the current api I could look at? On April 27, 2017 at 10:45:09, Otto Fowler (ottobackwa...@gmail.com) wrote: I thought as much. I have not worked with the framework before however. On April 27, 2017 at 10:20:20, Ryan Merriman (merrim...@gmail.com) wrote: I

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
I thought as much. I have not worked with the framework before however. On April 27, 2017 at 10:20:20, Ryan Merriman (merrim...@gmail.com) wrote: I think leveraging the REST application would work for this use case. Services already exist for most of the functions listed in your pseudocode

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
I think this approach should be extended to the parsers too down the road as I’ve mentioned before. Installing a parser installed the configurations as templates, and from there you create new instances ( named ). Those instances are equivalent to what we have now. Install parser ->

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
Maybe the templates can be installed somewhere, not ES but some store. The UI can list the available templates, and you can edit and deploy from the UI? They can still be packaged with the parser, just not go to ES. Does that address your concerns? On April 27, 2017 at 10:25:37, Otto Fowler

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
“As a parser developer, I want to build, test, and maintain my parser extension as a single deliverable ‘package’”. I think from the 3rd party developer perspective, everything required to make the parser work, should be packaged and installed with the parser. The point is they may only be working

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Ryan Merriman
I think leveraging the REST application would work for this use case. Services already exist for most of the functions listed in your pseudocode (HDFS read/write, Zookeeper read/write). Asynchronous functions are also supported so no issues there. On Thu, Apr 27, 2017 at 9:09 AM, Otto Fowler

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Simon Elliston Ball
Otto, Happy to help around this. Couple of questions and things to maybe think about… REST seems fine for sending the payload, but the unpacking job should probably be async (triggered by the job, but not responsible for replying, i.e. the initial call gets an ACCEPTED response immediately

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
Also, if you want to help that would be great ;) On April 27, 2017 at 09:30:59, Otto Fowler (ottobackwa...@gmail.com) wrote: So, assuming ( I know I know ) that METRON-777 eventually lands, it will have lain the framework for the extension system for parsers out, including the capability to

[DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
So, assuming ( I know I know ) that METRON-777 eventually lands, it will have lain the framework for the extension system for parsers out, including the capability to create parsers outside the metron tree. The next step is METRON-258, side loading of parsers. This will be the effort to