Re: Reliable place to get latest release info for script

2017-04-26 Thread Dima Kovalyov
Thank you Matt, That is great reply and exactly what I am looking for until 777 PR is in progress. - Dima On 04/27/2017 01:46 AM, Matt Foley wrote: > Hi Dima, > Regardless of the github release area, etc., the official release repository > for Apache Metron are via the Apache mirrors at >

Re: Reliable place to get latest release info for script

2017-04-26 Thread Matt Foley
Hi Dima, Regardless of the github release area, etc., the official release repository for Apache Metron are via the Apache mirrors at http://www.apache.org/dyn/closer.cgi/incubator/metron/ soon to be moved to http://www.apache.org/dyn/closer.cgi/metron/ Please go there, pick a mir

Re: Reliable place to get latest release info for script

2017-04-26 Thread Dima Kovalyov
Otto, had a look at 777 PR. Yes, it looks like something we would use. When do you think it will be released? And in meantime the question still stands, is there a way to get current release id/branch for the script? Thank you. - Dima On 04/26/2017 10:56 PM, Otto Fowler wrote: No, Take a look

Re: Reliable place to get latest release info for script

2017-04-26 Thread Otto Fowler
No, Take a look at METRON-777 PR. What I want is an extension mechanism, where you can: 1. create a project from a metron maven archetype 2. produce an extension ( parser, enrichment, stellar lib, ?? ) 3. install that extension through the management UI and configure All without touching the met

Re: Reliable place to get latest release info for script

2017-04-26 Thread Dima Kovalyov
Regarding parsers and enrichment, new java based topologies for bluecoat, msexchange, msserver, asa, etc. Batching, we use NiFi + Kylo for both stream and batch, but batch data send to java based tool that passes data through parsers/enrichments classes and stores in Hive. So what you suggesti

Re: Reliable place to get latest release info for script

2017-04-26 Thread Otto Fowler
Thanks! Parsers, and Enrichments, we have a plan or idea about. When you say extends batch processing… can you say what tech/component/part of stack you extended? New topologies? New WriterBolts? etc? I would like as complete a picture as possible of the things for which anyone would say: “If I

Re: Reliable place to get latest release info for script

2017-04-26 Thread Dima Kovalyov
Otto, Yes, we developed custom parsers and enrichments (in future we seek to open source them, but have no time to up them to the decent level currently). We also merged our custom tools that extends Metron batching processing and store data in HDFS and Hive to be used by tableau, zeppelin, etc

Re: Reliable place to get latest release info for script

2017-04-26 Thread Otto Fowler
Can I ask Dima, as comfortable you are in describing, what areas do you make changes or additions to that require you to maintain a fork? Custom parsers? Enrichments? Other? I have done some work, and have more planned and lined up to try to eliminate the requirement to develop in the metron tre

Reliable place to get latest release info for script

2017-04-26 Thread Dima Kovalyov
Hello, We want to merge latest metron release branch with out develpoment every time it is get released. What is the path we should take here in order to identify what release happened successfully? Latest release is 0.3.1 currently, but since you're preparing for 0.4.0 (which will take a while)