I agree with that concern. Being able to debug a running topology has helped in so many circumstances. Not sure how to accomplish this in a dockerized environment.
On Sat, Oct 1, 2016 at 13:11 Michael Miklavcic <[email protected]> wrote: > Any ideas on how debugging works when leveraging Docker? In spite of the > classpath troubles, one of the benefits of the current single-JVM approach > is that you can easily debug, set break points, etc within an IDE. Is that > still doable? Seems like there might need to be some remote debugging magic > done to accomplish this. > > On Fri, Sep 30, 2016 at 6:39 PM, James Sirota <[email protected]> wrote: > > > I agree with making an effort to create containers. I suggest doing it > on > > a feature branch until we are feature complete and are able to migrate > our > > integration tests into a dockerized environment. > > > > 30.09.2016, 14:52, "Nick Allen" <[email protected]>: > > > Relieve dependency version conflict issues? Umm, yes please. I'll > take a > > > second helping of that too. > > > > > > On Fri, Sep 30, 2016 at 5:00 PM, Ryan Merriman <[email protected]> > > wrote: > > > > > >> I would like to open up a discussion around creating Docker images > for > > >> Metron. Having this available would provide a leaner alternative to > > the > > >> ansible/vagrant environment for development tesing (and even demoing > > or > > >> exploring features). It could also relieve some of the dependency > > version > > >> conflict issues that we've been experiencing when running > integration > > tests > > >> in a single JVM. > > >> > > >> I would suggest the initial version be intended only for development > > and > > >> testing purposes. The general approach could be to create an image > for > > >> each service we depend on and use something like Docker compose to > > package > > >> them together. A Dockerfile would either install the service from > > scratch > > >> or extend a community image then add any Metron related dependencies > > or > > >> configurations on top. The metron-deployment project code could be > > used as > > >> a guide. > > >> > > >> I would like to see these images added initially to support > > development and > > >> testing: > > >> > > >> - Kafka with topics preconfigured > > >> - Storm with metron topology assets installed > > >> - Zookeeper with paths created and sample configs loaded > > >> - HBase with sample enrichments and threat intel loaded > > >> - Elasticsearch configured for Metron > > >> - MySQL with databases/tables/users created and geo data loaded > > >> > > >> Other images that could also be useful: > > >> > > >> - Images for each sensor > > >> - Ambari? > > >> - Solr > > >> > > >> Looking forward to hearing what everyone thinks. > > >> > > >> Ryan Merriman > > > > > > -- > > > Nick Allen <[email protected]> > > > > > > ------------------- > > Thank you, > > > > James Sirota > > PPMC- Apache Metron (Incubating) > > jsirota AT apache DOT org > > > > 30.09.2016, 14:52, "Nick Allen" <[email protected]>: > > > Relieve dependency version conflict issues? Umm, yes please. I'll take > a > > > second helping of that too. > > > > > > On Fri, Sep 30, 2016 at 5:00 PM, Ryan Merriman <[email protected]> > > wrote: > > > > > >> I would like to open up a discussion around creating Docker images > for > > >> Metron. Having this available would provide a leaner alternative to > the > > >> ansible/vagrant environment for development tesing (and even demoing > or > > >> exploring features). It could also relieve some of the dependency > > version > > >> conflict issues that we've been experiencing when running integration > > tests > > >> in a single JVM. > > >> > > >> I would suggest the initial version be intended only for development > > and > > >> testing purposes. The general approach could be to create an image > for > > >> each service we depend on and use something like Docker compose to > > package > > >> them together. A Dockerfile would either install the service from > > scratch > > >> or extend a community image then add any Metron related dependencies > or > > >> configurations on top. The metron-deployment project code could be > > used as > > >> a guide. > > >> > > >> I would like to see these images added initially to support > > development and > > >> testing: > > >> > > >> - Kafka with topics preconfigured > > >> - Storm with metron topology assets installed > > >> - Zookeeper with paths created and sample configs loaded > > >> - HBase with sample enrichments and threat intel loaded > > >> - Elasticsearch configured for Metron > > >> - MySQL with databases/tables/users created and geo data loaded > > >> > > >> Other images that could also be useful: > > >> > > >> - Images for each sensor > > >> - Ambari? > > >> - Solr > > >> > > >> Looking forward to hearing what everyone thinks. > > >> > > >> Ryan Merriman > > > > > > -- > > > Nick Allen <[email protected]> > > > > ------------------- > > Thank you, > > > > James Sirota > > PPMC- Apache Metron (Incubating) > > jsirota AT apache DOT org > > >
