Re: [DISCUSS] Split Elasticsearch and Kibana into separate MPack from Metron
Hi Mike, I agree. Def +1 on the first approach. Keep it with the Metron version. -D... On Wed, Feb 21, 2018 at 11:30 AM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Anyone have any opinions on how we should version the ES/Kibana MPack? > > We currently rev the Metron one based on current Metron version and apply > it to both the overall MPack as well as the individual service versions, > e.g. metron_mpack-0.4.3.0/common-services/METRON/0.4.3. For ES we have > been > keeping the service version matched to the current ES version because it's > independent of Metron, ie 5.6.2. The upshot is that we can handle this one > of two ways. > >1. Keep the same approach after splitting off ES and Kibana - ES MPack >version is set to the current Metron version (0.4.3) and the service > itself >is set to the ES version (5.6.2). >2. Use ES version for both the MPack version and service version > (5.6.2). > > > I personally recommend and prefer the first approach because it allows us > to make changes to the mpack itself without necessarily changing the > version of ES or Kibana, which is something that is likely to happen. It's > also consistent and seamless with our current versioning approach. Lastly, > I don't believe the ES versions provide much contextual sense in the Metron > world for an MPack version - setting the service version definitely makes > sense to indicate what exact ES version we're using, but the MPack is > really our way of providing custom functionality that wraps a specific > version of ES/Kibana. Hope this makes sense. > > Mike > > > On Sat, Feb 17, 2018 at 11:13 AM, Nick Allen wrote: > > > +1 I agree with Otto's idea. It makes a lot of sense for Elasticsearch > and > > Kibana to live in a separate Mpack. > > > > This provides us a path forward to support additional indexers like Solr. > > > > We should also not force our users to install an external component (like > > Elasticsearch) using the Mpack. There are just too many installation > > configurations for us to reasonably support; especially on larger > > installations. Supporting the Elasticsearch MPack is a project unto > > itself. That being said, the functionality will still be there for those > > that want to use it. > > > > > > > > On Fri, Feb 16, 2018 at 4:10 PM, Michael Miklavcic < > > michael.miklav...@gmail.com> wrote: > > > > > This came up earlier when discussing work around the ES upgrade: > > > https://lists.apache.org/thread.html/66280bc061afbba2c353221c3c05fd > > > 74b247b970921c009c29edc815@%3Cdev.metron.apache.org%3E > > > https://lists.apache.org/thread.html/8ec83b6a3ef39057c9466ff72a2f63 > > > c9308452f1ebc1804e67cb495b@%3Cdev.metron.apache.org%3E > > > > > > Looks like Otto made this suggestion and Kyle is on board. I was > > originally > > > opposed to this because it did not seem worth the effort to support 2 > > > separate MPacks. However, now that we are working on the Solr upgrade, > it > > > seems like an appropriate solution for enabling us to make the indexing > > > piece pluggable. I propose that we commence with this solution. > > > > > > Cheers, > > > Mike > > > > > >
Re: [DISCUSS] Split Elasticsearch and Kibana into separate MPack from Metron
I agree, the first approach makes the most sense to me. Jon On Wed, Feb 21, 2018 at 11:45 AM Nick Allen wrote: > +1 to the first approach, as you've laid it out. That makes the most sense > to me. We need a way to rev the version of the ES Mpack independent of the > ES version. > > On Wed, Feb 21, 2018 at 11:30 AM, Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > > > Anyone have any opinions on how we should version the ES/Kibana MPack? > > > > We currently rev the Metron one based on current Metron version and apply > > it to both the overall MPack as well as the individual service versions, > > e.g. metron_mpack-0.4.3.0/common-services/METRON/0.4.3. For ES we have > > been > > keeping the service version matched to the current ES version because > it's > > independent of Metron, ie 5.6.2. The upshot is that we can handle this > one > > of two ways. > > > >1. Keep the same approach after splitting off ES and Kibana - ES MPack > >version is set to the current Metron version (0.4.3) and the service > > itself > >is set to the ES version (5.6.2). > >2. Use ES version for both the MPack version and service version > > (5.6.2). > > > > > > I personally recommend and prefer the first approach because it allows us > > to make changes to the mpack itself without necessarily changing the > > version of ES or Kibana, which is something that is likely to happen. > It's > > also consistent and seamless with our current versioning approach. > Lastly, > > I don't believe the ES versions provide much contextual sense in the > Metron > > world for an MPack version - setting the service version definitely makes > > sense to indicate what exact ES version we're using, but the MPack is > > really our way of providing custom functionality that wraps a specific > > version of ES/Kibana. Hope this makes sense. > > > > Mike > > > > > > On Sat, Feb 17, 2018 at 11:13 AM, Nick Allen wrote: > > > > > +1 I agree with Otto's idea. It makes a lot of sense for Elasticsearch > > and > > > Kibana to live in a separate Mpack. > > > > > > This provides us a path forward to support additional indexers like > Solr. > > > > > > We should also not force our users to install an external component > (like > > > Elasticsearch) using the Mpack. There are just too many installation > > > configurations for us to reasonably support; especially on larger > > > installations. Supporting the Elasticsearch MPack is a project unto > > > itself. That being said, the functionality will still be there for > those > > > that want to use it. > > > > > > > > > > > > On Fri, Feb 16, 2018 at 4:10 PM, Michael Miklavcic < > > > michael.miklav...@gmail.com> wrote: > > > > > > > This came up earlier when discussing work around the ES upgrade: > > > > https://lists.apache.org/thread.html/66280bc061afbba2c353221c3c05fd > > > > 74b247b970921c009c29edc815@%3Cdev.metron.apache.org%3E > > > > https://lists.apache.org/thread.html/8ec83b6a3ef39057c9466ff72a2f63 > > > > c9308452f1ebc1804e67cb495b@%3Cdev.metron.apache.org%3E > > > > > > > > Looks like Otto made this suggestion and Kyle is on board. I was > > > originally > > > > opposed to this because it did not seem worth the effort to support 2 > > > > separate MPacks. However, now that we are working on the Solr > upgrade, > > it > > > > seems like an appropriate solution for enabling us to make the > indexing > > > > piece pluggable. I propose that we commence with this solution. > > > > > > > > Cheers, > > > > Mike > > > > > > > > > > -- Jon
Re: [DISCUSS] Split Elasticsearch and Kibana into separate MPack from Metron
+1 to the first approach, as you've laid it out. That makes the most sense to me. We need a way to rev the version of the ES Mpack independent of the ES version. On Wed, Feb 21, 2018 at 11:30 AM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Anyone have any opinions on how we should version the ES/Kibana MPack? > > We currently rev the Metron one based on current Metron version and apply > it to both the overall MPack as well as the individual service versions, > e.g. metron_mpack-0.4.3.0/common-services/METRON/0.4.3. For ES we have > been > keeping the service version matched to the current ES version because it's > independent of Metron, ie 5.6.2. The upshot is that we can handle this one > of two ways. > >1. Keep the same approach after splitting off ES and Kibana - ES MPack >version is set to the current Metron version (0.4.3) and the service > itself >is set to the ES version (5.6.2). >2. Use ES version for both the MPack version and service version > (5.6.2). > > > I personally recommend and prefer the first approach because it allows us > to make changes to the mpack itself without necessarily changing the > version of ES or Kibana, which is something that is likely to happen. It's > also consistent and seamless with our current versioning approach. Lastly, > I don't believe the ES versions provide much contextual sense in the Metron > world for an MPack version - setting the service version definitely makes > sense to indicate what exact ES version we're using, but the MPack is > really our way of providing custom functionality that wraps a specific > version of ES/Kibana. Hope this makes sense. > > Mike > > > On Sat, Feb 17, 2018 at 11:13 AM, Nick Allen wrote: > > > +1 I agree with Otto's idea. It makes a lot of sense for Elasticsearch > and > > Kibana to live in a separate Mpack. > > > > This provides us a path forward to support additional indexers like Solr. > > > > We should also not force our users to install an external component (like > > Elasticsearch) using the Mpack. There are just too many installation > > configurations for us to reasonably support; especially on larger > > installations. Supporting the Elasticsearch MPack is a project unto > > itself. That being said, the functionality will still be there for those > > that want to use it. > > > > > > > > On Fri, Feb 16, 2018 at 4:10 PM, Michael Miklavcic < > > michael.miklav...@gmail.com> wrote: > > > > > This came up earlier when discussing work around the ES upgrade: > > > https://lists.apache.org/thread.html/66280bc061afbba2c353221c3c05fd > > > 74b247b970921c009c29edc815@%3Cdev.metron.apache.org%3E > > > https://lists.apache.org/thread.html/8ec83b6a3ef39057c9466ff72a2f63 > > > c9308452f1ebc1804e67cb495b@%3Cdev.metron.apache.org%3E > > > > > > Looks like Otto made this suggestion and Kyle is on board. I was > > originally > > > opposed to this because it did not seem worth the effort to support 2 > > > separate MPacks. However, now that we are working on the Solr upgrade, > it > > > seems like an appropriate solution for enabling us to make the indexing > > > piece pluggable. I propose that we commence with this solution. > > > > > > Cheers, > > > Mike > > > > > >
Re: [DISCUSS] Split Elasticsearch and Kibana into separate MPack from Metron
Anyone have any opinions on how we should version the ES/Kibana MPack? We currently rev the Metron one based on current Metron version and apply it to both the overall MPack as well as the individual service versions, e.g. metron_mpack-0.4.3.0/common-services/METRON/0.4.3. For ES we have been keeping the service version matched to the current ES version because it's independent of Metron, ie 5.6.2. The upshot is that we can handle this one of two ways. 1. Keep the same approach after splitting off ES and Kibana - ES MPack version is set to the current Metron version (0.4.3) and the service itself is set to the ES version (5.6.2). 2. Use ES version for both the MPack version and service version (5.6.2). I personally recommend and prefer the first approach because it allows us to make changes to the mpack itself without necessarily changing the version of ES or Kibana, which is something that is likely to happen. It's also consistent and seamless with our current versioning approach. Lastly, I don't believe the ES versions provide much contextual sense in the Metron world for an MPack version - setting the service version definitely makes sense to indicate what exact ES version we're using, but the MPack is really our way of providing custom functionality that wraps a specific version of ES/Kibana. Hope this makes sense. Mike On Sat, Feb 17, 2018 at 11:13 AM, Nick Allen wrote: > +1 I agree with Otto's idea. It makes a lot of sense for Elasticsearch and > Kibana to live in a separate Mpack. > > This provides us a path forward to support additional indexers like Solr. > > We should also not force our users to install an external component (like > Elasticsearch) using the Mpack. There are just too many installation > configurations for us to reasonably support; especially on larger > installations. Supporting the Elasticsearch MPack is a project unto > itself. That being said, the functionality will still be there for those > that want to use it. > > > > On Fri, Feb 16, 2018 at 4:10 PM, Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > > > This came up earlier when discussing work around the ES upgrade: > > https://lists.apache.org/thread.html/66280bc061afbba2c353221c3c05fd > > 74b247b970921c009c29edc815@%3Cdev.metron.apache.org%3E > > https://lists.apache.org/thread.html/8ec83b6a3ef39057c9466ff72a2f63 > > c9308452f1ebc1804e67cb495b@%3Cdev.metron.apache.org%3E > > > > Looks like Otto made this suggestion and Kyle is on board. I was > originally > > opposed to this because it did not seem worth the effort to support 2 > > separate MPacks. However, now that we are working on the Solr upgrade, it > > seems like an appropriate solution for enabling us to make the indexing > > piece pluggable. I propose that we commence with this solution. > > > > Cheers, > > Mike > > >
Re: [DISCUSS] Split Elasticsearch and Kibana into separate MPack from Metron
+1 I agree with Otto's idea. It makes a lot of sense for Elasticsearch and Kibana to live in a separate Mpack. This provides us a path forward to support additional indexers like Solr. We should also not force our users to install an external component (like Elasticsearch) using the Mpack. There are just too many installation configurations for us to reasonably support; especially on larger installations. Supporting the Elasticsearch MPack is a project unto itself. That being said, the functionality will still be there for those that want to use it. On Fri, Feb 16, 2018 at 4:10 PM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > This came up earlier when discussing work around the ES upgrade: > https://lists.apache.org/thread.html/66280bc061afbba2c353221c3c05fd > 74b247b970921c009c29edc815@%3Cdev.metron.apache.org%3E > https://lists.apache.org/thread.html/8ec83b6a3ef39057c9466ff72a2f63 > c9308452f1ebc1804e67cb495b@%3Cdev.metron.apache.org%3E > > Looks like Otto made this suggestion and Kyle is on board. I was originally > opposed to this because it did not seem worth the effort to support 2 > separate MPacks. However, now that we are working on the Solr upgrade, it > seems like an appropriate solution for enabling us to make the indexing > piece pluggable. I propose that we commence with this solution. > > Cheers, > Mike >