Matt,
I think we are currently on version 0.4.2. If that is the case would the
next version be 0.4.3?
Ryan
On Fri, Nov 17, 2017 at 3:31 PM, Matt Foley <ma...@apache.org> wrote:
> (With release manager hat on)
>
> The community has proposed a release of Metron in the near future,
> focusing on Meta-alerts running in Elasticsearch.
> Congrats on getting so many of the below already done. At this point,
> only METRON-1252, and the discussion of how to handle joint release of the
> Metron bro plugin, remain as gating items for the release. I project these
> will be resolved next week, so let’s propose the following:
>
> Sometime next week, after the last bits are done, I’ll start the release
> process and create the release branch.
>
> The proposed new version will be 0.4.2, unless there are backward
> incompatible changes that support making it 0.5.0.
> Currently there are NO included Jiras labeled ‘backward-incompatible’, nor
> having Docs Text indicating so.
> ***If anyone knows that some of the commits included since 0.4.1 introduce
> backward incompatibility, please say so now on this thread, and mark the
> Jira as such.***
>
> The 90 or so jiras/commits already in master branch since 0.4.1 are listed
> below.
> Thanks,
> --Matt
>
> METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly Filters
> Some Records (nickwallen) closes apache/metron#832
> METRON-1294 IP addresses are not formatted correctly in facet and
> group results (merrimanr) closes apache/metron#827
> METRON-1291 Kafka produce REST endpoint does not work in a Kerberized
> cluster (merrimanr) closes apache/metron#826
> METRON-1290 Only first 10 alerts are update when a MetaAlert status is
> changed to inactive (justinleet) closes apache/metron#842
> METRON-1311 Service Check Should Check Elasticsearch Index Templates
> (nickwallen) closes apache/metron#839
> METRON-1289 Alert fields are lost when a MetaAlert is created
> (merrimanr) closes apache/metron#824
> METRON-1309 Change metron-deployment to pull the plugin from
> apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
> METRON-1310 Template Delete Action Deletes Search Indices (nickwallen)
> closes apache/metron#838
> METRON-1275: Fix Metron Documentation closes
> apache/incubator-metron#833
> METRON-1295 Unable to Configure Logging for REST API (nickwallen)
> closes apache/metron#828
> METRON-1307 Force install of java8 since java9 does not appear to work
> with the scripts (brianhurley via ottobackwards) closes apache/metron#835
> METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via
> cestella) closes apache/incubator-metron#829
> METRON-1281 Remove hard-coded indices from the Alerts UI (merrimanr)
> closes apache/metron#821
> METRON-1287 Full Dev Fails When Installing EPEL Repository
> (nickwallen) closes apache/metron#820
> METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list
> page (iraghumitra via merrimanr) closes apache/metron#819
> METRON-1283 Install Elasticsearch template as a part of the mpack
> startup scripts (anandsubbu via nickwallen) closes apache/metron#817
> METRON-1254: Conditionals as map keys do not function in Stellar
> closes apache/incubator-metron#801
> METRON-1261 Apply bro security patch (JonZeolla via ottobackwards)
> closes apache/metron#805
> METRON-1284 Remove extraneous dead query in ElasticsearchDao
> (justinleet) closes apache/metron#818
> METRON-1270: fix for warnings missing @return tag argument in
> metron-analytics/metron-profiler-common and metron-profiler-client closes
> apache/incubator-metron#810
> METRON-1272 Hide child alerts from searches and grouping if they
> belong to meta alerts (justinleet) closes apache/metron#811
> METRON-1224 Add time range selection to search control (iraghumitra
> via james-sirota) closes apache/metron#796
> METRON-1280 0.4.1 -> 0.4.2 missed a couple of projects (cestella via
> justinleet) closes apache/metron#816
> METRON-1243: Add a REST endpoint which allows us to get a list of all
> indice closes apache/incubator-metron#797
> METRON-1196 Increment master version number to 0.4.2 for on-going
> development (mattf-horton) closes apache/metron#767
> METRON-1278 Strip "Build Status" widget from root README.md
> in site-book build (mattf-horton) closes apache/metron#815
> METRON-1274 Master has failure in StormControllerIntegrationTest
> (merrimanr) closes apache/metron#813
> METRON-1266 Profiler - SASL Authentication Failed (nickwallen) closes
> apache/metron#809
> METRON-1260 Include Alerts UI in Ambari Service Check (nickwallen)
> closes apache/metron#804
> METRON-1251: Typo and formatting fixes for metron-rest README closes
> apache/incubator-metron#800
> METRON-1241: Enable the REST API to use a cache for the zookeeper
> config similar to the Bolts closes apache/incubator-metron#795
> METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list
> page (merrimanr) closes apache/metron#808
> METRON-1262 Unable to add comment for a alert in a meta-alert
> (merrimanr) closes apache/metron#806
> METRON-1263 Start Alerts UI service after Metron REST (anandsubbu via
> nickwallen) closes apache/metron#807
> METRON-1255 MetaAlert search is not filtering on status (merrimanr)
> closes apache/metron#802
> METRON-1249 Improve Metron MPack Service Checks (nickwallen) closes
> apache/metron#799
> METRON-1237 address javadoc warnings in metron-maas-common (dbist via
> james-sirota) closes apache/metron#792
> METRON-1240 address javadoc warnings in metron-platform and
> metron-analytics (dbist via james-sirota) closes apache/metron#794
> METRON-1226 Searching Can Errantly Query the Wrong Indices
> (nickwallen) closes apache/metron#793
> METRON-1081 Fix Alerts and Ops UI Notices file (james-sirota) closes
> apache/metron#682
> METRON-1123 Add group by option using faceted search capabilities of
> metron-rest-api (iraghumitra via james-sirota) closes apache/metron#768
> METRON-1223 Add support to add comments for alerts (iraghumitra via
> james-sirota) closes apache/metron#788
> METRON-1083 Add filters using faceted search capabilities of
> metron-rest-api (iraghumitra via james-sirota) closes apache/metron#710
> METRON-1232 Alert status changes are not reflected in list view
> (iraghumitra via merrimanr) closes apache/metron#787
> METRON-1247 REST search and findOne endpoints return unexpected or
> incorrect results for guids (justinleet) closes apache/metron#798
> METRON-1235: Document the properties pulled from the global
> configuration closes apache/incubator-metron#791
> METRON-1234: fix for WARNING 'dependencies.dependency.(
> groupId:artifactId:type:classifier)' must be unique:
> org.apache.hadoop:hadoop-yarn-api:jar (dbist via mmiklavc) closes
> apache/metron#790
> METRON-1222: fix warning for The expression ${parent.version} is
> deprecated. Please use ${project.parent.version} instead. (dbist via
> mmiklavc) closes apache/metron#782
> METRON-1220 Create documentation around alert nested field
> (justinleet) closes apache/metron#780
> METRON-1229 Management UI type is part of the declarations of 2
> modules (merrimanr) closes apache/metron#784
> METRON-1228: Configuration Management PUSH immediately does DUMP after
> (mmiklavc via mmiklavc) closes apache/metron#783
> METRON-1218 Metron REST should return better error messages
> (merrimanr) closes apache/metron#779
> METRON-1161 Add ability to edit parser command line options in the
> management UI (merrimanr) closes apache/metron#737
> METRON-1209: Make stellar repl take logging properties, like other CLI
> apps in metron closes apache/incubator-metron#772
> METRON-1059 address checkstyle warning AvoidStarImport in
> metron-stellar (dbist via ottobackwards) closes apache/metron#664
> METRON-1204 UI does not time out after being idle, but stops
> functioning (merrimanr) closes apache/metron#771
> METRON-1052: Add forensic similarity hash functions to Stellar closes
> apache/incubator-metron#781
> METRON-632: Added validation of "shew.enrichmentType" and
> "shew.keyColumns" closes apache/incubator-metron#732
> METRON-1194 Add Profiler Debug Functions to Profiler README
> (nickwallen via ottobackwards) closes apache/metron#765
> METRON-1055 Metron 0.4.0 manual installation guide for CentOS 6
> updates (lvets via ottobackwards) closes apache/metron#661
> METRON-1079 STELLAR NaN should be a keyword (ottobackwards) closes
> apache/metron#681
> METRON-1085 Add REST endpoint to save a user profile for the Alerts UI
> (merrimanr) closes apache/metron#694
> METRON-1208 MPack for Alerts UI (merrimanr) closes apache/metron#778
> METRON-1207 Make RPMs for Alerts UI (merrimanr) closes
> apache/metron#777
> METRON-1215 Fix link to RPMs chapter (DimDroll via justinleet) closes
> apache/metron#776
> METRON-1206 Make alerts UI conform to ops UI for install (merrimanr)
> closes apache/metron#773
> METRON-1195 Meta alerts improperly handle updates to non-alert fields
> (justinleet) closes apache/metron#766
> METRON-1189 Add alert escalation to the Alerts UI (merrimanr) closes
> apache/metron#762
> METRON-1156 Simulate Triage Rules in the Stellar REPL (nickwallen)
> closes apache/metron#733
> METRON-1198 Pycapa - No such configuration property
> 'sasl.kerberos.principal' (nickwallen) closes apache/metron#769
> METRON-1202 ElasticsearchDao Has extraneous sleep call (justinleet)
> closes apache/metron#770
> METRON-938 "service metron-rest start <password>" does not work on
> CentOS 7. (justinleet) closes apache/metron#757
> METRON-1182 Refactor Code in alert list to accommodate new view types
> (iraghumitra via merrimanr) closes apache/metron#756
> METRON-1188: Ambari global configuration management (mmiklavc) closes
> apache/metron#760
> METRON-1191 update public web site to point at 0.4.1 new release
> (mattf-horton) closes apache/metron#764
> METRON-1063 address javadoc warnings in metron-stellar (dbist via
> ottobackwards) closes apache/metron#668
> METRON-1190 Fix Meta Alert Type handling in calculation of scores
> (justinleet) closes apache/metron#763
> METRON-1187 Indexing/Profiler Kafka ACL Groups Not Setup Correctly
> (nickwallen) closes apache/metron#759
> METRON-1185: Stellar REPL does not work on a kerberized cluster when
> calling functions interacting with HBase closes apache/incubator-metron#755
> METRON-1186: Profiler Functions use classutils from shaded storm
> closes apache/incubator-metron#758
> METRON-1173: Fix pointers to old stellar docs closes
> apache/incubator-metron#746
> METRON-1179: Make STATS_ADD to take a list closes
> apache/incubator-metron#750
> METRON-1180: Make Stellar Shell accept zookeeper quorum as a CSV list
> and not require a port closes apache/incubator-metron#751
> METRON-1183 Improve KDC Setup Instructions (nickwallen) closes
> apache/metron#753
> METRON-1177 Stale running topologies seen post-kerberization and cause
> exceptions (nickwallen) closes apache/metron#748
> METRON-1158 Build backend for grouping alerts into meta alerts
> (justinleet) closes apache/metron#734
> METRON-1146: Add ability to parse JSON string into JSONObject for
> stellar closes apache/incubator-metron#727
> METRON-1176 REST: HDFS Service should support setting permissions on
> files when writing (ottobackwards) closes apache/metron#749
> METRON-1114 Add group by capabilities to search REST endpoint
> (merrimanr) closes apache/metron#702
> METRON-1167 Define Session Specific Global Configuration Values in the
> REPL (nickwallen) closes apache/metron#740
> METRON-1171: Better validation for the SUBSTRING stellar function
> closes apache/incubator-metron#745
>
>
>
> On 11/17/17, 11:59 AM, "Nick Allen" <n...@nickallen.org> wrote:
>
> I just wanted to send an update on where we are at. We've gotten a lot
> done here recently as you can see below.
>
> ✓ DONE (1) First, METRON-1289 needs to go in. This one was a fairly
> big
> effort and I am hearing that we are pretty close.
>
> ✓ DONE (2) METRON-1294 fixes an issue in how field types are
> looked-up.
>
> ✓ DONE (3) METRON-1290 is next. While this may have been fixed in
> M-1289, there may be some test cases we want from this PR.
>
> ✓ DONE (4) METRON-1301 addresses a problem with the sorting logic.
>
> ✓ DONE (5) METRON-1291 fixes an issue with escalation of metaalerts.
>
> (6) That leads us to Raghu's UI work in METRON-1252. This
> introduces the
> UI bits that depend on all the previous backend work.
>
> (7) At this point, we should have our best effort at running
> Metaalerts
> on Elasticsearch 2.x. I propose that we cut a release here.
>
> (8) After we cut the release, we can introduce the work for ES 5.x in
> METRON-939. I know we will need lots of help testing and reviewing
> this
> one.
>
>
>
> We also have an outstanding question that needs resolved BEFORE we
> release. We need to come to a consensus on how to release having
> moved our
> Bro Plugin to a separate repo. I don't think we've heard from
> everyone on
> this. I'd urge everyone to chime in so we can choose a path forward.
>
> If anyone is totally confused in regards to that discussion, I can try
> and
> send an options summary again as a separate discuss thread. The
> original
> chain was somewhere around here [1].
>
> [1]
> https://lists.apache.org/thread.html/54a4474881b97e559df24728b3a0e9
> 23a58345a282451085eef832ef@%3Cdev.metron.apache.org%3E
>
>
>
> On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen <n...@nickallen.org>
> wrote:
>
> > Hi Guys -
> >
> > I want to follow-up on this discussion. It sounds like most people
> are in
> > agreement with the general approach.
> >
> > A lot of people have been working hard on Metaalerts and
> Elasticsearch. I
> > have checked-in with those doing the heavy lifting and have compiled
> a more
> > detailed plan based on where we are at now. To the best of my
> knowledge
> > here is the plan of attack for finishing out this effort.
> >
> > (1) First, METRON-1289 needs to go in. This one was a fairly big
> effort
> > and I am hearing that we are pretty close.
> >
> > (2) METRON-1294 fixes an issue in how field types are looked-up.
> >
> > (3) METRON-1290 is next. While this may have been fixed in M-1289,
> > there may be some test cases we want from this PR.
> >
> > (4) METRON-1301 addresses a problem with the sorting logic.
> >
> > (5) METRON-1291 fixes an issue with escalation of metaalerts.
> >
> > (6) That leads us to Raghu's UI work in METRON-1252. This
> introduces
> > the UI bits that depend on all the previous backend work.
> >
> > (7) At this point, we should have our best effort at running
> Metaalerts
> > on Elasticsearch 2.x. I propose that we cut a release here.
> >
> > (8) After we cut the release, we can introduce the work for ES 5.x
> in
> > METRON-939. I know we will need lots of help testing and reviewing
> this
> > one.
> >
> > Please correct me if I am wrong. I will try and send out updates as
> we
> > make progress.
> >
> >
> >
> >
> >
> > On Mon, Nov 6, 2017 at 1:03 PM, zeo...@gmail.com <zeo...@gmail.com>
> wrote:
> >
> >> I agree, I think it's very reasonable to move in line with Nick's
> >> proposal. I would also suggest that we outline what the target
> versions
> >> would be to add in the METRON-777 components, since it has been
> functional
> >> for a very long time but not reviewed and has some really rockstar
> >> improvements.
> >>
> >> Jon
> >>
> >> On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler <
> ottobackwa...@gmail.com>
> >> wrote:
> >>
> >> > I think the ES cutover should be the start of the 0.5.x series,
> and we
> >> > continue on with 0.4.x for the
> >> > metadata improvements etc. We could chose to focus 0.5.x’s first
> >> releases
> >> > on not only ES but
> >> > getting a handle on kibana and the mpack situation as well.
> >> >
> >> >
> >> >
> >> >
> >> > On November 6, 2017 at 12:48:45, Michael Miklavcic (
> >> > michael.miklav...@gmail.com) wrote:
> >> >
> >> > I agree with your proposal, Nick. I think having a stabilizing
> release
> >> > prior to upgrading ES/Kibana makes sense.
> >> >
> >> > On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen <n...@nickallen.org>
> wrote:
> >> >
> >> > > I would like to start a discussion around upcoming releases. We
> have a
> >> > > couple separate significant tracks of work that we need to
> reconcile
> >> in
> >> > our
> >> > > release schedule.
> >> > >
> >> > > (1) We have had (and have in review) a good number of bug fixes
> >> required
> >> > to
> >> > > support Metaalerts on the existing Elasticsearch 2.x
> infrastructure.
> >> > >
> >> > >
> >> > > (2) We also have ongoing work to upgrade our infrastructure to
> >> > > Elasticsearch 5.x, which will not be backwards compatible.
> >> > >
> >> > >
> >> > > I would like to see a release that has our best work on ES 2.x
> before
> >> we
> >> > > migrate to 5.x. I would propose the following.
> >> > >
> >> > > Release N+1: Introduce Metaalerts running on ES 2.x
> >> > >
> >> > > Release N+2: Cut-over to ES 5.x
> >> > >
> >> > >
> >> > > (Q) Is it worth cutting a separate release for ES 2.x? Is there
> a
> >> better
> >> > > way to handle the cut-over to 5.x?
> >> > >
> >> >
> >> --
> >>
> >> Jon
> >>
> >
> >
>
>
>
>
