Yes, but let’s see if anyone else find other issues.

 

From: Otto Fowler <ottobackwa...@gmail.com>
Date: Saturday, December 9, 2017 at 6:16 AM
To: Matt Foley <mfo...@hortonworks.com>, "dev@metron.apache.org" 
<dev@metron.apache.org>
Subject: Re: [DISCUSS] Upcoming Release

 

So RC2 then?

 

On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com) wrote:

Hah, here it is: https://github.com/apache/metron/pull/743 
“This problem seems to only reproduce when one unrolls a tarball rather than 
cloning from github.” 

Heh, the exclusion at https://github.com/apache/metron/blob/master/pom.xml#L351 
is still there, but the hashcode in the bundle.css file name has changed from 
a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version of Font 
Awesome fonts change? 


On 12/8/17, 5:26 PM, "Matt Foley" <ma...@apache.org> wrote: 

I remember having trouble with this bundle.css file on the last release, but I 
can’t remember what we did about it. Anybody? 

On 12/8/17, 1:41 PM, "Otto Fowler" <ottobackwa...@gmail.com> wrote: 

Steps 

- Downloaded tar.gz’s, asc files and KEYS 
- Verified signing of both tar.gz’s 
- searched for rouge 0.4.1 entries 
- verified the main pom.xml 
- built : 

mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T 
2C surefire:test@unit-tests && time mvn -q 
surefire:test@integration-tests && time mvn -q test --projects 
metron-interface/metron-config && time build_utils/verify_licenses.sh 

Found rat error: 


***************************************************** 
Summary 
------- 
Generated at: 2017-12-08T16:33:27-05:00 

Notes: 3 
Binaries: 193 
Archives: 0 
Standards: 75 

Apache Licensed: 74 
Generated Documents: 0 

JavaDocs are generated, thus a license header is optional. 
Generated files do not require license headers. 

1 Unknown Licenses 

***************************************************** 

Files with unapproved licenses: 

/Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
 

***************************************************** 





***************************************************** 
Summary 
------- 
Generated at: 2017-12-08T16:33:27-05:00 

Notes: 3 
Binaries: 193 
Archives: 0 
Standards: 75 

Apache Licensed: 74 
Generated Documents: 0 

JavaDocs are generated, thus a license header is optional. 
Generated files do not require license headers. 

1 Unknown Licenses 

***************************************************** 

Files with unapproved licenses: 


/Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
 

***************************************************** 



On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org) wrote: 

Colleagues, 
I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to 
https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/ 

Given the complexity of this RC, I’d appreciate if a couple people would be 
willing to kick the tires before we put it up for a vote. 

I will myself be going thru the Verify Build process this weekend, as I 
won’t be able to do it Friday. 

Thanks, 
--Matt 


On 12/4/17, 2:05 PM, "zeo...@gmail.com" <zeo...@gmail.com> wrote: 

Can we resolve the conversation regarding the second repo? I was waiting 
to get more input/preferences from people There's also a documentation 
update that fixes a few broken Stellar docs that already has aa +1, I just 
need to merge it. 

Jon 

On Mon, Dec 4, 2017, 17:01 Casey Stella <ceste...@gmail.com> wrote: 

> I would be in favor of a release at this point. 
> 
> On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley <ma...@apache.org> wrote: 
> 
> > Hey all, 
> > I see METRON-1252 was resolved over the weekend. Shall I go ahead and 
> > start the process with 0.4.2 release? 
> > Does anyone have any commits they feel strongly should go in before 
0.4.2 
> > is done, or are we ready to call it good? 
> > 
> > I believe there is consensus the 0.4.2 release should include a release 
> of 
> > the current state of the metron-bro-plugin-kafka. I will continue the 
> > discussion in that thread as to the process for accomplishing that, but 
> > plan on it happening. 
> > 
> > Regards, 
> > --Matt 
> > 
> > On 11/26/17, 6:26 PM, "Matt Foley" <ma...@apache.org> wrote: 
> > 
> > Hope everyone (at least in the U.S.) had a great Thanksgiving 
> holiday. 
> > Regarding status of the release effort, still pending METRON-1252, so 
> > not making the release branch yet. 
> > 
> > Regards, 
> > --Matt 
> > 
> > On 11/17/17, 1:32 PM, "Matt Foley" <ma...@apache.org> wrote: 
> > 
> > (With release manager hat on) 
> > 
> > The community has proposed a release of Metron in the near 
> future, 
> > focusing on Meta-alerts running in Elasticsearch. 
> > Congrats on getting so many of the below already done. At this 
> > point, only METRON-1252, and the discussion of how to handle joint 
> release 
> > of the Metron bro plugin, remain as gating items for the release. I 
> > project these will be resolved next week, so let’s propose the 
following: 
> > 
> > Sometime next week, after the last bits are done, I’ll start the 
> > release process and create the release branch. 
> > 
> > The proposed new version will be 0.4.2, unless there are backward 
> > incompatible changes that support making it 0.5.0. 
> > Currently there are NO included Jiras labeled 
> > ‘backward-incompatible’, nor having Docs Text indicating so. 
> > ***If anyone knows that some of the commits included since 0.4.1 
> > introduce backward incompatibility, please say so now on this thread, 
and 
> > mark the Jira as such.*** 
> > 
> > The 90 or so jiras/commits already in master branch since 0.4.1 
> > are listed below. 
> > Thanks, 
> > --Matt 
> > 
> > METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly 
> > Filters Some Records (nickwallen) closes apache/metron#832 
> > METRON-1294 IP addresses are not formatted correctly in facet 
> > and group results (merrimanr) closes apache/metron#827 
> > METRON-1291 Kafka produce REST endpoint does not work in a 
> > Kerberized cluster (merrimanr) closes apache/metron#826 
> > METRON-1290 Only first 10 alerts are update when a MetaAlert 
> > status is changed to inactive (justinleet) closes apache/metron#842 
> > METRON-1311 Service Check Should Check Elasticsearch Index 
> > Templates (nickwallen) closes apache/metron#839 
> > METRON-1289 Alert fields are lost when a MetaAlert is created 
> > (merrimanr) closes apache/metron#824 
> > METRON-1309 Change metron-deployment to pull the plugin from 
> > apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837 
> > METRON-1310 Template Delete Action Deletes Search Indices 
> > (nickwallen) closes apache/metron#838 
> > METRON-1275: Fix Metron Documentation closes 
> > apache/incubator-metron#833 
> > METRON-1295 Unable to Configure Logging for REST API 
> > (nickwallen) closes apache/metron#828 
> > METRON-1307 Force install of java8 since java9 does not 
> appear 
> > to work with the scripts (brianhurley via ottobackwards) closes 
> > apache/metron#835 
> > METRON-1296 Full Dev Fails to Deploy Index Templates 
> > (nickwallen via cestella) closes apache/incubator-metron#829 
> > METRON-1281 Remove hard-coded indices from the Alerts UI 
> > (merrimanr) closes apache/metron#821 
> > METRON-1287 Full Dev Fails When Installing EPEL Repository 
> > (nickwallen) closes apache/metron#820 
> > METRON-1267 Alerts UI returns a 404 when refreshing the 
> > alerts-list page (iraghumitra via merrimanr) closes apache/metron#819 
> > METRON-1283 Install Elasticsearch template as a part of the 
> > mpack startup scripts (anandsubbu via nickwallen) closes 
> apache/metron#817 
> > METRON-1254: Conditionals as map keys do not function in 
> > Stellar closes apache/incubator-metron#801 
> > METRON-1261 Apply bro security patch (JonZeolla via 
> > ottobackwards) closes apache/metron#805 
> > METRON-1284 Remove extraneous dead query in ElasticsearchDao 
> > (justinleet) closes apache/metron#818 
> > METRON-1270: fix for warnings missing @return tag argument in 
> > metron-analytics/metron-profiler-common and metron-profiler-client 
closes 
> > apache/incubator-metron#810 
> > METRON-1272 Hide child alerts from searches and grouping if 
> > they belong to meta alerts (justinleet) closes apache/metron#811 
> > METRON-1224 Add time range selection to search control 
> > (iraghumitra via james-sirota) closes apache/metron#796 
> > METRON-1280 0.4.1 -> 0.4.2 missed a couple of projects 
> > (cestella via justinleet) closes apache/metron#816 
> > METRON-1243: Add a REST endpoint which allows us to get a 
> list 
> > of all indice closes apache/incubator-metron#797 
> > METRON-1196 Increment master version number to 0.4.2 for 
> > on-going development (mattf-horton) closes apache/metron#767 
> > METRON-1278 Strip &quot;Build Status&quot; widget from root 
> > README.md in site-book build (mattf-horton) closes apache/metron#815 
> > METRON-1274 Master has failure in 
> > StormControllerIntegrationTest (merrimanr) closes apache/metron#813 
> > METRON-1266 Profiler - SASL Authentication Failed 
> (nickwallen) 
> > closes apache/metron#809 
> > METRON-1260 Include Alerts UI in Ambari Service Check 
> > (nickwallen) closes apache/metron#804 
> > METRON-1251: Typo and formatting fixes for metron-rest README 
> > closes apache/incubator-metron#800 
> > METRON-1241: Enable the REST API to use a cache for the 
> > zookeeper config similar to the Bolts closes 
apache/incubator-metron#795 
> > METRON-1267 Alerts UI returns a 404 when refreshing the 
> > alerts-list page (merrimanr) closes apache/metron#808 
> > METRON-1262 Unable to add comment for a alert in a meta-alert 
> > (merrimanr) closes apache/metron#806 
> > METRON-1263 Start Alerts UI service after Metron REST 
> > (anandsubbu via nickwallen) closes apache/metron#807 
> > METRON-1255 MetaAlert search is not filtering on status 
> > (merrimanr) closes apache/metron#802 
> > METRON-1249 Improve Metron MPack Service Checks (nickwallen) 
> > closes apache/metron#799 
> > METRON-1237 address javadoc warnings in metron-maas-common 
> > (dbist via james-sirota) closes apache/metron#792 
> > METRON-1240 address javadoc warnings in metron-platform and 
> > metron-analytics (dbist via james-sirota) closes apache/metron#794 
> > METRON-1226 Searching Can Errantly Query the Wrong Indices 
> > (nickwallen) closes apache/metron#793 
> > METRON-1081 Fix Alerts and Ops UI Notices file (james-sirota) 
> > closes apache/metron#682 
> > METRON-1123 Add group by option using faceted search 
> > capabilities of metron-rest-api (iraghumitra via james-sirota) closes 
> > apache/metron#768 
> > METRON-1223 Add support to add comments for alerts 
> > (iraghumitra via james-sirota) closes apache/metron#788 
> > METRON-1083 Add filters using faceted search capabilities of 
> > metron-rest-api (iraghumitra via james-sirota) closes apache/metron#710 
> > METRON-1232 Alert status changes are not reflected in list 
> > view (iraghumitra via merrimanr) closes apache/metron#787 
> > METRON-1247 REST search and findOne endpoints return 
> > unexpected or incorrect results for guids (justinleet) closes 
> > apache/metron#798 
> > METRON-1235: Document the properties pulled from the global 
> > configuration closes apache/incubator-metron#791 
> > METRON-1234: fix for WARNING 'dependencies.dependency.( 
> > groupId:artifactId:type:classifier)' must be unique: 
> > org.apache.hadoop:hadoop-yarn-api:jar (dbist via mmiklavc) closes 
> > apache/metron#790 
> > METRON-1222: fix warning for The expression ${parent.version} 
> > is deprecated. Please use ${project.parent.version} instead. (dbist via 
> > mmiklavc) closes apache/metron#782 
> > METRON-1220 Create documentation around alert nested field 
> > (justinleet) closes apache/metron#780 
> > METRON-1229 Management UI type is part of the declarations of 
> > 2 modules (merrimanr) closes apache/metron#784 
> > METRON-1228: Configuration Management PUSH immediately does 
> > DUMP after (mmiklavc via mmiklavc) closes apache/metron#783 
> > METRON-1218 Metron REST should return better error messages 
> > (merrimanr) closes apache/metron#779 
> > METRON-1161 Add ability to edit parser command line options 
> in 
> > the management UI (merrimanr) closes apache/metron#737 
> > METRON-1209: Make stellar repl take logging properties, like 
> > other CLI apps in metron closes apache/incubator-metron#772 
> > METRON-1059 address checkstyle warning AvoidStarImport in 
> > metron-stellar (dbist via ottobackwards) closes apache/metron#664 
> > METRON-1204 UI does not time out after being idle, but stops 
> > functioning (merrimanr) closes apache/metron#771 
> > METRON-1052: Add forensic similarity hash functions to 
> Stellar 
> > closes apache/incubator-metron#781 
> > METRON-632: Added validation of "shew.enrichmentType" and 
> > "shew.keyColumns" closes apache/incubator-metron#732 
> > METRON-1194 Add Profiler Debug Functions to Profiler README 
> > (nickwallen via ottobackwards) closes apache/metron#765 
> > METRON-1055 Metron 0.4.0 manual installation guide for CentOS 
> > 6 updates (lvets via ottobackwards) closes apache/metron#661 
> > METRON-1079 STELLAR NaN should be a keyword (ottobackwards) 
> > closes apache/metron#681 
> > METRON-1085 Add REST endpoint to save a user profile for the 
> > Alerts UI (merrimanr) closes apache/metron#694 
> > METRON-1208 MPack for Alerts UI (merrimanr) closes 
> > apache/metron#778 
> > METRON-1207 Make RPMs for Alerts UI (merrimanr) closes 
> > apache/metron#777 
> > METRON-1215 Fix link to RPMs chapter (DimDroll via 
> justinleet) 
> > closes apache/metron#776 
> > METRON-1206 Make alerts UI conform to ops UI for install 
> > (merrimanr) closes apache/metron#773 
> > METRON-1195 Meta alerts improperly handle updates to 
> non-alert 
> > fields (justinleet) closes apache/metron#766 
> > METRON-1189 Add alert escalation to the Alerts UI (merrimanr) 
> > closes apache/metron#762 
> > METRON-1156 Simulate Triage Rules in the Stellar REPL 
> > (nickwallen) closes apache/metron#733 
> > METRON-1198 Pycapa - No such configuration property 
> > 'sasl.kerberos.principal' (nickwallen) closes apache/metron#769 
> > METRON-1202 ElasticsearchDao Has extraneous sleep call 
> > (justinleet) closes apache/metron#770 
> > METRON-938 "service metron-rest start <password>" does not 
> > work on CentOS 7. (justinleet) closes apache/metron#757 
> > METRON-1182 Refactor Code in alert list to accommodate new 
> > view types (iraghumitra via merrimanr) closes apache/metron#756 
> > METRON-1188: Ambari global configuration management 
> (mmiklavc) 
> > closes apache/metron#760 
> > METRON-1191 update public web site to point at 0.4.1 new 
> > release (mattf-horton) closes apache/metron#764 
> > METRON-1063 address javadoc warnings in metron-stellar (dbist 
> > via ottobackwards) closes apache/metron#668 
> > METRON-1190 Fix Meta Alert Type handling in calculation of 
> > scores (justinleet) closes apache/metron#763 
> > METRON-1187 Indexing/Profiler Kafka ACL Groups Not Setup 
> > Correctly (nickwallen) closes apache/metron#759 
> > METRON-1185: Stellar REPL does not work on a kerberized 
> > cluster when calling functions interacting with HBase closes 
> > apache/incubator-metron#755 
> > METRON-1186: Profiler Functions use classutils from shaded 
> > storm closes apache/incubator-metron#758 
> > METRON-1173: Fix pointers to old stellar docs closes 
> > apache/incubator-metron#746 
> > METRON-1179: Make STATS_ADD to take a list closes 
> > apache/incubator-metron#750 
> > METRON-1180: Make Stellar Shell accept zookeeper quorum as a 
> > CSV list and not require a port closes apache/incubator-metron#751 
> > METRON-1183 Improve KDC Setup Instructions (nickwallen) 
> closes 
> > apache/metron#753 
> > METRON-1177 Stale running topologies seen post-kerberization 
> > and cause exceptions (nickwallen) closes apache/metron#748 
> > METRON-1158 Build backend for grouping alerts into meta 
> alerts 
> > (justinleet) closes apache/metron#734 
> > METRON-1146: Add ability to parse JSON string into JSONObject 
> > for stellar closes apache/incubator-metron#727 
> > METRON-1176 REST: HDFS Service should support setting 
> > permissions on files when writing (ottobackwards) closes 
> apache/metron#749 
> > METRON-1114 Add group by capabilities to search REST endpoint 
> > (merrimanr) closes apache/metron#702 
> > METRON-1167 Define Session Specific Global Configuration 
> > Values in the REPL (nickwallen) closes apache/metron#740 
> > METRON-1171: Better validation for the SUBSTRING stellar 
> > function closes apache/incubator-metron#745 
> > 
> > 
> > 
> > On 11/17/17, 11:59 AM, "Nick Allen" <n...@nickallen.org> wrote: 
> > 
> > I just wanted to send an update on where we are at. We've 
> > gotten a lot 
> > done here recently as you can see below. 
> > 
> > ✓ DONE (1) First, METRON-1289 needs to go in. This one was 
> > a fairly big 
> > effort and I am hearing that we are pretty close. 
> > 
> > ✓ DONE (2) METRON-1294 fixes an issue in how field types 
> are 
> > looked-up. 
> > 
> > ✓ DONE (3) METRON-1290 is next. While this may have been 
> > fixed in 
> > M-1289, there may be some test cases we want from this PR. 
> > 
> > ✓ DONE (4) METRON-1301 addresses a problem with the sorting 
> > logic. 
> > 
> > ✓ DONE (5) METRON-1291 fixes an issue with escalation of 
> > metaalerts. 
> > 
> > (6) That leads us to Raghu's UI work in METRON-1252. This 
> > introduces the 
> > UI bits that depend on all the previous backend work. 
> > 
> > (7) At this point, we should have our best effort at 
> running 
> > Metaalerts 
> > on Elasticsearch 2.x. I propose that we cut a release here. 
> > 
> > (8) After we cut the release, we can introduce the work for 
> > ES 5.x in 
> > METRON-939. I know we will need lots of help testing and 
> > reviewing this 
> > one. 
> > 
> > 
> > 
> > We also have an outstanding question that needs resolved 
> > BEFORE we 
> > release. We need to come to a consensus on how to release 
> > having moved our 
> > Bro Plugin to a separate repo. I don't think we've heard 
> from 
> > everyone on 
> > this. I'd urge everyone to chime in so we can choose a path 
> > forward. 
> > 
> > If anyone is totally confused in regards to that discussion, 
> I 
> > can try and 
> > send an options summary again as a separate discuss thread. 
> > The original 
> > chain was somewhere around here [1]. 
> > 
> > [1] 
> > https://lists.apache.org/thread.html/ 
> > 54a4474881b97e559df24728b3a0e923a58345a282451085eef832ef@% 
> > 3Cdev.metron.apache.org%3E 
> > 
> > 
> > 
> > On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen < 
> > n...@nickallen.org> wrote: 
> > 
> > > Hi Guys - 
> > > 
> > > I want to follow-up on this discussion. It sounds like 
> most 
> > people are in 
> > > agreement with the general approach. 
> > > 
> > > A lot of people have been working hard on Metaalerts and 
> > Elasticsearch. I 
> > > have checked-in with those doing the heavy lifting and have 
> > compiled a more 
> > > detailed plan based on where we are at now. To the best of 
> > my knowledge 
> > > here is the plan of attack for finishing out this effort. 
> > > 
> > > (1) First, METRON-1289 needs to go in. This one was a 
> > fairly big effort 
> > > and I am hearing that we are pretty close. 
> > > 
> > > (2) METRON-1294 fixes an issue in how field types are 
> > looked-up. 
> > > 
> > > (3) METRON-1290 is next. While this may have been fixed 
> > in M-1289, 
> > > there may be some test cases we want from this PR. 
> > > 
> > > (4) METRON-1301 addresses a problem with the sorting 
> logic. 
> > > 
> > > (5) METRON-1291 fixes an issue with escalation of 
> > metaalerts. 
> > > 
> > > (6) That leads us to Raghu's UI work in METRON-1252. 
> This 
> > introduces 
> > > the UI bits that depend on all the previous backend work. 
> > > 
> > > (7) At this point, we should have our best effort at 
> > running Metaalerts 
> > > on Elasticsearch 2.x. I propose that we cut a release here. 
> > > 
> > > (8) After we cut the release, we can introduce the work 
> > for ES 5.x in 
> > > METRON-939. I know we will need lots of help testing and 
> > reviewing this 
> > > one. 
> > > 
> > > Please correct me if I am wrong. I will try and send out 
> > updates as we 
> > > make progress. 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > On Mon, Nov 6, 2017 at 1:03 PM, zeo...@gmail.com < 
> > zeo...@gmail.com> wrote: 
> > > 
> > >> I agree, I think it's very reasonable to move in line with 
> > Nick's 
> > >> proposal. I would also suggest that we outline what the 
> > target versions 
> > >> would be to add in the METRON-777 components, since it has 
> > been functional 
> > >> for a very long time but not reviewed and has some really 
> > rockstar 
> > >> improvements. 
> > >> 
> > >> Jon 
> > >> 
> > >> On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler < 
> > ottobackwa...@gmail.com> 
> > >> wrote: 
> > >> 
> > >> > I think the ES cutover should be the start of the 0.5.x 
> > series, and we 
> > >> > continue on with 0.4.x for the 
> > >> > metadata improvements etc. We could chose to focus 
> > 0.5.x’s first 
> > >> releases 
> > >> > on not only ES but 
> > >> > getting a handle on kibana and the mpack situation as 
> > well. 
> > >> > 
> > >> > 
> > >> > 
> > >> > 
> > >> > On November 6, 2017 at 12:48:45, Michael Miklavcic ( 
> > >> > michael.miklav...@gmail.com) wrote: 
> > >> > 
> > >> > I agree with your proposal, Nick. I think having a 
> > stabilizing release 
> > >> > prior to upgrading ES/Kibana makes sense. 
> > >> > 
> > >> > On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen < 
> > n...@nickallen.org> wrote: 
> > >> > 
> > >> > > I would like to start a discussion around upcoming 
> > releases. We have a 
> > >> > > couple separate significant tracks of work that we 
> need 
> > to reconcile 
> > >> in 
> > >> > our 
> > >> > > release schedule. 
> > >> > > 
> > >> > > (1) We have had (and have in review) a good number of 
> > bug fixes 
> > >> required 
> > >> > to 
> > >> > > support Metaalerts on the existing Elasticsearch 2.x 
> > infrastructure. 
> > >> > > 
> > >> > > 
> > >> > > (2) We also have ongoing work to upgrade our 
> > infrastructure to 
> > >> > > Elasticsearch 5.x, which will not be backwards 
> > compatible. 
> > >> > > 
> > >> > > 
> > >> > > I would like to see a release that has our best work 
> on 
> > ES 2.x before 
> > >> we 
> > >> > > migrate to 5.x. I would propose the following. 
> > >> > > 
> > >> > > Release N+1: Introduce Metaalerts running on ES 2.x 
> > >> > > 
> > >> > > Release N+2: Cut-over to ES 5.x 
> > >> > > 
> > >> > > 
> > >> > > (Q) Is it worth cutting a separate release for ES 2.x? 
> > Is there a 
> > >> better 
> > >> > > way to handle the cut-over to 5.x? 
> > >> > > 
> > >> > 
> > >> -- 
> > >> 
> > >> Jon 
> > >> 
> > > 
> > > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> 
-- 

Jon 





Reply via email to