So, I definitely hear what you're saying Otto, but I think since we had Solr support before (though it didn't really work as well as it is going to), I think we can do the solr work in a point release. I'd also be willing to move to 0.6 directly after Solr comes in if we think its' big enough. All that being said, I tend to agree that 0.5.0 is merited based on the performance improvements alone.
Just my $0.02. On Wed, May 16, 2018 at 8:12 AM Nick Allen <n...@nickallen.org> wrote: > There seemed to be broad support for 0.5.0 in the previous discuss > thread. Reopening that discussion is likely to add another couple days of > discussion, which I would prefer to avoid. Just as a reminder, this is a > 50,000 foot view of the changes. > > - Significant performance improvements in Parsing, Enrichments, and > Indexing > - Support for Ubuntu installations > - Support for Storm 1.1 and HDP 2.6 > - Support for Elasticsearch 5.x > - Event time processing in the Profiler > - Complex document support in the JSONMapParser > - Support for the Elasticsearch X-Pack > - Run Stellar in a Zeppelin Notebook > - Oodles of bug fixes and usability improvements > > > > > On Wed, May 16, 2018 at 7:24 AM, Nick Allen <n...@nickallen.org> wrote: > > > Going to 0.5.0 is well justified without Solr IMO. > > > > On Wed, May 16, 2018, 7:01 AM Otto Fowler <ottobackwa...@gmail.com> > wrote: > > > >> My question is: Is updating the version a .4->.5 worthy change or would > >> adding Solr be that change? > >> Should we do another, last .4.x release and bump to .5 when solr hits? > >> > >> > >> On May 15, 2018 at 17:31:27, Nick Allen (n...@nickallen.org) wrote: > >> > >> +1 That plan works for me. > >> > >> IMHO, I don't think there are any open PRs or JIRAs that we need to > block > >> the release on. > >> > >> I'd be open to cutting a release sooner too, but waiting until Tuesday > >> also > >> works. > >> > >> On Tue, May 15, 2018 at 4:39 PM, Justin Leet <justinjl...@gmail.com> > >> wrote: > >> > >> > I have a minor adjustment to the proposed timeframe. I'd like to move > >> the > >> > tentative date for starting the RC process to Tuesday the 22nd. I have > >> a > >> > prior engagement on Monday that will consume the majority of the day. > I > >> > was just too excited about releasing. Sorry about the trouble. > >> > > >> > Justin > >> > > >> > On Tue, May 15, 2018 at 4:26 PM, Justin Leet <justinjl...@gmail.com> > >> > wrote: > >> > > >> > > Hi all, > >> > > > >> > > Based on a thread from last week > >> > > < > https://lists.apache.org/thread.html/8e795ad4faecb4256f1cfa36ddbf75 > >> > 7c9ef9b3d104eab9df6573d9a4@%3Cdev.metron.apache.org%3E>, > >> > > I'll be taking over as the release manager for our next release. > >> Thanks > >> > > again to Matt for his work shepherding our previous releases! > >> > > > >> > > Version Number > >> > > This thread > >> > > < > https://lists.apache.org/thread.html/d906ce1491b775d8719bdb1d9bca72 > >> > b9ec7fe255260ab955b5cdd695@%3Cdev.metron.apache.org%3E> had > >> > > a lot of support for the release bumping to 0.5.0 so I'd like to > work > >> off > >> > > that, but we can adjust as needed. > >> > > > >> > > Proposed Timeframe > >> > > I would tentatively like to start work on the RC this coming Monday > >> 21st. > >> > > This can be shifted as needed, either to make time for more PRs to > >> make > >> > it > >> > > in or if we'd like to get things moving faster. > >> > > > >> > > I'm proposing we create this release from the Metron master branch, > >> plus > >> > > any commits the community considers necessary for the release and > can > >> get > >> > > in the very near future. > >> > > > >> > > JIRA status > >> > > There are 24 open PRs at https://github.com/apache/metron/pulls. If > >> we > >> > > consider any of these necessary for this release, we should work on > >> > getting > >> > > them closed. > >> > > > >> > > There have been 123 commits since the current release (see the end > of > >> the > >> > > message). I'm sure there'll be updates in Jira required to get > >> > everything > >> > > up to date. I will follow up on that, and I'd ask that we work on > >> > > preemptively cleaning our Jira up. > >> > > > >> > > Please respond with any specific PRs we'd like to have in for 0.5.0, > >> > along > >> > > with any Jira issues we feel are worth working on. In particular, in > >> the > >> > > previous discussion, https://issues.apache.org/ > >> jira/browse/METRON-1550 > >> > came > >> > > up as something to potentially pull in, but I didn't see a consensus > >> > form. > >> > > > >> > > Completed PRs as of May 15 as generated by Nick's nifty one liner > >> > > git log --pretty="%cr %s" tags/apache-metron-0.4.2-release..HEAD > >> > > > >> > > Thanks, > >> > > Justin > >> > > > >> > > 3 hours ago METRON-1552: Add gzip file validation check to the geo > >> loader > >> > > (mmiklavc via mmiklavc) closes apache/metron#1011 > >> > > 23 hours ago METRON-1551 Profiler Should Not Use Java Serialization > >> > > (nickwallen) closes apache/metron#1012 > >> > > 4 days ago METRON-1549: Add empty object test to > >> > WriterBoltIntegrationTest > >> > > implementation (mmiklavc via mmiklavc) closes apache/metron#1009 > >> > > 6 days ago METRON-1541 Mvn clean results in git status having > deleted > >> > > files. (justinleet via nickwallen) closes apache/metron#1003 > >> > > 6 days ago METRON-1461 MIN MAX stellar function should take a stats > >> or > >> > > list object and return min/max (MohanDV via nickwallen) closes > >> > > apache/metron#942 > >> > > 6 days ago METRON-1184 EC2 Deployment - Updating control_path to > >> > > accommodate for Linux (Ahmed Shah via ottobackwards) closes > >> > > apache/metron#754 > >> > > 6 days ago METRON-1530 Default proxy config settings in > >> metron-contrib > >> > > need to be updated (sardell via merrimanr) closes apache/metron#998 > >> > > 11 days ago METRON-1545 Upgrade Spring and Spring Boot (merrimanr) > >> closes > >> > > apache/metron#1008 > >> > > 13 days ago METRON-1543 Unable to Set Parser Output Topic in Sensor > >> > Config > >> > > (nickwallen) closes apache/metron#1007 > >> > > 3 weeks ago METRON-1539: Specialized RENAME field transformer closes > >> > > apache/incubator-metron#1002 > >> > > 3 weeks ago METRON-1520: Add caching for stellar field > >> transformations > >> > > closes apache/incubator-metron#990 > >> > > 3 weeks ago METRON-1529 CONFIG_GET Fails to Retrieve Latest Config > >> When > >> > > Run in Zeppelin REPL (nickwallen) closes apache/metron#997 > >> > > 3 weeks ago METRON-1511 Unable to Serialize Profiler Configuration > >> > > (nickwallen) closes apache/metron#982 > >> > > 4 weeks ago METRON-1528: Fix missing file in metron.spec (mmiklavc > >> via > >> > > mmiklavc) closes apache/metron#996 > >> > > 4 weeks ago METRON-1445: Update performance tuning guide with more > >> > > explicit parameter instructions (mmiklavc via mmiklavc) closes > >> > > apache/metron#988 > >> > > 4 weeks ago METRON-1502 Upgrade Doxia plugin to 1.8 (justinleet) > >> closes > >> > > apache/metron#974 > >> > > 4 weeks ago METRON-1527: Remove dead test file sitting in source > >> folder > >> > > (mmiklavc via mmiklavc) closes apache/metron#994 > >> > > 4 weeks ago METRON-1499 Enable Configuration of Unified Enrichment > >> > > Topology via Ambari (nickwallen) closes apache/metron#984 > >> > > 4 weeks ago METRON-1515: Errors loading stellar functions currently > >> bomb > >> > > the entire topology, they should be recoverable closes > >> > > apache/incubator-metron#985 > >> > > 5 weeks ago METRON-1522 Fix the typo errors at profile debugger > >> readme > >> > > (MohanDV via nickwallen) closes apache/metron#992 > >> > > 5 weeks ago METRON-1519 Indexing Error Topic Property Not Displayed > >> in > >> > > MPack (nickwallen) closes apache/metron#987 > >> > > 5 weeks ago METRON-1347: Indexing Topology should fail tuples > without > >> a > >> > > source.type (cstella via mmiklavc) closes apache/metron#863 > >> > > 5 weeks ago Add support for Vagrant Cachier plugin if present > >> > > (simonellistonball via mmiklavc) closes apache/metron#993 > >> > > 5 weeks ago METRON-1521: JSONMapParser is no longer serializable > >> closes > >> > > apache/incubator-metron#991 > >> > > 5 weeks ago METRON-1516 Support for Ansible 2.5.0 (ottobackwards) > >> closes > >> > > apache/metron#989 > >> > > 5 weeks ago METRON-1494 Profiler Emits Messages to Kafka When Not > >> Needed > >> > > (nickwallen) closes apache/metron#967 > >> > > 5 weeks ago METRON-1510 Update Metron website to include info about > >> > github > >> > > update subscription (anandsubbu) closes apache/metron#981 > >> > > 5 weeks ago METRON-1518 Build Failure When Using Profile HDP-2.5.0.0 > >> > > (nickwallen) closes apache/metron#986 > >> > > 5 weeks ago METRON-1465:Support for Elasticsearch X-pack (wardbekker > >> via > >> > > mmiklavc) closes apache/metron#946 > >> > > 5 weeks ago METRON-1504: Enriching missing values does not match the > >> > > semantics between the new enrichment topology and old closes > >> > > apache/incubator-metron#976 > >> > > 5 weeks ago METRON-1505 Intermittent Profiler Integration Test > >> Failure > >> > > (nickwallen) closes apache/metron#977 > >> > > 6 weeks ago METRON-1449 Set Zookeeper URL for Stellar Running in > >> Zeppelin > >> > > Notebook (nickwallen) closes apache/metron#931 > >> > > 6 weeks ago METRON-1462: Separate ES and Kibana from Metron Mpack > >> > > (mmiklavc via mmiklavc) closes apache/metron#943 > >> > > 6 weeks ago METRON-1501 Parser messages that fail to validate are > >> dropped > >> > > silently (cestella via justinleet) closes apache/metron#972 > >> > > 6 weeks ago METRON-1497 Rest endpoint '/api/v1/search/search' needs > >> to > >> > > handle null when elastic search response return null for > >> getAggregations > >> > > (MohanDV via justinleet) closes apache/metron#968 > >> > > 8 weeks ago METRON-1500 Enhance 'prepare-commit' to Support Feature > >> > > Branches (nickwallen) closes apache/metron#971 > >> > > 8 weeks ago METRON-1491: The indexing topology restart logic is > wrong > >> > > (cstella via mmiklavc) closes apache/metron#964 > >> > > 8 weeks ago METRON-590 Enable Use of Event Time in Profiler > >> (nickwallen) > >> > > closes apache/metron#965 > >> > > 8 weeks ago METRON-1483: Create a tool to monitor performance of the > >> > > topologies closes apache/incubator-metron#958 > >> > > 8 weeks ago METRON-1487 Define Performance Benchmarks for Enrichment > >> > > Topology (nickwallen) closes apache/metron#961 > >> > > 9 weeks ago METRON-1493 Unhelpful Error Message When Assignment > >> > > Expressions Fail (nickwallen) closes apache/metron#966 > >> > > 9 weeks ago METRON-1397 Support for JSON Path and complex documents > >> in > >> > > JSONMapParser closes apache/incubator-metron#914 > >> > > 9 weeks ago METRON-1299 In MetronError tests, don't test for > HostName > >> if > >> > > getHostName wouldn't work closes apache/incubator-metron#924 > >> > > 9 weeks ago METRON-1485 Upgrade vagrant for dev environments closes > >> > > apache/incubator-metron#959 > >> > > 9 weeks ago METRON-1488: user_settings hbase table does not have > acls > >> set > >> > > up for kerberos closes apache/metron#962 > >> > > 9 weeks ago METRON-1490: Better error message when user specifies an > >> > > enrichment type that doesn't exist closes apache/metron#963 > >> > > 10 weeks ago METRON-1468 Add support for > >> apache/metron-bro-plugin-kafka > >> > to > >> > > prepare-commit (JonZeolla) closes apache/metron#948 > >> > > 10 weeks ago METRON-1471: Migrate shuffle connections to local or > >> shuffle > >> > > closes apache/metron#949 > >> > > 10 weeks ago METRON-1467: Replace guava caches in places where the > >> > > keyspace might be large closes apache/metron#947 > >> > > 10 weeks ago METRON-1463: Adjust the groupings and shuffles in > >> enrichment > >> > > to be more efficient closes apache/metron#944 > >> > > 10 weeks ago METRON-1460: Create a complementary non-split-join > >> > enrichment > >> > > topology closes apache/metron#940 > >> > > 2 months ago METRON-1470 Update jquery to version 3+ (xyztdanid via > >> > > merrimanr) closes apache/metron#950 > >> > > 2 months ago METRON-1450 Add REST endpoint docs for index topology > >> split > >> > > (MohanDV via nickwallen) closes apache/metron#936 > >> > > 2 months ago METRON-1337 List of facets should not be hardcoded > >> > > (merrimanr) closes apache/metron#853 > >> > > 2 months ago METRON-1452 Rebase Dev Environment on Latest CentOS 6 > >> > > (nickwallen) closes apache/metron#933 > >> > > 3 months ago METRON-1457 Move ASF links to main page in the Metron > >> > website > >> > > (anandsubbu) closes apache/metron#938 > >> > > 3 months ago METRON-1394 Create Rest endpoint to add the ACL for > >> current > >> > > user to kafka topics (MohanDV via merrimanr) closes > apache/metron#895 > >> > > 3 months ago METRON-941 native PaloAlto parser corrupts message when > >> > > having a comma in the payload (ctramnitz via justinleet) closes > >> > > apache/metron#579 > >> > > 3 months ago METRON-1386: Fix Metron Website Required Links closes > >> > > apache/incubator-metron#935 > >> > > 3 months ago METRON-1455: Patch and Replace methods in the REST > >> > > UpdateController return 400 this closes apache/incubator-metron#937 > >> > > 3 months ago METRON-1318 Update MacOS Instructions for AWS > >> (wardbekker > >> > via > >> > > nickwallen) closes apache/metron#930 > >> > > 3 months ago METRON-1451 On Centos full dev, Metron Indexing shows > up > >> as > >> > > stopped (anandsubbu via merrimanr) closes apache/metron#932 > >> > > 3 months ago METRON-1444: Add Ubuntu Repositories for Elasticsearch > >> to > >> > the > >> > > Mpack (mmiklavc via mmiklavc) closes apache/metron#928 > >> > > 3 months ago METRON-1273: Website documentation link should point to > >> the > >> > > current site-book (JonZeolla via mmiklavc) closes apache/metron#812 > >> > > 3 months ago METRON-1447 Heap Size Not Set Correctly by MPack for ES > >> 5.x > >> > > (nickwallen) closes apache/metron#927 > >> > > 3 months ago METRON-1446: Fix openjdk issue with Ubuntu (mmiklavc > via > >> > > mmiklavc) closes apache/metron#926 > >> > > 3 months ago METRON-1442: Split rest end points for indexing > topology > >> > into > >> > > random access indexing and batch indexing this closes > >> > > apache/incubator-metron#923 > >> > > 3 months ago METRON-1443 Missing Critical MPack Install Instruction > >> for > >> > > Ubuntu (nickwallen) closes apache/metron#925 > >> > > 3 months ago METRON-1438 STELLAR: Move shell functions to common > from > >> > > metron-management (ottobackwards) closes apache/metron#920 > >> > > 3 months ago METRON-1435 Management UI cannot save json objects in > >> > > advanced config (merrimanr) closes apache/metron#917 > >> > > 3 months ago METRON-1439 Turn off git pager in platform-info script > >> > > (justinleet) closes apache/metron#919 > >> > > 4 months ago METRON-1091 STELLAR Shell: Stand Alone installation > >> > > (ottobackwards) closes apache/metron#690 > >> > > 4 months ago METRON-1427: Add support for storm 1.1 and hdp 2.6 > >> (cstella > >> > > via mmiklavc) closes apache/metron#907 > >> > > 4 months ago METRON-1391 Typos in Documentation/Examples within > >> > > metron-management/README.md (havran via ottobackwards) closes > >> > > apache/metron#890 > >> > > 4 months ago METRON-1389 Zeppelin notebook import does not work with > >> > > Ambari 2.6 (anandsubbu) closes apache/metron#888 > >> > > 4 months ago METRON-1432 JDK Install Fails on Ubuntu Development > >> > > Environment (nickwallen) closes apache/metron#913 > >> > > 4 months ago METRON-1431 Add REGEXP_REPLACE function to Stellar > >> (havran > >> > > via ottobackwards) closes apache/metron#912 > >> > > 4 months ago METRON-1410 Some more upgrade fallout... Can't > >> restart > >> > > Metron Indexing. (ottobackwards) closes apache/metron#901 > >> > > 4 months ago METRON-1370 Create Full Dev Equivalent for Ubuntu > >> > (nickwallen > >> > > via cestella) closes apache/incubator-metron#903 > >> > > 4 months ago METRON-1430: Isolate jackson from being used as > >> arguments or > >> > > returns from JSONUtils closes apache/metron#910 > >> > > 4 months ago METRON-1398 Exclude the basic-error-controller from > >> being > >> > > added to the swagger description (MohanDV via merrimanr) closes > >> > > apache/metron#898 > >> > > 4 months ago METRON-1392 Fix a test case to expect an Exception when > >> > > replication factor more than number of brokers while creating topic > >> > > (MohanDV via merrimanr) closes apache/metron#892 > >> > > 4 months ago METRON-1413 Add Metron Commit Tool (nickwallen) closes > >> > > apache/metron#902 > >> > > 4 months ago METRON-1429 SearchIntegrationTest refactor (merrimanr) > >> > closes > >> > > apache/metron#909 > >> > > 4 months ago METRON-1426: > SensorIndexingConfigControllerIntegrationTest > >> > >> > > fails intermittently closes apache/metron#906 > >> > > 4 months ago METRON-1417: Disable pcap-service by default in Monit > >> > > (mmiklavc via mmiklavc) closes apache/metron#905 > >> > > 4 months ago METRON-1400: Elasticsearch service check fails in > Ambari > >> > > (mmiklavc via mmiklavc) closes apache/metron#904 > >> > > 4 months ago METRON-1428: Travis build failing from metron-config > >> > > (mmiklavc via mmiklavc) closes apache/metron#908 > >> > > 4 months ago METRON-1302: Split up Indexing Topology into batch and > >> > random > >> > > access sections closes apache/incubator-metron#831 > >> > > 4 months ago METRON-1395 Documentation missing for Produce a message > >> to a > >> > > Kafka topic Rest API endpoint (MohanDV via nickwallen) closes > >> > > apache/metron#897 > >> > > 4 months ago METRON-1411 Fix sed command in Upgrading.md > (justinleet) > >> > > closes apache/metron#900 > >> > > 4 months ago METRON-1326: Metron deploy with Kerberos fails on > Ambari > >> 2.5 > >> > > during ES service stop (mmiklavc via mmiklavc) closes > >> apache/metron#894 > >> > > 4 months ago METRON-1380: Create a typosquatting use-case closes > >> > > apache/incubator-metron#882 > >> > > 4 months ago METRON-1230: As a stopgap prior to METRON-777, add more > >> > > simplistic sideloading of custom Parsers closes > >> > apache/incubator-metron#785 > >> > > 4 months ago METRON-1378: Create a summarizer closes > >> > > apache/incubator-metron#879 > >> > > 4 months ago METRON-1231 Separate Sensor name and topic in the > >> Management > >> > > UI (merrimanr) closes apache/metron#786 > >> > > 4 months ago METRON-1382 Run Stellar in a Zeppelin Notebook > >> (nickwallen) > >> > > closes apache/metron#884 > >> > > 4 months ago METRON-1396 Fix .gitignore files to not ignore > >> themselves > >> > > (justinleet) closes apache/metron#896 > >> > > 4 months ago METRON-1366: Add an entropy stellar function (cstella > >> via > >> > > mmiklavc) closes apache/metron#872 > >> > > 4 months ago METRON-1390: Swagger UI for "Web Security Config" > >> Controller > >> > > needs request method (MohanDV via mmiklavc) closes apache/metron#889 > >> > > 4 months ago METRON-1393: Fix bro Elasticsearch template (mmiklavc > >> via > >> > > mmiklavc) closes apache/metron#893 > >> > > 4 months ago METRON-1379: Add an OBJECT_GET stellar function closes > >> > > apache/incubator-metron#880 > >> > > 4 months ago METRON-939: Upgrade ElasticSearch and Kibana (mmiklavc > >> via > >> > > mmiklavc) closes apache/metron#840 > >> > > 4 months ago METRON-1377: Stellar function to generate typosquatted > >> > > domains (similar to dnstwist) closes apache/incubator-metron#878 > >> > > 4 months ago METRON-1385 Missing "properties" in index > >> template > >> > > causes ElasticsearchColumnMetadataDao.getColumnMetadata to fail > >> > > (merrimanr) closes apache/metron#886 > >> > > 4 months ago METRON-1388 update public web site to point at 0.4.2 > new > >> > > release (mattf-horton) closes apache/metron#887 > >> > > 4 months ago METRON-1362 Improve Metron Deployment README > >> (nickwallen) > >> > > closes apache/metron#869 > >> > > 4 months ago METRON-1384 Increment master version number to 0.4.3 > for > >> > > on-going development (mattf-horton via nickwallen) closes > >> > apache/metron#885 > >> > > 4 months ago METRON-1381 Add Apache license to MD files and remove > >> the > >> > Rat > >> > > exclusion (justinleet) closes apache/metron#883 > >> > > 4 months ago METRON-1071 Create CONTRIBUTING.md (justinleet) closes > >> > > apache/metron#881 > >> > > 5 months ago METRON-1373 RAT failure for > >> metron-interface/metron-alerts > >> > > (mattf-horton) closes apache/metron#875 > >> > > 5 months ago METRON-1351 Create Installable Packages for Ubuntu > >> Trusty > >> > > (nickwallen) closes apache/metron#868 > >> > > 5 months ago METRON-1376 RC Check Script should have named > parameters > >> > > (ottobackwards via nickwallen) closes apache/metron#877 > >> > > 5 months ago METRON-1365: Allow PROFILE_GET to return a default > value > >> for > >> > > a profile and entity that does not have a value written. closes > >> > > apache/incubator-metron#871 > >> > > 5 months ago METRON-1348 Metron Service Checks Use Wrong Hostname > >> > > (nickwallen) closes apache/metron#864 > >> > > 5 months ago METRON-1350: Add reservoir sampling functions to > Stellar > >> > > closes apache/incubator-metron#867 > >> > > 5 months ago METRON-1374 Script the RC checking process > >> (ottobackwards) > >> > > closes apache/metron#876 > >> > > 5 months ago METRON-1372 Validate JIRA for Releases (nickwallen) > >> closes > >> > > apache/metron#874 > >> > > 5 months ago METRON-1345: Update EC2 README for custom Ansible > >> (mmiklavc > >> > > via mmiklavc) closes apache/metron#859 > >> > > 5 months ago METRON-1349 Full Dev Builds Metron Twice (nickwallen) > >> closes > >> > > apache/metron#866 > >> > > 5 months ago METRON-1343 Swagger UI for User Controller needs > request > >> > > method (MohanDV via ottobackwards) closes apache/metron#862 > >> > > 5 months ago METRON-1306: When index template install fails, we > >> should > >> > > fail the install closes apache/incubator-metron#834 > >> > > 5 months ago METRON-1341 Projection FieldTransformation > >> > (simonellistonball > >> > > via ottobackwards) closes apache/metron#861 > >> > > > >> > > > >> > > >> > >> >
