There seemed to be broad support for 0.5.0 in the previous discuss thread. Reopening that discussion is likely to add another couple days of discussion, which I would prefer to avoid. Just as a reminder, this is a 50,000 foot view of the changes.
- Significant performance improvements in Parsing, Enrichments, and Indexing - Support for Ubuntu installations - Support for Storm 1.1 and HDP 2.6 - Support for Elasticsearch 5.x - Event time processing in the Profiler - Complex document support in the JSONMapParser - Support for the Elasticsearch X-Pack - Run Stellar in a Zeppelin Notebook - Oodles of bug fixes and usability improvements On Wed, May 16, 2018 at 7:24 AM, Nick Allen <n...@nickallen.org> wrote: > Going to 0.5.0 is well justified without Solr IMO. > > On Wed, May 16, 2018, 7:01 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > >> My question is: Is updating the version a .4->.5 worthy change or would >> adding Solr be that change? >> Should we do another, last .4.x release and bump to .5 when solr hits? >> >> >> On May 15, 2018 at 17:31:27, Nick Allen (n...@nickallen.org) wrote: >> >> +1 That plan works for me. >> >> IMHO, I don't think there are any open PRs or JIRAs that we need to block >> the release on. >> >> I'd be open to cutting a release sooner too, but waiting until Tuesday >> also >> works. >> >> On Tue, May 15, 2018 at 4:39 PM, Justin Leet <justinjl...@gmail.com> >> wrote: >> >> > I have a minor adjustment to the proposed timeframe. I'd like to move >> the >> > tentative date for starting the RC process to Tuesday the 22nd. I have >> a >> > prior engagement on Monday that will consume the majority of the day. I >> > was just too excited about releasing. Sorry about the trouble. >> > >> > Justin >> > >> > On Tue, May 15, 2018 at 4:26 PM, Justin Leet <justinjl...@gmail.com> >> > wrote: >> > >> > > Hi all, >> > > >> > > Based on a thread from last week >> > > <https://lists.apache.org/thread.html/8e795ad4faecb4256f1cfa36ddbf75 >> > 7c9ef9b3d104eab9df6573d9a4@%3Cdev.metron.apache.org%3E>, >> > > I'll be taking over as the release manager for our next release. >> Thanks >> > > again to Matt for his work shepherding our previous releases! >> > > >> > > Version Number >> > > This thread >> > > <https://lists.apache.org/thread.html/d906ce1491b775d8719bdb1d9bca72 >> > b9ec7fe255260ab955b5cdd695@%3Cdev.metron.apache.org%3E> had >> > > a lot of support for the release bumping to 0.5.0 so I'd like to work >> off >> > > that, but we can adjust as needed. >> > > >> > > Proposed Timeframe >> > > I would tentatively like to start work on the RC this coming Monday >> 21st. >> > > This can be shifted as needed, either to make time for more PRs to >> make >> > it >> > > in or if we'd like to get things moving faster. >> > > >> > > I'm proposing we create this release from the Metron master branch, >> plus >> > > any commits the community considers necessary for the release and can >> get >> > > in the very near future. >> > > >> > > JIRA status >> > > There are 24 open PRs at https://github.com/apache/metron/pulls. If >> we >> > > consider any of these necessary for this release, we should work on >> > getting >> > > them closed. >> > > >> > > There have been 123 commits since the current release (see the end of >> the >> > > message). I'm sure there'll be updates in Jira required to get >> > everything >> > > up to date. I will follow up on that, and I'd ask that we work on >> > > preemptively cleaning our Jira up. >> > > >> > > Please respond with any specific PRs we'd like to have in for 0.5.0, >> > along >> > > with any Jira issues we feel are worth working on. In particular, in >> the >> > > previous discussion, https://issues.apache.org/ >> jira/browse/METRON-1550 >> > came >> > > up as something to potentially pull in, but I didn't see a consensus >> > form. >> > > >> > > Completed PRs as of May 15 as generated by Nick's nifty one liner >> > > git log --pretty="%cr %s" tags/apache-metron-0.4.2-release..HEAD >> > > >> > > Thanks, >> > > Justin >> > > >> > > 3 hours ago METRON-1552: Add gzip file validation check to the geo >> loader >> > > (mmiklavc via mmiklavc) closes apache/metron#1011 >> > > 23 hours ago METRON-1551 Profiler Should Not Use Java Serialization >> > > (nickwallen) closes apache/metron#1012 >> > > 4 days ago METRON-1549: Add empty object test to >> > WriterBoltIntegrationTest >> > > implementation (mmiklavc via mmiklavc) closes apache/metron#1009 >> > > 6 days ago METRON-1541 Mvn clean results in git status having deleted >> > > files. (justinleet via nickwallen) closes apache/metron#1003 >> > > 6 days ago METRON-1461 MIN MAX stellar function should take a stats >> or >> > > list object and return min/max (MohanDV via nickwallen) closes >> > > apache/metron#942 >> > > 6 days ago METRON-1184 EC2 Deployment - Updating control_path to >> > > accommodate for Linux (Ahmed Shah via ottobackwards) closes >> > > apache/metron#754 >> > > 6 days ago METRON-1530 Default proxy config settings in >> metron-contrib >> > > need to be updated (sardell via merrimanr) closes apache/metron#998 >> > > 11 days ago METRON-1545 Upgrade Spring and Spring Boot (merrimanr) >> closes >> > > apache/metron#1008 >> > > 13 days ago METRON-1543 Unable to Set Parser Output Topic in Sensor >> > Config >> > > (nickwallen) closes apache/metron#1007 >> > > 3 weeks ago METRON-1539: Specialized RENAME field transformer closes >> > > apache/incubator-metron#1002 >> > > 3 weeks ago METRON-1520: Add caching for stellar field >> transformations >> > > closes apache/incubator-metron#990 >> > > 3 weeks ago METRON-1529 CONFIG_GET Fails to Retrieve Latest Config >> When >> > > Run in Zeppelin REPL (nickwallen) closes apache/metron#997 >> > > 3 weeks ago METRON-1511 Unable to Serialize Profiler Configuration >> > > (nickwallen) closes apache/metron#982 >> > > 4 weeks ago METRON-1528: Fix missing file in metron.spec (mmiklavc >> via >> > > mmiklavc) closes apache/metron#996 >> > > 4 weeks ago METRON-1445: Update performance tuning guide with more >> > > explicit parameter instructions (mmiklavc via mmiklavc) closes >> > > apache/metron#988 >> > > 4 weeks ago METRON-1502 Upgrade Doxia plugin to 1.8 (justinleet) >> closes >> > > apache/metron#974 >> > > 4 weeks ago METRON-1527: Remove dead test file sitting in source >> folder >> > > (mmiklavc via mmiklavc) closes apache/metron#994 >> > > 4 weeks ago METRON-1499 Enable Configuration of Unified Enrichment >> > > Topology via Ambari (nickwallen) closes apache/metron#984 >> > > 4 weeks ago METRON-1515: Errors loading stellar functions currently >> bomb >> > > the entire topology, they should be recoverable closes >> > > apache/incubator-metron#985 >> > > 5 weeks ago METRON-1522 Fix the typo errors at profile debugger >> readme >> > > (MohanDV via nickwallen) closes apache/metron#992 >> > > 5 weeks ago METRON-1519 Indexing Error Topic Property Not Displayed >> in >> > > MPack (nickwallen) closes apache/metron#987 >> > > 5 weeks ago METRON-1347: Indexing Topology should fail tuples without >> a >> > > source.type (cstella via mmiklavc) closes apache/metron#863 >> > > 5 weeks ago Add support for Vagrant Cachier plugin if present >> > > (simonellistonball via mmiklavc) closes apache/metron#993 >> > > 5 weeks ago METRON-1521: JSONMapParser is no longer serializable >> closes >> > > apache/incubator-metron#991 >> > > 5 weeks ago METRON-1516 Support for Ansible 2.5.0 (ottobackwards) >> closes >> > > apache/metron#989 >> > > 5 weeks ago METRON-1494 Profiler Emits Messages to Kafka When Not >> Needed >> > > (nickwallen) closes apache/metron#967 >> > > 5 weeks ago METRON-1510 Update Metron website to include info about >> > github >> > > update subscription (anandsubbu) closes apache/metron#981 >> > > 5 weeks ago METRON-1518 Build Failure When Using Profile HDP-2.5.0.0 >> > > (nickwallen) closes apache/metron#986 >> > > 5 weeks ago METRON-1465:Support for Elasticsearch X-pack (wardbekker >> via >> > > mmiklavc) closes apache/metron#946 >> > > 5 weeks ago METRON-1504: Enriching missing values does not match the >> > > semantics between the new enrichment topology and old closes >> > > apache/incubator-metron#976 >> > > 5 weeks ago METRON-1505 Intermittent Profiler Integration Test >> Failure >> > > (nickwallen) closes apache/metron#977 >> > > 6 weeks ago METRON-1449 Set Zookeeper URL for Stellar Running in >> Zeppelin >> > > Notebook (nickwallen) closes apache/metron#931 >> > > 6 weeks ago METRON-1462: Separate ES and Kibana from Metron Mpack >> > > (mmiklavc via mmiklavc) closes apache/metron#943 >> > > 6 weeks ago METRON-1501 Parser messages that fail to validate are >> dropped >> > > silently (cestella via justinleet) closes apache/metron#972 >> > > 6 weeks ago METRON-1497 Rest endpoint '/api/v1/search/search' needs >> to >> > > handle null when elastic search response return null for >> getAggregations >> > > (MohanDV via justinleet) closes apache/metron#968 >> > > 8 weeks ago METRON-1500 Enhance 'prepare-commit' to Support Feature >> > > Branches (nickwallen) closes apache/metron#971 >> > > 8 weeks ago METRON-1491: The indexing topology restart logic is wrong >> > > (cstella via mmiklavc) closes apache/metron#964 >> > > 8 weeks ago METRON-590 Enable Use of Event Time in Profiler >> (nickwallen) >> > > closes apache/metron#965 >> > > 8 weeks ago METRON-1483: Create a tool to monitor performance of the >> > > topologies closes apache/incubator-metron#958 >> > > 8 weeks ago METRON-1487 Define Performance Benchmarks for Enrichment >> > > Topology (nickwallen) closes apache/metron#961 >> > > 9 weeks ago METRON-1493 Unhelpful Error Message When Assignment >> > > Expressions Fail (nickwallen) closes apache/metron#966 >> > > 9 weeks ago METRON-1397 Support for JSON Path and complex documents >> in >> > > JSONMapParser closes apache/incubator-metron#914 >> > > 9 weeks ago METRON-1299 In MetronError tests, don't test for HostName >> if >> > > getHostName wouldn't work closes apache/incubator-metron#924 >> > > 9 weeks ago METRON-1485 Upgrade vagrant for dev environments closes >> > > apache/incubator-metron#959 >> > > 9 weeks ago METRON-1488: user_settings hbase table does not have acls >> set >> > > up for kerberos closes apache/metron#962 >> > > 9 weeks ago METRON-1490: Better error message when user specifies an >> > > enrichment type that doesn't exist closes apache/metron#963 >> > > 10 weeks ago METRON-1468 Add support for >> apache/metron-bro-plugin-kafka >> > to >> > > prepare-commit (JonZeolla) closes apache/metron#948 >> > > 10 weeks ago METRON-1471: Migrate shuffle connections to local or >> shuffle >> > > closes apache/metron#949 >> > > 10 weeks ago METRON-1467: Replace guava caches in places where the >> > > keyspace might be large closes apache/metron#947 >> > > 10 weeks ago METRON-1463: Adjust the groupings and shuffles in >> enrichment >> > > to be more efficient closes apache/metron#944 >> > > 10 weeks ago METRON-1460: Create a complementary non-split-join >> > enrichment >> > > topology closes apache/metron#940 >> > > 2 months ago METRON-1470 Update jquery to version 3+ (xyztdanid via >> > > merrimanr) closes apache/metron#950 >> > > 2 months ago METRON-1450 Add REST endpoint docs for index topology >> split >> > > (MohanDV via nickwallen) closes apache/metron#936 >> > > 2 months ago METRON-1337 List of facets should not be hardcoded >> > > (merrimanr) closes apache/metron#853 >> > > 2 months ago METRON-1452 Rebase Dev Environment on Latest CentOS 6 >> > > (nickwallen) closes apache/metron#933 >> > > 3 months ago METRON-1457 Move ASF links to main page in the Metron >> > website >> > > (anandsubbu) closes apache/metron#938 >> > > 3 months ago METRON-1394 Create Rest endpoint to add the ACL for >> current >> > > user to kafka topics (MohanDV via merrimanr) closes apache/metron#895 >> > > 3 months ago METRON-941 native PaloAlto parser corrupts message when >> > > having a comma in the payload (ctramnitz via justinleet) closes >> > > apache/metron#579 >> > > 3 months ago METRON-1386: Fix Metron Website Required Links closes >> > > apache/incubator-metron#935 >> > > 3 months ago METRON-1455: Patch and Replace methods in the REST >> > > UpdateController return 400 this closes apache/incubator-metron#937 >> > > 3 months ago METRON-1318 Update MacOS Instructions for AWS >> (wardbekker >> > via >> > > nickwallen) closes apache/metron#930 >> > > 3 months ago METRON-1451 On Centos full dev, Metron Indexing shows up >> as >> > > stopped (anandsubbu via merrimanr) closes apache/metron#932 >> > > 3 months ago METRON-1444: Add Ubuntu Repositories for Elasticsearch >> to >> > the >> > > Mpack (mmiklavc via mmiklavc) closes apache/metron#928 >> > > 3 months ago METRON-1273: Website documentation link should point to >> the >> > > current site-book (JonZeolla via mmiklavc) closes apache/metron#812 >> > > 3 months ago METRON-1447 Heap Size Not Set Correctly by MPack for ES >> 5.x >> > > (nickwallen) closes apache/metron#927 >> > > 3 months ago METRON-1446: Fix openjdk issue with Ubuntu (mmiklavc via >> > > mmiklavc) closes apache/metron#926 >> > > 3 months ago METRON-1442: Split rest end points for indexing topology >> > into >> > > random access indexing and batch indexing this closes >> > > apache/incubator-metron#923 >> > > 3 months ago METRON-1443 Missing Critical MPack Install Instruction >> for >> > > Ubuntu (nickwallen) closes apache/metron#925 >> > > 3 months ago METRON-1438 STELLAR: Move shell functions to common from >> > > metron-management (ottobackwards) closes apache/metron#920 >> > > 3 months ago METRON-1435 Management UI cannot save json objects in >> > > advanced config (merrimanr) closes apache/metron#917 >> > > 3 months ago METRON-1439 Turn off git pager in platform-info script >> > > (justinleet) closes apache/metron#919 >> > > 4 months ago METRON-1091 STELLAR Shell: Stand Alone installation >> > > (ottobackwards) closes apache/metron#690 >> > > 4 months ago METRON-1427: Add support for storm 1.1 and hdp 2.6 >> (cstella >> > > via mmiklavc) closes apache/metron#907 >> > > 4 months ago METRON-1391 Typos in Documentation/Examples within >> > > metron-management/README.md (havran via ottobackwards) closes >> > > apache/metron#890 >> > > 4 months ago METRON-1389 Zeppelin notebook import does not work with >> > > Ambari 2.6 (anandsubbu) closes apache/metron#888 >> > > 4 months ago METRON-1432 JDK Install Fails on Ubuntu Development >> > > Environment (nickwallen) closes apache/metron#913 >> > > 4 months ago METRON-1431 Add REGEXP_REPLACE function to Stellar >> (havran >> > > via ottobackwards) closes apache/metron#912 >> > > 4 months ago METRON-1410 Some more upgrade fallout... Can't >> restart >> > > Metron Indexing. (ottobackwards) closes apache/metron#901 >> > > 4 months ago METRON-1370 Create Full Dev Equivalent for Ubuntu >> > (nickwallen >> > > via cestella) closes apache/incubator-metron#903 >> > > 4 months ago METRON-1430: Isolate jackson from being used as >> arguments or >> > > returns from JSONUtils closes apache/metron#910 >> > > 4 months ago METRON-1398 Exclude the basic-error-controller from >> being >> > > added to the swagger description (MohanDV via merrimanr) closes >> > > apache/metron#898 >> > > 4 months ago METRON-1392 Fix a test case to expect an Exception when >> > > replication factor more than number of brokers while creating topic >> > > (MohanDV via merrimanr) closes apache/metron#892 >> > > 4 months ago METRON-1413 Add Metron Commit Tool (nickwallen) closes >> > > apache/metron#902 >> > > 4 months ago METRON-1429 SearchIntegrationTest refactor (merrimanr) >> > closes >> > > apache/metron#909 >> > > 4 months ago METRON-1426: SensorIndexingConfigControllerIntegrationTest >> >> > > fails intermittently closes apache/metron#906 >> > > 4 months ago METRON-1417: Disable pcap-service by default in Monit >> > > (mmiklavc via mmiklavc) closes apache/metron#905 >> > > 4 months ago METRON-1400: Elasticsearch service check fails in Ambari >> > > (mmiklavc via mmiklavc) closes apache/metron#904 >> > > 4 months ago METRON-1428: Travis build failing from metron-config >> > > (mmiklavc via mmiklavc) closes apache/metron#908 >> > > 4 months ago METRON-1302: Split up Indexing Topology into batch and >> > random >> > > access sections closes apache/incubator-metron#831 >> > > 4 months ago METRON-1395 Documentation missing for Produce a message >> to a >> > > Kafka topic Rest API endpoint (MohanDV via nickwallen) closes >> > > apache/metron#897 >> > > 4 months ago METRON-1411 Fix sed command in Upgrading.md (justinleet) >> > > closes apache/metron#900 >> > > 4 months ago METRON-1326: Metron deploy with Kerberos fails on Ambari >> 2.5 >> > > during ES service stop (mmiklavc via mmiklavc) closes >> apache/metron#894 >> > > 4 months ago METRON-1380: Create a typosquatting use-case closes >> > > apache/incubator-metron#882 >> > > 4 months ago METRON-1230: As a stopgap prior to METRON-777, add more >> > > simplistic sideloading of custom Parsers closes >> > apache/incubator-metron#785 >> > > 4 months ago METRON-1378: Create a summarizer closes >> > > apache/incubator-metron#879 >> > > 4 months ago METRON-1231 Separate Sensor name and topic in the >> Management >> > > UI (merrimanr) closes apache/metron#786 >> > > 4 months ago METRON-1382 Run Stellar in a Zeppelin Notebook >> (nickwallen) >> > > closes apache/metron#884 >> > > 4 months ago METRON-1396 Fix .gitignore files to not ignore >> themselves >> > > (justinleet) closes apache/metron#896 >> > > 4 months ago METRON-1366: Add an entropy stellar function (cstella >> via >> > > mmiklavc) closes apache/metron#872 >> > > 4 months ago METRON-1390: Swagger UI for "Web Security Config" >> Controller >> > > needs request method (MohanDV via mmiklavc) closes apache/metron#889 >> > > 4 months ago METRON-1393: Fix bro Elasticsearch template (mmiklavc >> via >> > > mmiklavc) closes apache/metron#893 >> > > 4 months ago METRON-1379: Add an OBJECT_GET stellar function closes >> > > apache/incubator-metron#880 >> > > 4 months ago METRON-939: Upgrade ElasticSearch and Kibana (mmiklavc >> via >> > > mmiklavc) closes apache/metron#840 >> > > 4 months ago METRON-1377: Stellar function to generate typosquatted >> > > domains (similar to dnstwist) closes apache/incubator-metron#878 >> > > 4 months ago METRON-1385 Missing "properties" in index >> template >> > > causes ElasticsearchColumnMetadataDao.getColumnMetadata to fail >> > > (merrimanr) closes apache/metron#886 >> > > 4 months ago METRON-1388 update public web site to point at 0.4.2 new >> > > release (mattf-horton) closes apache/metron#887 >> > > 4 months ago METRON-1362 Improve Metron Deployment README >> (nickwallen) >> > > closes apache/metron#869 >> > > 4 months ago METRON-1384 Increment master version number to 0.4.3 for >> > > on-going development (mattf-horton via nickwallen) closes >> > apache/metron#885 >> > > 4 months ago METRON-1381 Add Apache license to MD files and remove >> the >> > Rat >> > > exclusion (justinleet) closes apache/metron#883 >> > > 4 months ago METRON-1071 Create CONTRIBUTING.md (justinleet) closes >> > > apache/metron#881 >> > > 5 months ago METRON-1373 RAT failure for >> metron-interface/metron-alerts >> > > (mattf-horton) closes apache/metron#875 >> > > 5 months ago METRON-1351 Create Installable Packages for Ubuntu >> Trusty >> > > (nickwallen) closes apache/metron#868 >> > > 5 months ago METRON-1376 RC Check Script should have named parameters >> > > (ottobackwards via nickwallen) closes apache/metron#877 >> > > 5 months ago METRON-1365: Allow PROFILE_GET to return a default value >> for >> > > a profile and entity that does not have a value written. closes >> > > apache/incubator-metron#871 >> > > 5 months ago METRON-1348 Metron Service Checks Use Wrong Hostname >> > > (nickwallen) closes apache/metron#864 >> > > 5 months ago METRON-1350: Add reservoir sampling functions to Stellar >> > > closes apache/incubator-metron#867 >> > > 5 months ago METRON-1374 Script the RC checking process >> (ottobackwards) >> > > closes apache/metron#876 >> > > 5 months ago METRON-1372 Validate JIRA for Releases (nickwallen) >> closes >> > > apache/metron#874 >> > > 5 months ago METRON-1345: Update EC2 README for custom Ansible >> (mmiklavc >> > > via mmiklavc) closes apache/metron#859 >> > > 5 months ago METRON-1349 Full Dev Builds Metron Twice (nickwallen) >> closes >> > > apache/metron#866 >> > > 5 months ago METRON-1343 Swagger UI for User Controller needs request >> > > method (MohanDV via ottobackwards) closes apache/metron#862 >> > > 5 months ago METRON-1306: When index template install fails, we >> should >> > > fail the install closes apache/incubator-metron#834 >> > > 5 months ago METRON-1341 Projection FieldTransformation >> > (simonellistonball >> > > via ottobackwards) closes apache/metron#861 >> > > >> > > >> > >> >>
