GitHub user kwmonroe opened a pull request:
https://github.com/apache/bigtop/pull/168
BIGTOP-2617: refresh juju spark-processing bundle
Similar to BIGTOP-2616, we've made improvements to the spark-processing
bundle that are ready for review and inclusion upstream:
- update readme for consistency with other bundles as well as calling out
new sparkbench capabilities
- refresh bundled charms, including the use of bigtop-zookeeper
- reduce spark machine count (2 units instead of 3, which still allows HA
without as much $$$ in cloud resources)
- ganglia/rsyslog have been updated from trusty to xenial
- add new machine constraints since aws now defaults to an 8gb root disk
(we want 32g at a minimum)
- run the bigtop smoke tests for spark/zookeeper
- ensure the proper amulet test harness is included in the bundle tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/juju-solutions/bigtop bug/sp-cwr-fixes
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bigtop/pull/168.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #168
----
commit 6c26d315e74ffddfa61dad7a8aa9bb0251706bd0
Author: Kevin W Monroe <[email protected]>
Date: 2016-10-28T15:14:48Z
reduce machine count (2 sparks instead of 3); refresh ganglia/rsyslog; use
bigtop zk; bump amulet timeout to an hour
commit 56f03b60aa3f438aa848aa7a0087816f6cb274d5
Author: Kevin W Monroe <[email protected]>
Date: 2016-10-28T16:58:09Z
use charm smoke tests in our bundle test
commit c02b8f8286952ad21a4ea9c1df63e035afea6cff
Author: Kevin W Monroe <[email protected]>
Date: 2016-10-28T17:26:37Z
bump spark revno (longer test timeout)
commit 62e66da9f08e821c2a8931f1e9dda8361fe32b0d
Author: Kevin W Monroe <[email protected]>
Date: 2016-10-28T21:11:00Z
bump revno to pick up spark and zk test fixes
commit 78c6d20b6b00eeab5912ae3f7b6279e8303488dc
Author: Kevin W Monroe <[email protected]>
Date: 2016-10-28T21:12:17Z
fix pgrep
commit 1fd7e88230a4141d046db9713a71ec0bd89a96e6
Author: Kevin W Monroe <[email protected]>
Date: 2016-10-28T21:40:36Z
pickup zk-test fix from zk
commit 32ac3e230c01ba0bfba373b76d6bc143f381b3b6
Author: Kevin W Monroe <[email protected]>
Date: 2016-10-28T23:12:21Z
refresh readme to be consistent with other bundles; add zk<->ganglia
monitoring; add a working ganglia unit
commit ffafa4f10cf0c15a1a910d8abcc7a6ef3f97e279
Author: Kevin W Monroe <[email protected]>
Date: 2016-10-31T16:41:15Z
update smoke failure message
commit 37b8387948834a7ecb14f5b1cddcd84e572cd300
Author: Kevin W Monroe <[email protected]>
Date: 2016-10-31T22:31:02Z
bump spark revno for spark-bench fixes
commit 412de035785fe0cc863714de7c985d23ef832ab1
Author: Kevin W Monroe <[email protected]>
Date: 2016-11-02T04:23:17Z
refresh spark charm and readme for new sparkbench suite
commit ee85b893f47448bfcbca787fe7d81f8812d0647b
Author: Kevin W Monroe <[email protected]>
Date: 2016-11-02T22:46:32Z
charm revision bump
- new ganglia charms report app version and fix '-broken' hook failures
- new spark charm correctly runs sparkbench regardless of spark exe mode
commit ac67e45672aa87b7db2d391cfc92201065067970
Author: Kevin W Monroe <[email protected]>
Date: 2016-11-04T22:50:41Z
bump spark rev to pick up testcase changes (be consistent with other bigtop
charms) and app status reporting (no hard coded strings)
commit d6230776c24a564bdfdb9e471fee2b963edeb160
Author: Kevin W Monroe <[email protected]>
Date: 2016-11-05T22:37:57Z
bump spark rev for updated test case and sparkbench composite is now seconds
commit a72b3af219ecf21004c37d85f38c0872542ef242
Author: Kevin W Monroe <[email protected]>
Date: 2016-11-07T17:49:08Z
bump zk revno to pick up test rename and app version
commit c6a83e7596aec9b2316027eb19b693a684593f16
Author: Cory Johns <[email protected]>
Date: 2016-11-14T21:57:37Z
spark-processing tests.yaml should include PPA for Amulet
commit 73db79d1390b078ad1addd43f855adcb7aae5647
Author: Kevin W Monroe <[email protected]>
Date: 2016-11-15T18:48:43Z
bump rsyslog charm revs for lint fix
commit 1e053bb5cfe2041af6efbd63efb3e667df78a053
Author: Kevin W Monroe <[email protected]>
Date: 2016-11-23T01:36:35Z
charm rev bump, new constraints
- bump zk to latest (consistent bigtop_ prefix, readme tweaks)
- aws defaults to 8gb root filesystem; bump this to 20gb
commit 2afcc96b9eb3895af201f98f916a9bfa847949d3
Author: Kevin W Monroe <[email protected]>
Date: 2016-11-24T19:48:14Z
rev charms to pick up new swap handler
commit 7b63bdd05982c3e94bef30e6ae0442372584e4c6
Author: Kevin W Monroe <[email protected]>
Date: 2016-11-28T23:12:24Z
workaround azure root-disk: https://bugs.launchpad.net/juju/+bug/1645408
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
