[GitHub] bigtop pull request: Adding Juju charms and bundles to the Bigtop ...

2016-05-09 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/108 Adding Juju charms and bundles to the Bigtop repo Initial PR to include Juju charms and bundles in the Bigtop repo. See the "Bigtop and Juju" note to the dev list on May 9 for more

[GitHub] bigtop issue #108: BIGTOP-2435 Add Juju charm, bundle, and testplan sources

2016-06-03 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/108 In an effort to reduce the size of this request, I have pulled out the Juju bundles (bigtop-deploy/juju) and Cloud Weather Report (bigtop-tests/cloud-weather-report). This just leaves the core set

[GitHub] bigtop pull request #112: BIGTOP-2468: Add Juju hadoop-processing bundle

2016-06-03 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/112 BIGTOP-2468: Add Juju hadoop-processing bundle We split the original Juju enablement issue (BIGTOP-2435) into multiple components for ease of review. This is the proposal to include the hadoop

[GitHub] bigtop pull request #108: BIGTOP-2435 Add Juju charms for hadoop component

2016-06-03 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/108 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop issue #108: BIGTOP-2435 Add Juju charms for hadoop component

2016-06-03 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/108 Thanks c0s! https://github.com/apache/bigtop/commit/d639645e7726577e5898f5da3722c6be65b2c9bd has the goods. Closing this out. --- If your project is set up for it, you can reply to this email

[GitHub] bigtop issue #112: BIGTOP-2468: Add Juju hadoop-processing bundle

2016-06-03 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/112 Conflicts with #108 resolved --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] bigtop pull request #112: BIGTOP-2468: Add Juju hadoop-processing bundle

2016-06-03 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/112 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop issue #112: BIGTOP-2468: Add Juju hadoop-processing bundle

2016-06-03 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/112 Thanks c0s and Cory! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] bigtop pull request #114: BIGTOP-2472: zeppelin puppet module needs to expos...

2016-06-06 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/114 BIGTOP-2472: zeppelin puppet module needs to expose ZEPPELIN_WAR_TEMPDIR You can merge this pull request into a Git repository by running: $ git pull https://github.com/juju-solutions/bigtop

[GitHub] bigtop issue #114: BIGTOP-2472: zeppelin puppet module needs to expose ZEPPE...

2016-06-06 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/114 Negative c0s, your commit is in there, but that only appears to affect the zeppelin deb. Your change to zeppelin-env.sh is not present in the puppet: https://github.com/apache/bigtop/blob

[GitHub] bigtop issue #114: BIGTOP-2472: zeppelin puppet module needs to expose ZEPPE...

2016-06-06 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/114 This came in with: https://github.com/apache/bigtop/commit/9299713a2c59b7c605ec8a3d118244562485bf59 Closing out PR. --- If your project is set up for it, you can reply to this

[GitHub] bigtop pull request #114: BIGTOP-2472: zeppelin puppet module needs to expos...

2016-06-06 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/114 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop pull request #115: BIGTOP-2475: hadoop_pig puppet module needs hadoop...

2016-06-08 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/115 BIGTOP-2475: hadoop_pig puppet module needs hadoop::common You can merge this pull request into a Git repository by running: $ git pull https://github.com/juju-solutions/bigtop BIGTOP-2475

[GitHub] bigtop issue #123: BIGTOP-2483: Add Mahout Charm

2016-06-17 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/123 Hey Kostas, this doesn't feel right to me. I've added a hadoop-client -> openjdk relation, followed by a hadoop-client -> mahout relation, but mahout still reports "Waiting

[GitHub] bigtop issue #115: BIGTOP-2475: hadoop_pig puppet module needs hadoop::commo...

2016-08-01 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/115 Closing PR; this came in with: https://github.com/apache/bigtop/commit/8efa779116eae0430ac173c5f1c7d44d4e13ef0b --- If your project is set up for it, you can reply to this email and have

[GitHub] bigtop pull request #115: BIGTOP-2475: hadoop_pig puppet module needs hadoop...

2016-08-01 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/115 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop pull request #144: BIGTOP-2525: committer test: add myself to team-li...

2016-08-30 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/144 BIGTOP-2525: committer test: add myself to team-list.xml Test my committer rights by adding myself to the team-list.xml. You can merge this pull request into a Git repository by running

[GitHub] bigtop issue #140: BIGTOP-2522: Add spark processing juju bundle

2016-09-19 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/140 Let's tweak this a bit. Some ELK components (including the *beat apps) aren't available on ppc64le (or s390x), so I'd like to see this go with the same monitors/loggers as `ha

[GitHub] bigtop issue #120: BIGTOP-2480: Fix occasional timeout on hadoop-processing ...

2016-09-30 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/120 @johnsca is this still valid? It's not clear to me how `00-deploy` is doing anything different than `01-deploy.py` is doing with its `d.setup` and `wait_for_messages` calls:

[GitHub] bigtop issue #147: BIGTOP-2547: Don't render ip-hostname-check in hdfs-site....

2016-10-04 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/147 Closed by https://github.com/apache/bigtop/commit/f1f5619a3a34e539a7b719712a885a26cdd161c4 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] bigtop pull request #137: BIGTOP-2516: Add Zeppelin Charm

2016-10-04 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/137#discussion_r81864360 --- Diff: bigtop-packages/src/charm/zeppelin/layer-zeppelin/tests/01-zeppelin-smoke.py --- @@ -0,0 +1,66 @@ +#!/usr/bin/env python3

[GitHub] bigtop pull request #137: BIGTOP-2516: Add Zeppelin Charm

2016-10-04 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/137#discussion_r81858734 --- Diff: bigtop-packages/src/charm/zeppelin/layer-zeppelin/README.md --- @@ -0,0 +1,119 @@ + +## Overview + +Apache Zeppelin is a web-based

[GitHub] bigtop pull request #137: BIGTOP-2516: Add Zeppelin Charm

2016-10-04 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/137#discussion_r81862021 --- Diff: bigtop-packages/src/charm/zeppelin/layer-zeppelin/lib/charms/layer/bigtop_zeppelin.py --- @@ -0,0 +1,238 @@ +# Licensed to the Apache

[GitHub] bigtop pull request #137: BIGTOP-2516: Add Zeppelin Charm

2016-10-04 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/137#discussion_r81859067 --- Diff: bigtop-packages/src/charm/zeppelin/layer-zeppelin/README.md --- @@ -0,0 +1,119 @@ + +## Overview + +Apache Zeppelin is a web-based

[GitHub] bigtop pull request #137: BIGTOP-2516: Add Zeppelin Charm

2016-10-04 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/137#discussion_r81864099 --- Diff: bigtop-packages/src/charm/zeppelin/layer-zeppelin/reactive/zeppelin.py --- @@ -0,0 +1,158 @@ +# Licensed to the Apache Software Foundation

[GitHub] bigtop pull request #148: BIGTOP-2548: Refresh charms for Juju 2.0 and Xenia...

2016-10-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/148 BIGTOP-2548: Refresh charms for Juju 2.0 and Xenial - update layer repo to upstream bigtop location - update metadata to inherit tags defined in bigtop base layer - update charm tests to

[GitHub] bigtop issue #120: BIGTOP-2480: Fix occasional timeout on hadoop-processing ...

2016-10-07 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/120 Let's kill this JIRA/PR @johnsca.. I've added `deployment_timeout` to the bundle refresh jira, which already bumped charm revnos: https://issues.apache.org/jira/browse/B

[GitHub] bigtop issue #129: BIGTOP-2476 Add Zookeeper Charm

2016-10-10 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/129 LGTM, verified deployment and smoke tests on azure and aws. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] bigtop issue #148: BIGTOP-2548: Refresh charms for Juju 2.0 and Xenial

2016-10-12 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/148 Thanks for the review guys. @johnsca you're right in that this was larger than expected, mostly due to the same changes (README/metadata/layer/smoke-test) going into each of the 4 hadoop c

[GitHub] bigtop pull request #149: Update juju hadoop bundle for Juju 2.0 and Xenial

2016-10-12 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/149 Update juju hadoop bundle for Juju 2.0 and Xenial - Update README with xenial and juju2 instructions. Callout juju1 equivalent commands where applicable. - Colocate apps that can live

[GitHub] bigtop issue #149: BIGTOP-2550: Update juju hadoop bundle for Juju 2.0 and X...

2016-10-12 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/149 ASF gitbot didn't link this PR to the jira. I think that's because I didn't title it correctly at the onset. Closing out to make a new one. --- If your project is set up for it, y

[GitHub] bigtop pull request #149: BIGTOP-2550: Update juju hadoop bundle for Juju 2....

2016-10-12 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/149 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop pull request #150: BIGTOP-2550: Update juju hadoop bundle for Juju 2....

2016-10-12 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/150 BIGTOP-2550: Update juju hadoop bundle for Juju 2.0 and Xenial - Update README with xenial and juju2 instructions. Callout juju1 equivalent commands where applicable. - Colocate apps that can

[GitHub] bigtop issue #149: BIGTOP-2550: Update juju hadoop bundle for Juju 2.0 and X...

2016-10-12 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/149 Duped PR. See #150. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] bigtop pull request #151: BIGTOP-2553: add rpc/http/https ports to template ...

2016-10-17 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/151 BIGTOP-2553: add rpc/http/https ports to template for non-HA mode Honor the `[namenode|http|https]_port` settings in non-HA mode. You can merge this pull request into a Git repository by running

[GitHub] bigtop issue #151: BIGTOP-2553: add rpc/http/https ports to template for non...

2016-10-17 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/151 @johnsca, `namenode_hosts` has only 1 item in non-HA mode. Note that once we hit the `else` to enter the "non HA" condition, we're not in any loop -- `namenode_hosts[0]` is how t

[GitHub] bigtop issue #151: BIGTOP-2553: add rpc/http/https ports to template for non...

2016-10-18 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/151 I think you're saying the `namenode_hosts[host, index]` loop is the only difference, but that's a pretty big difference. The hdfs-site.xml property keys are not the same in HA and no

[GitHub] bigtop pull request #151: BIGTOP-2553: add rpc/http/https ports to template ...

2016-10-19 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/151 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop issue #151: BIGTOP-2553: add rpc/http/https ports to template for non...

2016-10-19 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/151 Pushed upstream with: https://github.com/apache/bigtop/commit/cd55c144955e36188640c199b3100fefb3b28612 I forgot to add a "closes #151" to the commit, so i'm closi

[GitHub] bigtop pull request #152: BIGTOP-2554: expose bind-host options in hieradata

2016-10-19 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/152 BIGTOP-2554: expose bind-host options in hieradata Expose bind-host for RM and NN (rpc, http, and https) as hieradata options so we can set the appropriate config xml changes. You can merge this

[GitHub] bigtop pull request #153: BIGTOP-2555: hadoop charms should use bind-host ov...

2016-10-21 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/153 BIGTOP-2555: hadoop charms should use bind-host overrides - remove unused import - Bind NN and RM to 0.0.0.0 (all interfaces) using the hiera options from BIGTOP-2554. This fixes a problem

[GitHub] bigtop issue #153: BIGTOP-2555: hadoop charms should use bind-host overrides

2016-10-25 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/153 Thanks for the eyeballs @ktsakalozos and @johnsca! I missed a mapred.jobhistory binding. You will no longer see any 127.0.x.y bindings. Also, thanks @johnsca for pointing out the terasort

[GitHub] bigtop issue #153: BIGTOP-2555: hadoop charms should use bind-host overrides

2016-10-25 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/153 Charms in ~bigdata-dev namespace have been refreshed and verified that terasort works on lxd again: ``` results: meta: composite: direction: asc units: secs

[GitHub] bigtop issue #150: BIGTOP-2550: Update juju hadoop bundle for Juju 2.0 and X...

2016-10-25 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/150 Thanks all. I've refreshed the charm revision numbers to pick up the recent bind-host options (bigtop-2554) and fix an issue with terasorting on lxd. --- If your project is set up for it, yo

[GitHub] bigtop pull request #155: BIGTOP-2571: cwr driven updates to the zeppelin ju...

2016-11-04 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/155 BIGTOP-2571: cwr driven updates to the zeppelin juju charm Simplify our smoke test (it's must simpler and faster to smoke zeppelin/spark than it is to smoke zeppelin/hadoop).

[GitHub] bigtop issue #154: BIGTOP-2560: Spark charm failing automated tests

2016-11-04 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/154 @ktsakalozos i feel like i hijacked this PR from you, but I can assure you it was for the greater good! My recent commits have come as a result of watching cwr for the spark-processing bundle

[GitHub] bigtop pull request #156: BIGTOP-2575: zk charm test updates

2016-11-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/156 BIGTOP-2575: zk charm test updates Update the Juju charm for Zookeeper: - ensure we deploy correct zk charm and watch for correct status during tests - use new syntax for executing

[GitHub] bigtop pull request #157: BIGTOP-2577: kafka charm test updates

2016-11-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/157 BIGTOP-2577: kafka charm test updates Similar updates to what we did for the zk charm (BIGTOP-2575). Update the Juju charm for Kafka: - update readme with language consistent with other

[GitHub] bigtop pull request #156: BIGTOP-2575: zk charm test updates

2016-11-10 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/156#discussion_r87508997 --- Diff: bigtop-packages/src/charm/zookeeper/layer-zookeeper/tests/10-bind-address.py --- @@ -34,16 +33,16 @@ class TestBindClientPort(unittest.TestCase

[GitHub] bigtop issue #157: BIGTOP-2577: kafka charm test updates

2016-11-10 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/157 Good catch in zookeeper -- you suggested the better approach of relaxing the charm definition in the test.. Most recent commit should address your concerns. Thanks @petevg! --- If your project

[GitHub] bigtop pull request #162: BIGTOP-2570: ensure bigtop services are started

2016-11-16 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/162 BIGTOP-2570: ensure bigtop services are started We need to make sure our bigtop services are actually started before setting the `.started` state. Without this fix, our NN will set

[GitHub] bigtop issue #162: BIGTOP-2570: ease hadoop charm debugging

2016-11-18 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/162 Though this started with a systemctl service that failed to start, it has turned into a crusade to make charm actions/status/logging better so we can more easily debug problems like this in the

[GitHub] bigtop pull request #163: BIGTOP-2603: NN/RM charm should include a spark us...

2016-11-22 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/163 BIGTOP-2603: NN/RM charm should include a spark user/group When NN/RM are not colocated with spark, they do not know about the spark user/group. When in `yarn-client` mode, the spark charm will

[GitHub] bigtop pull request #166: BIGTOP-2561: add juju bundle for hadoop-spark

2016-12-02 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/166 BIGTOP-2561: add juju bundle for hadoop-spark This combines hadoop-processing and spark-processing into a single bundle: https://jujucharms.com/hadoop-spark/ Test results are

[GitHub] bigtop pull request #167: BIGTOP-2616: refresh juju hadoop-processing bundle

2016-12-02 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/167 BIGTOP-2616: refresh juju hadoop-processing bundle We've made a number of improvements to the hadoop-processing bundle that are ready for review and inclusion upstream: - ganglia/rsyslog

[GitHub] bigtop pull request #168: BIGTOP-2617: refresh juju spark-processing bundle

2016-12-02 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/168 BIGTOP-2617: refresh juju spark-processing bundle Similar to BIGTOP-2616, we've made improvements to the spark-processing bundle that are ready for review and inclusion upstream: - u

[GitHub] bigtop issue #161: BIGTOP-2586: hadoop-processing tests.yaml should include ...

2016-12-02 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/161 This went in with PR #167 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] bigtop issue #163: BIGTOP-2603: NN/RM charm should include a spark user/grou...

2016-12-05 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/163 Yeah, we probably should. `oozie` and `hue` come to mind. We chatted about this a little while ago. It would really be nice if the charm that relied on `hadoop-plugin` would send user info over

[GitHub] bigtop pull request #178: BIGTOP-2682: update juju bundles with recent charm...

2017-02-03 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/178 BIGTOP-2682: update juju bundles with recent charm revs You can merge this pull request into a Git repository by running: $ git pull https://github.com/juju-solutions/bigtop bug/BIGTOP-2682

[GitHub] bigtop pull request #187: BIGTOP-2703: refresh juju charms/bundles with metr...

2017-03-20 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/187 BIGTOP-2703: refresh juju charms/bundles with metric and CI support While this looks like a large PR, the majority is simply bumping charm revisions/locations in yaml files and updating READMEs to

[GitHub] bigtop pull request #189: BIGTOP-2712: Juju CI driven updates

2017-03-23 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/189 BIGTOP-2712: Juju CI driven updates The original point of this branch was to refresh the hbase charm (readme was out-of-date, needed lint love). However, as soon as ran CI on this and recent

[GitHub] bigtop pull request #194: BIGTOP-2737: Spark charm doesn't handle HA or exam...

2017-04-10 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/194 BIGTOP-2737: Spark charm doesn't handle HA or examples well See [jira](https://issues.apache.org/jira/browse/BIGTOP-2737) for details, but in general, the spark charm didn't transition

[GitHub] bigtop pull request #195: BIGTOP-2739: refresh juju bundles with latest char...

2017-04-12 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/195 BIGTOP-2739: refresh juju bundles with latest charm revs See [jira](https://issues.apache.org/jira/browse/BIGTOP-2739) for details. Notables: - add machine constraints back to workaround

[GitHub] bigtop pull request #196: BIGTOP-2740: hbase 1.1.3 does not work on ppc64le

2017-04-17 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/196 BIGTOP-2740: hbase 1.1.3 does not work on ppc64le HBase 1.1.x was broken on non-Intel arches until v1.1.4 ([HBASE-15322](https://issues.apache.org/jira/browse/HBASE-15322)). Pick up this

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-21 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/197 BIGTOP-2747: new charm revs for bigtop-1.2 Bring charm/bundle source up to date with bigtop-1.2. This is mostly a README/metadata refactor with sprinkles of a more consistent UX for charm

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-24 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/197#discussion_r112987956 --- Diff: bigtop-deploy/juju/hadoop-spark/bundle.yaml --- @@ -29,21 +29,21 @@ services: - "2" - "3" pl

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-24 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/197#discussion_r112989226 --- Diff: bigtop-packages/src/charm/hadoop/layer-hadoop-resourcemanager/actions/smoke-test --- @@ -43,6 +43,6 @@ smoke_env = { bigtop = Bigtop

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-24 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/197#discussion_r112990094 --- Diff: bigtop-packages/src/charm/pig/layer-pig/actions/smoke-test --- @@ -63,8 +62,8 @@ else: # The smoke tests analyze /etc/passwd

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-24 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/197#discussion_r112990603 --- Diff: bigtop-packages/src/charm/spark/layer-spark/actions/restart-spark-job-history-server --- @@ -15,22 +15,22 @@ # limitations under the

[GitHub] bigtop pull request #196: BIGTOP-2740: hbase 1.1.3 does not work on ppc64le

2017-05-05 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/196 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop issue #196: BIGTOP-2740: hbase 1.1.3 does not work on ppc64le

2017-05-05 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/196 Closed in https://github.com/apache/bigtop/commit/905bfe464864c8d77393b6dcf6788f3866e887fa --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] bigtop pull request #205: BIGTOP-2764: deployment failure when roles include...

2017-05-10 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/205 BIGTOP-2764: deployment failure when roles include spark::common and spark::yarn* We need to ensure the `spark-datanucleus` package is only defined once. There's a hack in place so it is a

[GitHub] bigtop pull request #206: BIGTOP-2765: update roles logic in spark/zeppelin

2017-05-10 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/206 BIGTOP-2765: update roles logic in spark/zeppelin Rework role logic for spark/zeppelin charms. - spark: only include master/worker when running in standalone mode - spark: include

[GitHub] bigtop pull request #208: BIGTOP-2766. [Puppet] Spark worker startup failed ...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/208#discussion_r116011550 --- Diff: bigtop-deploy/puppet/modules/spark/templates/spark-defaults.conf --- @@ -13,7 +13,15 @@ # See the License for the specific language

[GitHub] bigtop pull request #208: BIGTOP-2766. [Puppet] Spark worker startup failed ...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/208#discussion_r116002658 --- Diff: bigtop-deploy/puppet/manifests/cluster.pp --- @@ -72,6 +72,10 @@ master => ["spark-master"], worker =>

[GitHub] bigtop issue #208: BIGTOP-2766. [Puppet] Spark worker startup failed due to ...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/208 @evans-ye while you're updating spark's init.pp, would you consider replacing the `spark-datanucleus` definition with an `import spark::datanucleus`? https://github.com/evans

[GitHub] bigtop pull request #208: BIGTOP-2766. [Puppet] Spark worker startup failed ...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/208#discussion_r116045104 --- Diff: bigtop-deploy/puppet/modules/spark/templates/spark-defaults.conf --- @@ -13,7 +13,15 @@ # See the License for the specific language

[GitHub] bigtop issue #205: BIGTOP-2764: deployment failure when roles include spark:...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/205 I've updated the original workaround to include `spark::datanucleus` instead of redefining the package. This is much cleaner than including multiple comment blocks to explain the 2268

[GitHub] bigtop issue #205: BIGTOP-2764: deployment failure when roles include spark:...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/205 [spark-39](https://jujucharms.com/spark/39) contains this fix, and is green across clouds for hadoop + spark in yarn mode: http://bigtop.charm.qa/cwr_bundle_hadoop_spark/33/report.html

[GitHub] bigtop pull request #208: BIGTOP-2766. [Puppet] Spark worker startup failed ...

2017-05-12 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/208#discussion_r116296932 --- Diff: bigtop-deploy/puppet/modules/spark/templates/spark-defaults.conf --- @@ -13,7 +13,15 @@ # See the License for the specific language

[GitHub] bigtop issue #205: BIGTOP-2764: deployment failure when roles include spark:...

2017-05-16 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/205 Closed by https://github.com/apache/bigtop/commit/73864a30b29ee89655772ce82b4041c2564d0711 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] bigtop pull request #205: BIGTOP-2764: deployment failure when roles include...

2017-05-16 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/205 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop pull request #211: BIGTOP-2770: Juju charm/bundle refresh

2017-05-16 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/211 BIGTOP-2770: Juju charm/bundle refresh See [BIGTOP-2770](https://issues.apache.org/jira/browse/BIGTOP-2770) for details. You can merge this pull request into a Git repository by running

[GitHub] bigtop pull request #216: BIGTOP-2777: make hbase charm more robust

2017-05-23 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/216 BIGTOP-2777: make hbase charm more robust See [bigtop-2777](https://issues.apache.org/jira/browse/BIGTOP-2777) for details: - fix port numbers - expose heap and thrift config

[GitHub] bigtop pull request #217: BIGTOP-2779: new hive charm

2017-05-25 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/217 BIGTOP-2779: new hive charm note to upstream: squash ~16 before commit Key notes: - new hive charm - readme: be like the other bigtop charms - readme: describe metastore/hbase

[GitHub] bigtop pull request #228: BIGTOP-2783: new charm icons for zookeeper and zep...

2017-06-01 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/228 BIGTOP-2783: new charm icons for zookeeper and zeppelin The design team at Canonical has refreshed the charm icons for Zookeeper and Zeppelin to match the upstream project logos. Replace the

[GitHub] bigtop pull request #229: BIGTOP-2795: spark charm: fix sparkpi and rework s...

2017-06-02 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/229 BIGTOP-2795: spark charm: fix sparkpi and rework start/stop logic See [bigtop-2795](https://issues.apache.org/jira/browse/BIGTOP-2795) for details. - add bundle descriptions to the hadoop

[GitHub] bigtop issue #229: BIGTOP-2795: spark charm: fix sparkpi and rework start/st...

2017-06-02 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/229 During test, I found the `submit` action to be waaay too complex. I refactored it to use a simpler set of arguments, and then documented this in the readme. While I was in the readme, I

[GitHub] bigtop pull request #230: BIGTOP-2797: zeppelin charm external role handling

2017-06-04 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/230 BIGTOP-2797: zeppelin charm external role handling See [bigtop 2797](https://issues.apache.org/jira/browse/BIGTOP-2797) for details. Basically, the zeppelin charm should work by itself, with

[GitHub] bigtop pull request #233: BIGTOP-2801: charm race condition when gathering m...

2017-06-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/233 BIGTOP-2801: charm race condition when gathering metrics - only run the metric collector if the charm is ready. - drive by fix for the namenode; 8020 is used intra-cluster and doesn't ne

[GitHub] bigtop pull request #238: BIGTOP-2821: expose extra config options for spark

2017-06-23 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/238 BIGTOP-2821: expose extra config options for spark Let users configure extra library paths and driver/executor memory by exposing these options in puppet and using them in `spark-defaults.conf

[GitHub] bigtop pull request #239: BIGTOP-2282: spark charm: leverage puppet config, ...

2017-06-23 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/239 BIGTOP-2282: spark charm: leverage puppet config, gpu enablement - Document new install-cuda config option for installing cuda on capable hardware. - Leverage library and mem config options

[GitHub] bigtop pull request #241: BIGTOP-2827: juju bundle refresh (june 2017)

2017-06-29 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/241 BIGTOP-2827: juju bundle refresh (june 2017) Refresh Juju bundles. See [BIGTOP-2827](https://issues.apache.org/jira/browse/BIGTOP-2827) for details. You can merge this pull request into a Git

[GitHub] bigtop issue #244: BIGTOP-2829. [iTest] build failed during Maven integratio...

2017-07-03 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/244 +1, LGTM. Fwiw, i was getting the same error (failsafe-summary.xml: no such file) with v2.17, 2.18, and 2.18.1. I didn't dig too deep, but this fits the timeframe and seems rel

[GitHub] bigtop pull request #246: BIGTOP-2834: spark charm: refactor for restricted ...

2017-07-06 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/246 BIGTOP-2834: spark charm: refactor for restricted networks; lib cleanup See [BIGTOP-2834](https://issues.apache.org/jira/browse/BIGTOP-2834) for details. - Make the pagerank sample data a

[GitHub] bigtop pull request #251: BIGTOP-2835: puppet fails when bigtop::jdk_preinst...

2017-07-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/251 BIGTOP-2835: puppet fails when bigtop::jdk_preinstalled is true Many modules require `Package["jdk"]`. Ensure it is available regardless of the `bigtop::jdk_preinstalled` option. Modi

[GitHub] bigtop pull request #252: BIGTOP-2836: charm metric collector race condition

2017-07-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/252 BIGTOP-2836: charm metric collector race condition Ensure `echo 0` is the last thing to run so that the metric hook does not cause a failed deployment. Works in all tested scenarios

[GitHub] bigtop issue #244: BIGTOP-2829. [iTest] build failed during Maven integratio...

2017-07-07 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/244 Hey @evans-ye, I see this has been committed upstream. Just FYI, if you included a "Closes #244" in the upstream commit message, the github bot would have automatically clos

[GitHub] bigtop issue #251: BIGTOP-2835: puppet fails when bigtop::jdk_preinstalled i...

2017-07-08 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/251 @c0s, thanks for taking a look! FYI, hadoop (various classes), zookeeper, and zeppelin will fail to deploy with `jdk_preinstalled: True` without this commit. They require `Package[jdk

[GitHub] bigtop pull request #257: BIGTOP-2844: zeppelin charm: spark config should m...

2017-07-14 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/257 BIGTOP-2844: zeppelin charm: spark config should match zeppelin spark config See [BIGTOP-2844](https://issues.apache.org/jira/browse/BIGTOP-2844) for details. You can merge this pull request into

  1   2   >