[GitHub] bigtop pull request #400: Add Juju storage support to Kafka charm.

2018-10-03 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/400#discussion_r222415197 --- Diff: bigtop-packages/src/charm/kafka/layer-kafka/reactive/kafka.py --- @@ -90,3 +98,39 @@ def serve_client(client, zookeeper

[GitHub] bigtop pull request #400: Add Juju storage support to Kafka charm.

2018-10-03 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/400#discussion_r222411244 --- Diff: bigtop-packages/src/charm/kafka/layer-kafka/lib/charms/layer/bigtop_kafka.py --- @@ -58,10 +60,14 @@ def configure_kafka(self, zk_units

[GitHub] bigtop pull request #400: Add Juju storage support to Kafka charm.

2018-10-03 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/400#discussion_r222415839 --- Diff: bigtop-packages/src/charm/kafka/layer-kafka/reactive/kafka.py --- @@ -38,9 +40,10 @@ def configure_kafka(zk): hookenv.status_set

[GitHub] bigtop pull request #382: Bigtop-3070 support Spark Thrift Server in Charm

2018-08-31 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/382#discussion_r214485628 --- Diff: bigtop-packages/src/charm/spark/layer-spark/reactive/spark.py --- @@ -318,3 +318,10 @@ def client_present(client): def client_should_stop

[GitHub] bigtop pull request #382: Bigtop-3070 support Spark Thrift Server in Charm

2018-08-31 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/382#discussion_r214485587 --- Diff: bigtop-packages/src/charm/spark/layer-spark/reactive/spark.py --- @@ -318,3 +318,10 @@ def client_present(client): def client_should_stop

[GitHub] bigtop pull request #382: Bigtop-3070 support Spark Thrift Server in Charm

2018-08-31 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/382#discussion_r214484656 --- Diff: bigtop-packages/src/charm/spark/layer-spark/lib/charms/layer/bigtop_spark.py --- @@ -400,6 +400,10 @@ def start(self): else

[GitHub] bigtop pull request #382: Bigtop-3070 support Spark Thrift Server in Charm

2018-08-31 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/382#discussion_r214485545 --- Diff: bigtop-packages/src/charm/spark/layer-spark/reactive/spark.py --- @@ -236,7 +235,8 @@ def reconfigure_spark(): # Almost all config

[GitHub] bigtop pull request #382: Bigtop-3070 support Spark Thrift Server in Charm

2018-08-24 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/382#discussion_r212650418 --- Diff: bigtop-packages/src/charm/spark/layer-spark/reactive/spark.py --- @@ -236,7 +235,8 @@ def reconfigure_spark(): # Almost all config

[GitHub] bigtop issue #382: Bigtop-3070 support Spark Thrift Server in Charm

2018-08-23 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/382 One other thing @buggtb, since this opens a new port, we'll need 1 listed in layer.yaml: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/spark/layer-

[GitHub] bigtop pull request #382: Bigtop-3070 support Spark Thrift Server in Charm

2018-08-23 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/382#discussion_r212387730 --- Diff: bigtop-packages/src/charm/spark/layer-spark/reactive/spark.py --- @@ -236,7 +235,8 @@ def reconfigure_spark(): # Almost all config

[GitHub] bigtop pull request #377: BIGTOP-3060: juju bundle refresh (august 2018)

2018-08-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/377 BIGTOP-3060: juju bundle refresh (august 2018) - Bump charm revisions to latest - A few lint fixes in layer-zookeeper You can merge this pull request into a Git repository by running

[GitHub] bigtop pull request #349: BIGTOP-3014: juju: use charm-env for shebangs

2018-03-25 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/349 BIGTOP-3014: juju: use charm-env for shebangs See [BIGTOP-3014](https://issues.apache.org/jira/browse/BIGTOP-3014) for the meats. Lots of files changed here, but it's just the she

[GitHub] bigtop issue #348: BIGTOP-3015: hadoop-spark bundle correction

2018-03-25 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/348 @jamesbeedy, the reason `hadoop-client` was in this bundle was to act as an endpoint for people that were accustomed to running jobs and managing hdfs from there. iow, muscle memory and scripts

[GitHub] bigtop pull request #347: BIGTOP-3013: remove bigtop_version override for ka...

2018-03-12 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/347 BIGTOP-3013: remove bigtop_version override for kafka Since kafka is no longer available in the bigtop repos, we need to install it from a CI repo. This should be a 1.2.1 repo, not master

[GitHub] bigtop issue #341: BIGTOP-3005 Add zkpeer-relation-changed hook to zookeeper...

2018-03-06 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/341 Rev 42 contains this update and has been released to the edge channel in the charm store: https://jujucharms.com/zookeeper/42 Pending CI tests to move it through to stable. ---

[GitHub] bigtop issue #345: BIGTOP-3010: juju bundle refresh (feb 2018)

2018-03-06 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/345 Picked up new changes for hive (#344) and zookeeper (#341). We should be g2g, but I'll let CWR finish on these latest revs before merging upstream. ---

[GitHub] bigtop issue #341: BIGTOP-3005 Add zkpeer-relation-changed hook to zookeeper...

2018-03-06 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/341 +1, thx for the explanation @Pekkari. A quick deploy with this addition lgtm. ---

[GitHub] bigtop issue #341: BIGTOP-3005 Add zkpeer-relation-changed hook to zookeeper...

2018-03-05 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/341 @Pekkari I don't understand what a `zkpeer.changed` handler buys us given that we already run a `check_cluster` when peers join and depart. Am I missing a case where a peer would change th

[GitHub] bigtop pull request #344: BIGTOP-3007: add properties to hive-site.xml for z...

2018-03-04 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/344#discussion_r172070816 --- Diff: bigtop-packages/src/charm/hive/layer-hive/lib/charms/layer/bigtop_hive.py --- @@ -36,31 +36,45 @@ def install(self, hbase=None

[GitHub] bigtop pull request #345: BIGTOP-3010: juju bundle refresh (feb 2018)

2018-03-02 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/345 BIGTOP-3010: juju bundle refresh (feb 2018) Charms were refreshed back when 1.2.1 was released, but the bundle revisions and READMEs in the upstream source were not updated. This PR fixes that

[GitHub] bigtop pull request #344: BIGTOP-3007: add properties to hive-site.xml for z...

2018-03-02 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/344#discussion_r171912690 --- Diff: bigtop-deploy/puppet/hieradata/bigtop/cluster.yaml --- @@ -192,6 +192,9 @@ hadoop_hive::common_config::hbase_zookeeper_quorum: "%{

[GitHub] bigtop pull request #344: BIGTOP-3007: add properties to hive-site.xml for z...

2018-03-02 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/344#discussion_r171910993 --- Diff: bigtop-deploy/puppet/hieradata/bigtop/cluster.yaml --- @@ -192,6 +192,9 @@ hadoop_hive::common_config::hbase_zookeeper_quorum: "%{

[GitHub] bigtop pull request #344: BIGTOP-3007: add properties to hive-site.xml for z...

2018-03-02 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/344#discussion_r171911797 --- Diff: bigtop-packages/src/charm/hive/layer-hive/lib/charms/layer/bigtop_hive.py --- @@ -69,35 +83,44 @@ def configure_hive(self): config

[GitHub] bigtop pull request #344: BIGTOP-3007: add properties to hive-site.xml for z...

2018-03-02 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/344#discussion_r171912318 --- Diff: bigtop-packages/src/charm/hive/layer-hive/reactive/hive.py --- @@ -78,9 +87,13 @@ def install_hive(hadoop): else: hbserver

[GitHub] bigtop pull request #298: BIGTOP-2910: zeppelin charm: support bigtop upgrad...

2017-10-19 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/298 BIGTOP-2910: zeppelin charm: support bigtop upgrade See [BIGTOP-2910](https://issues.apache.org/jira/browse/BIGTOP-2910) for details. Make the zeppelin charm react to `bigtop_version

[GitHub] bigtop pull request #297: BIGTOP-2903: update protobuf rpm links

2017-10-11 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/297 BIGTOP-2903: update protobuf rpm links point to fedora archives for older protobuf ppc64le rpm location You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] bigtop pull request #281: BIGTOP-2874: juju bundle refresh (august 2017)

2017-08-29 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/281 BIGTOP-2874: juju bundle refresh (august 2017) Refresh Juju bundles. See [BIGTOP-2874](https://issues.apache.org/jira/browse/BIGTOP-2874) for details. You can merge this pull request into a Git

[GitHub] bigtop pull request #275: BIGTOP-2867: spark charm: make bigtop version conf...

2017-08-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/275 BIGTOP-2867: spark charm: make bigtop version configurable See [BIGTOP-2867](https://issues.apache.org/jira/browse/BIGTOP-2867) for details on the upgrade support provided by this PR. You can

[GitHub] bigtop issue #251: BIGTOP-2835: puppet fails when bigtop::jdk_preinstalled i...

2017-08-03 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/251 This landed in master with the following commit: https://github.com/apache/bigtop/commit/40e796bd693a8bc6243ee39a6540357e1d559ea8 Closing out this PR. --- If your project is set

[GitHub] bigtop pull request #251: BIGTOP-2835: puppet fails when bigtop::jdk_preinst...

2017-08-03 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/251 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop issue #257: BIGTOP-2844: zeppelin charm: spark config should match ze...

2017-08-03 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/257 This has been working well for a few weeks now. Merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] bigtop pull request #257: BIGTOP-2844: zeppelin charm: spark config should m...

2017-07-14 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/257 BIGTOP-2844: zeppelin charm: spark config should match zeppelin spark config See [BIGTOP-2844](https://issues.apache.org/jira/browse/BIGTOP-2844) for details. You can merge this pull request into

[GitHub] bigtop issue #251: BIGTOP-2835: puppet fails when bigtop::jdk_preinstalled i...

2017-07-08 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/251 @c0s, thanks for taking a look! FYI, hadoop (various classes), zookeeper, and zeppelin will fail to deploy with `jdk_preinstalled: True` without this commit. They require `Package[jdk

[GitHub] bigtop issue #244: BIGTOP-2829. [iTest] build failed during Maven integratio...

2017-07-07 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/244 Hey @evans-ye, I see this has been committed upstream. Just FYI, if you included a "Closes #244" in the upstream commit message, the github bot would have automatically clos

[GitHub] bigtop pull request #252: BIGTOP-2836: charm metric collector race condition

2017-07-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/252 BIGTOP-2836: charm metric collector race condition Ensure `echo 0` is the last thing to run so that the metric hook does not cause a failed deployment. Works in all tested scenarios

[GitHub] bigtop pull request #251: BIGTOP-2835: puppet fails when bigtop::jdk_preinst...

2017-07-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/251 BIGTOP-2835: puppet fails when bigtop::jdk_preinstalled is true Many modules require `Package["jdk"]`. Ensure it is available regardless of the `bigtop::jdk_preinstalled` option. Modi

[GitHub] bigtop pull request #246: BIGTOP-2834: spark charm: refactor for restricted ...

2017-07-06 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/246 BIGTOP-2834: spark charm: refactor for restricted networks; lib cleanup See [BIGTOP-2834](https://issues.apache.org/jira/browse/BIGTOP-2834) for details. - Make the pagerank sample data a

[GitHub] bigtop issue #244: BIGTOP-2829. [iTest] build failed during Maven integratio...

2017-07-03 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/244 +1, LGTM. Fwiw, i was getting the same error (failsafe-summary.xml: no such file) with v2.17, 2.18, and 2.18.1. I didn't dig too deep, but this fits the timeframe and seems rel

[GitHub] bigtop pull request #241: BIGTOP-2827: juju bundle refresh (june 2017)

2017-06-29 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/241 BIGTOP-2827: juju bundle refresh (june 2017) Refresh Juju bundles. See [BIGTOP-2827](https://issues.apache.org/jira/browse/BIGTOP-2827) for details. You can merge this pull request into a Git

[GitHub] bigtop pull request #239: BIGTOP-2282: spark charm: leverage puppet config, ...

2017-06-23 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/239 BIGTOP-2282: spark charm: leverage puppet config, gpu enablement - Document new install-cuda config option for installing cuda on capable hardware. - Leverage library and mem config options

[GitHub] bigtop pull request #238: BIGTOP-2821: expose extra config options for spark

2017-06-23 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/238 BIGTOP-2821: expose extra config options for spark Let users configure extra library paths and driver/executor memory by exposing these options in puppet and using them in `spark-defaults.conf

[GitHub] bigtop pull request #233: BIGTOP-2801: charm race condition when gathering m...

2017-06-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/233 BIGTOP-2801: charm race condition when gathering metrics - only run the metric collector if the charm is ready. - drive by fix for the namenode; 8020 is used intra-cluster and doesn't ne

[GitHub] bigtop pull request #230: BIGTOP-2797: zeppelin charm external role handling

2017-06-04 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/230 BIGTOP-2797: zeppelin charm external role handling See [bigtop 2797](https://issues.apache.org/jira/browse/BIGTOP-2797) for details. Basically, the zeppelin charm should work by itself, with

[GitHub] bigtop issue #229: BIGTOP-2795: spark charm: fix sparkpi and rework start/st...

2017-06-02 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/229 During test, I found the `submit` action to be waaay too complex. I refactored it to use a simpler set of arguments, and then documented this in the readme. While I was in the readme, I

[GitHub] bigtop pull request #229: BIGTOP-2795: spark charm: fix sparkpi and rework s...

2017-06-02 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/229 BIGTOP-2795: spark charm: fix sparkpi and rework start/stop logic See [bigtop-2795](https://issues.apache.org/jira/browse/BIGTOP-2795) for details. - add bundle descriptions to the hadoop

[GitHub] bigtop pull request #228: BIGTOP-2783: new charm icons for zookeeper and zep...

2017-06-01 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/228 BIGTOP-2783: new charm icons for zookeeper and zeppelin The design team at Canonical has refreshed the charm icons for Zookeeper and Zeppelin to match the upstream project logos. Replace the

[GitHub] bigtop pull request #217: BIGTOP-2779: new hive charm

2017-05-25 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/217 BIGTOP-2779: new hive charm note to upstream: squash ~16 before commit Key notes: - new hive charm - readme: be like the other bigtop charms - readme: describe metastore/hbase

[GitHub] bigtop pull request #216: BIGTOP-2777: make hbase charm more robust

2017-05-23 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/216 BIGTOP-2777: make hbase charm more robust See [bigtop-2777](https://issues.apache.org/jira/browse/BIGTOP-2777) for details: - fix port numbers - expose heap and thrift config

[GitHub] bigtop pull request #211: BIGTOP-2770: Juju charm/bundle refresh

2017-05-16 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/211 BIGTOP-2770: Juju charm/bundle refresh See [BIGTOP-2770](https://issues.apache.org/jira/browse/BIGTOP-2770) for details. You can merge this pull request into a Git repository by running

[GitHub] bigtop pull request #205: BIGTOP-2764: deployment failure when roles include...

2017-05-16 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/205 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop issue #205: BIGTOP-2764: deployment failure when roles include spark:...

2017-05-16 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/205 Closed by https://github.com/apache/bigtop/commit/73864a30b29ee89655772ce82b4041c2564d0711 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] bigtop pull request #208: BIGTOP-2766. [Puppet] Spark worker startup failed ...

2017-05-12 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/208#discussion_r116296932 --- Diff: bigtop-deploy/puppet/modules/spark/templates/spark-defaults.conf --- @@ -13,7 +13,15 @@ # See the License for the specific language

[GitHub] bigtop issue #205: BIGTOP-2764: deployment failure when roles include spark:...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/205 [spark-39](https://jujucharms.com/spark/39) contains this fix, and is green across clouds for hadoop + spark in yarn mode: http://bigtop.charm.qa/cwr_bundle_hadoop_spark/33/report.html

[GitHub] bigtop issue #205: BIGTOP-2764: deployment failure when roles include spark:...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/205 I've updated the original workaround to include `spark::datanucleus` instead of redefining the package. This is much cleaner than including multiple comment blocks to explain the 2268

[GitHub] bigtop pull request #208: BIGTOP-2766. [Puppet] Spark worker startup failed ...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/208#discussion_r116045104 --- Diff: bigtop-deploy/puppet/modules/spark/templates/spark-defaults.conf --- @@ -13,7 +13,15 @@ # See the License for the specific language

[GitHub] bigtop issue #208: BIGTOP-2766. [Puppet] Spark worker startup failed due to ...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/208 @evans-ye while you're updating spark's init.pp, would you consider replacing the `spark-datanucleus` definition with an `import spark::datanucleus`? https://github.com/evans

[GitHub] bigtop pull request #208: BIGTOP-2766. [Puppet] Spark worker startup failed ...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/208#discussion_r116002658 --- Diff: bigtop-deploy/puppet/manifests/cluster.pp --- @@ -72,6 +72,10 @@ master => ["spark-master"], worker =>

[GitHub] bigtop pull request #208: BIGTOP-2766. [Puppet] Spark worker startup failed ...

2017-05-11 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/208#discussion_r116011550 --- Diff: bigtop-deploy/puppet/modules/spark/templates/spark-defaults.conf --- @@ -13,7 +13,15 @@ # See the License for the specific language

[GitHub] bigtop pull request #206: BIGTOP-2765: update roles logic in spark/zeppelin

2017-05-10 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/206 BIGTOP-2765: update roles logic in spark/zeppelin Rework role logic for spark/zeppelin charms. - spark: only include master/worker when running in standalone mode - spark: include

[GitHub] bigtop pull request #205: BIGTOP-2764: deployment failure when roles include...

2017-05-10 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/205 BIGTOP-2764: deployment failure when roles include spark::common and spark::yarn* We need to ensure the `spark-datanucleus` package is only defined once. There's a hack in place so it is a

[GitHub] bigtop issue #196: BIGTOP-2740: hbase 1.1.3 does not work on ppc64le

2017-05-05 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/196 Closed in https://github.com/apache/bigtop/commit/905bfe464864c8d77393b6dcf6788f3866e887fa --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] bigtop pull request #196: BIGTOP-2740: hbase 1.1.3 does not work on ppc64le

2017-05-05 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/196 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-24 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/197#discussion_r112990603 --- Diff: bigtop-packages/src/charm/spark/layer-spark/actions/restart-spark-job-history-server --- @@ -15,22 +15,22 @@ # limitations under the

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-24 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/197#discussion_r112990094 --- Diff: bigtop-packages/src/charm/pig/layer-pig/actions/smoke-test --- @@ -63,8 +62,8 @@ else: # The smoke tests analyze /etc/passwd

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-24 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/197#discussion_r112989226 --- Diff: bigtop-packages/src/charm/hadoop/layer-hadoop-resourcemanager/actions/smoke-test --- @@ -43,6 +43,6 @@ smoke_env = { bigtop = Bigtop

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-24 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/197#discussion_r112987956 --- Diff: bigtop-deploy/juju/hadoop-spark/bundle.yaml --- @@ -29,21 +29,21 @@ services: - "2" - "3" pl

[GitHub] bigtop pull request #197: BIGTOP-2747: new charm revs for bigtop-1.2

2017-04-21 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/197 BIGTOP-2747: new charm revs for bigtop-1.2 Bring charm/bundle source up to date with bigtop-1.2. This is mostly a README/metadata refactor with sprinkles of a more consistent UX for charm

[GitHub] bigtop pull request #196: BIGTOP-2740: hbase 1.1.3 does not work on ppc64le

2017-04-17 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/196 BIGTOP-2740: hbase 1.1.3 does not work on ppc64le HBase 1.1.x was broken on non-Intel arches until v1.1.4 ([HBASE-15322](https://issues.apache.org/jira/browse/HBASE-15322)). Pick up this

[GitHub] bigtop pull request #195: BIGTOP-2739: refresh juju bundles with latest char...

2017-04-12 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/195 BIGTOP-2739: refresh juju bundles with latest charm revs See [jira](https://issues.apache.org/jira/browse/BIGTOP-2739) for details. Notables: - add machine constraints back to workaround

[GitHub] bigtop pull request #194: BIGTOP-2737: Spark charm doesn't handle HA or exam...

2017-04-10 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/194 BIGTOP-2737: Spark charm doesn't handle HA or examples well See [jira](https://issues.apache.org/jira/browse/BIGTOP-2737) for details, but in general, the spark charm didn't transition

[GitHub] bigtop pull request #189: BIGTOP-2712: Juju CI driven updates

2017-03-23 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/189 BIGTOP-2712: Juju CI driven updates The original point of this branch was to refresh the hbase charm (readme was out-of-date, needed lint love). However, as soon as ran CI on this and recent

[GitHub] bigtop pull request #187: BIGTOP-2703: refresh juju charms/bundles with metr...

2017-03-20 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/187 BIGTOP-2703: refresh juju charms/bundles with metric and CI support While this looks like a large PR, the majority is simply bumping charm revisions/locations in yaml files and updating READMEs to

[GitHub] bigtop pull request #178: BIGTOP-2682: update juju bundles with recent charm...

2017-02-03 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/178 BIGTOP-2682: update juju bundles with recent charm revs You can merge this pull request into a Git repository by running: $ git pull https://github.com/juju-solutions/bigtop bug/BIGTOP-2682

[GitHub] bigtop issue #163: BIGTOP-2603: NN/RM charm should include a spark user/grou...

2016-12-05 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/163 Yeah, we probably should. `oozie` and `hue` come to mind. We chatted about this a little while ago. It would really be nice if the charm that relied on `hadoop-plugin` would send user info over

[GitHub] bigtop issue #161: BIGTOP-2586: hadoop-processing tests.yaml should include ...

2016-12-02 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/161 This went in with PR #167 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] bigtop pull request #168: BIGTOP-2617: refresh juju spark-processing bundle

2016-12-02 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/168 BIGTOP-2617: refresh juju spark-processing bundle Similar to BIGTOP-2616, we've made improvements to the spark-processing bundle that are ready for review and inclusion upstream: - u

[GitHub] bigtop pull request #167: BIGTOP-2616: refresh juju hadoop-processing bundle

2016-12-02 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/167 BIGTOP-2616: refresh juju hadoop-processing bundle We've made a number of improvements to the hadoop-processing bundle that are ready for review and inclusion upstream: - ganglia/rsyslog

[GitHub] bigtop pull request #166: BIGTOP-2561: add juju bundle for hadoop-spark

2016-12-02 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/166 BIGTOP-2561: add juju bundle for hadoop-spark This combines hadoop-processing and spark-processing into a single bundle: https://jujucharms.com/hadoop-spark/ Test results are

[GitHub] bigtop pull request #163: BIGTOP-2603: NN/RM charm should include a spark us...

2016-11-22 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/163 BIGTOP-2603: NN/RM charm should include a spark user/group When NN/RM are not colocated with spark, they do not know about the spark user/group. When in `yarn-client` mode, the spark charm will

[GitHub] bigtop issue #162: BIGTOP-2570: ease hadoop charm debugging

2016-11-18 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/162 Though this started with a systemctl service that failed to start, it has turned into a crusade to make charm actions/status/logging better so we can more easily debug problems like this in the

[GitHub] bigtop pull request #162: BIGTOP-2570: ensure bigtop services are started

2016-11-16 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/162 BIGTOP-2570: ensure bigtop services are started We need to make sure our bigtop services are actually started before setting the `.started` state. Without this fix, our NN will set

[GitHub] bigtop issue #157: BIGTOP-2577: kafka charm test updates

2016-11-10 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/157 Good catch in zookeeper -- you suggested the better approach of relaxing the charm definition in the test.. Most recent commit should address your concerns. Thanks @petevg! --- If your project

[GitHub] bigtop pull request #156: BIGTOP-2575: zk charm test updates

2016-11-10 Thread kwmonroe
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/156#discussion_r87508997 --- Diff: bigtop-packages/src/charm/zookeeper/layer-zookeeper/tests/10-bind-address.py --- @@ -34,16 +33,16 @@ class TestBindClientPort(unittest.TestCase

[GitHub] bigtop pull request #157: BIGTOP-2577: kafka charm test updates

2016-11-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/157 BIGTOP-2577: kafka charm test updates Similar updates to what we did for the zk charm (BIGTOP-2575). Update the Juju charm for Kafka: - update readme with language consistent with other

[GitHub] bigtop pull request #156: BIGTOP-2575: zk charm test updates

2016-11-07 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/156 BIGTOP-2575: zk charm test updates Update the Juju charm for Zookeeper: - ensure we deploy correct zk charm and watch for correct status during tests - use new syntax for executing

[GitHub] bigtop issue #154: BIGTOP-2560: Spark charm failing automated tests

2016-11-04 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/154 @ktsakalozos i feel like i hijacked this PR from you, but I can assure you it was for the greater good! My recent commits have come as a result of watching cwr for the spark-processing bundle

[GitHub] bigtop pull request #155: BIGTOP-2571: cwr driven updates to the zeppelin ju...

2016-11-04 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/155 BIGTOP-2571: cwr driven updates to the zeppelin juju charm Simplify our smoke test (it's must simpler and faster to smoke zeppelin/spark than it is to smoke zeppelin/hadoop).

[GitHub] bigtop issue #150: BIGTOP-2550: Update juju hadoop bundle for Juju 2.0 and X...

2016-10-25 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/150 Thanks all. I've refreshed the charm revision numbers to pick up the recent bind-host options (bigtop-2554) and fix an issue with terasorting on lxd. --- If your project is set up for it, yo

[GitHub] bigtop issue #153: BIGTOP-2555: hadoop charms should use bind-host overrides

2016-10-25 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/153 Charms in ~bigdata-dev namespace have been refreshed and verified that terasort works on lxd again: ``` results: meta: composite: direction: asc units: secs

[GitHub] bigtop issue #153: BIGTOP-2555: hadoop charms should use bind-host overrides

2016-10-25 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/153 Thanks for the eyeballs @ktsakalozos and @johnsca! I missed a mapred.jobhistory binding. You will no longer see any 127.0.x.y bindings. Also, thanks @johnsca for pointing out the terasort

[GitHub] bigtop pull request #153: BIGTOP-2555: hadoop charms should use bind-host ov...

2016-10-21 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/153 BIGTOP-2555: hadoop charms should use bind-host overrides - remove unused import - Bind NN and RM to 0.0.0.0 (all interfaces) using the hiera options from BIGTOP-2554. This fixes a problem

[GitHub] bigtop pull request #152: BIGTOP-2554: expose bind-host options in hieradata

2016-10-19 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/152 BIGTOP-2554: expose bind-host options in hieradata Expose bind-host for RM and NN (rpc, http, and https) as hieradata options so we can set the appropriate config xml changes. You can merge this

[GitHub] bigtop issue #151: BIGTOP-2553: add rpc/http/https ports to template for non...

2016-10-19 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/151 Pushed upstream with: https://github.com/apache/bigtop/commit/cd55c144955e36188640c199b3100fefb3b28612 I forgot to add a "closes #151" to the commit, so i'm closi

[GitHub] bigtop pull request #151: BIGTOP-2553: add rpc/http/https ports to template ...

2016-10-19 Thread kwmonroe
Github user kwmonroe closed the pull request at: https://github.com/apache/bigtop/pull/151 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] bigtop issue #151: BIGTOP-2553: add rpc/http/https ports to template for non...

2016-10-18 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/151 I think you're saying the `namenode_hosts[host, index]` loop is the only difference, but that's a pretty big difference. The hdfs-site.xml property keys are not the same in HA and no

[GitHub] bigtop issue #151: BIGTOP-2553: add rpc/http/https ports to template for non...

2016-10-17 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/151 @johnsca, `namenode_hosts` has only 1 item in non-HA mode. Note that once we hit the `else` to enter the "non HA" condition, we're not in any loop -- `namenode_hosts[0]` is how t

[GitHub] bigtop pull request #151: BIGTOP-2553: add rpc/http/https ports to template ...

2016-10-17 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/151 BIGTOP-2553: add rpc/http/https ports to template for non-HA mode Honor the `[namenode|http|https]_port` settings in non-HA mode. You can merge this pull request into a Git repository by running

[GitHub] bigtop issue #149: BIGTOP-2550: Update juju hadoop bundle for Juju 2.0 and X...

2016-10-12 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/149 Duped PR. See #150. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] bigtop pull request #150: BIGTOP-2550: Update juju hadoop bundle for Juju 2....

2016-10-12 Thread kwmonroe
GitHub user kwmonroe opened a pull request: https://github.com/apache/bigtop/pull/150 BIGTOP-2550: Update juju hadoop bundle for Juju 2.0 and Xenial - Update README with xenial and juju2 instructions. Callout juju1 equivalent commands where applicable. - Colocate apps that can

[GitHub] bigtop issue #149: BIGTOP-2550: Update juju hadoop bundle for Juju 2.0 and X...

2016-10-12 Thread kwmonroe
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/149 ASF gitbot didn't link this PR to the jira. I think that's because I didn't title it correctly at the onset. Closing out to make a new one. --- If your project is set up for it, y

  1   2   >