[GitHub] spark pull request: [SPARK-4137] [EC2] Don't change working dir on...

2014-11-05 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2988#discussion_r19923159 --- Diff: ec2/spark_ec2.py --- @@ -718,12 +726,16 @@ def get_num_disks(instance_type): return 1 -# Deploy the configuration file

[GitHub] spark pull request: [SPARK-4137] [EC2] Don't change working dir on...

2014-11-05 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2988#discussion_r19922788 --- Diff: ec2/spark_ec2.py --- @@ -718,12 +726,16 @@ def get_num_disks(instance_type): return 1 -# Deploy the configuration file

[GitHub] spark pull request: [EC2] Factor out Mesos spark-ec2 branch

2014-11-03 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3008#issuecomment-61511143 Word. Thanks @shivaram. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [EC2] Factor out Mesos spark-ec2 branch

2014-11-03 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3008#issuecomment-61509758 cc @JoshRosen @pwendell for final review / merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-4137] [EC2] Don't change working dir on...

2014-11-03 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2988#discussion_r19746328 --- Diff: ec2/spark_ec2.py --- @@ -718,12 +726,16 @@ def get_num_disks(instance_type): return 1 -# Deploy the configuration file

[GitHub] spark pull request: [SPARK-4137] [EC2] Don't change working dir on...

2014-10-29 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2988#discussion_r19587131 --- Diff: ec2/spark_ec2.py --- @@ -718,12 +726,16 @@ def get_num_disks(instance_type): return 1 -# Deploy the configuration file

[GitHub] spark pull request: [SPARK-4137] [EC2] Don't change working dir on...

2014-10-29 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2988#issuecomment-61041277 @shivaram I took your suggestion and tested to make sure `spark-ec2` still creates a functioning EC2 cluster. This is ready for another review. --- If your

[GitHub] spark pull request: [EC2] Factor out Mesos spark-ec2 branch

2014-10-29 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3008#issuecomment-61041154 cc @shivaram @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [EC2] Factor out Mesos spark-ec2 branch

2014-10-29 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3008#issuecomment-61033954 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [EC2] Factor out Mesos spark-ec2 branch

2014-10-29 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/3008 [EC2] Factor out Mesos spark-ec2 branch We reference a specific branch in two places. This patch makes it one place. You can merge this pull request into a Git repository by running: $ git

[GitHub] spark pull request: [SPARK-4137] [EC2] Don't change working dir on...

2014-10-29 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2988#issuecomment-61023615 So basically you're saying go with option 2, right? From what I can see, `deploy.generic` may be the only file we need to fix the path for. Is that right?

[GitHub] spark pull request: [SPARK-4137] [EC2] Don't change working dir on...

2014-10-29 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2988#issuecomment-61016125 Actually, we have a few options here for `spark-ec2` and the underlying `spark_ec2.py` script. 1. We leave the script as-is and just document the fact that

[GitHub] spark pull request: [SPARK-4137] [EC2] Don't change working dir on...

2014-10-29 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2988#issuecomment-61013749 @shivaram Oh. Glad I pinged you. :) In my brief testing I didn't allow `launch` to go all the way through. This is obviously broken. Will update

[GitHub] spark pull request: [SPARK-4137] [EC2] Don't change working dir on...

2014-10-29 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2988#issuecomment-61007294 cc @shivaram @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [EC2] Don't change working dir on user

2014-10-28 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2988#issuecomment-60874638 Phantom new classes _again_? :angry: I'll look into this tomorrow. Thought it was a resolved issue... --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [EC2] Don't change working dir on user

2014-10-28 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/2988 [EC2] Don't change working dir on user The issue was uncovered after [this discussion](https://issues.apache.org/jira/browse/SPARK-3398?focusedCommentId=14187471

[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...

2014-10-19 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2845#discussion_r19064710 --- Diff: dev/run-tests-jenkins --- @@ -92,12 +92,39 @@ function post_message () { echo " > api_response: ${api_response}" >&a

[GitHub] spark pull request: [SPARK-3873] [build] Add style checker to enfo...

2014-10-11 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2757#issuecomment-58769459 Wow, there are a lot of changes in this PR. Perhaps this was the most ignored style rule, eh? cc @pwendell --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-3854] Scala style: require spaces befor...

2014-10-11 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2761#issuecomment-58769389 > I quite like standardizing style, but doesn't this have the same problem mentioned before, that it's going to break a lot of potential merge commits?

[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...

2014-10-09 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2716#issuecomment-58579969 > Most of code in JsonRDD are inferring types and converting objects into inferred types, these two can only be done in Python Ah yeah, I'm assum

[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...

2014-10-09 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2716#issuecomment-58561205 > Perhaps we can extract the schema merging code from JSON and use it here as well after inferring the per row schema in python. That sounds good to me! Sch

[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...

2014-10-08 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2716#issuecomment-58437772 @davies I believe this PR also relates to the features discussed in [SPARK-2870](https://issues.apache.org/jira/browse/SPARK-2870). Since you are already doing schema

[GitHub] spark pull request: [SPARK-3412] [PySpark] Replace Epydoc with Sph...

2014-10-07 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2689#issuecomment-58290191 This looks much cleaner and more modern. Nice work. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3758] [Windows] Wrong EOL character in ...

2014-10-07 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2612#discussion_r18541644 --- Diff: dev/lint-windows-cmd --- @@ -0,0 +1,40 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-3758] [Windows] Wrong EOL character in ...

2014-10-07 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2612#discussion_r18540369 --- Diff: dev/run-tests --- @@ -130,7 +130,11 @@ echo "=" CUR

[GitHub] spark pull request: [SPARK-3758] [Windows] Wrong EOL character in ...

2014-10-07 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2612#discussion_r18530699 --- Diff: dev/lint-windows-cmd --- @@ -0,0 +1,40 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-3758] [Windows] Wrong EOL character in ...

2014-10-07 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2612#discussion_r18530666 --- Diff: dev/lint-windows-cmd --- @@ -0,0 +1,40 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-3758] [Windows] Wrong EOL character in ...

2014-10-07 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2612#discussion_r18530534 --- Diff: dev/lint-windows-cmd --- @@ -0,0 +1,40 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-10-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-58101817 BTW, related side note: I just used Packer to create an example image from a template, and it looks like Packer follows a similar pattern of first waiting for the

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-10-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-58098032 @pwendell Is this good to merge in? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-10-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-58097924 @pwendell @JoshRosen Is this PR ready to be merged in? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-10-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-58047057 > Why was this error happened in jenkins? Is this because of many commits? No, I believe this is related to Jenkins being hosted on some servers with spo

[GitHub] spark pull request: [SPARK-3758] [Windows] Wrong EOL character in ...

2014-10-04 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2612#issuecomment-57909026 > I'm worried since most of us develop on mac/linux that this will end up in a weird state where there are mixed EOL characters. I'd worry about

[GitHub] spark pull request: [SPARK-3218, SPARK-3219, SPARK-3261, SPARK-342...

2014-10-02 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2419#issuecomment-57679766 @derrickburns Side note: It looks like a lot of [extraneous commits](https://github.com/apache/spark/pull/2419#commits-pushed-471d5f2) got pulled into this PR by

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-57566795 > This patch adds the following public classes (experimental): println(s"Failed to load main class $childMainClass.") FYI: I believe I have thes

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-10-01 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2606#discussion_r18318253 --- Diff: dev/run-tests-jenkins --- @@ -84,42 +98,46 @@ function post_message () { fi } + +# We diff master...$ghprbActualCommit

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-10-01 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2606#discussion_r18318239 --- Diff: dev/run-tests-jenkins --- @@ -84,42 +98,46 @@ function post_message () { fi } + +# We diff master...$ghprbActualCommit

[GitHub] spark pull request: [SPARK-3479] [Build] [WIP] Report failed test ...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57565606 cc @pwendell This PR is ready for review. Here are examples of the messages posted when: * [all tests pass](https://github.com/apache/spark/pull/2606

[GitHub] spark pull request: [SQL] Kill dangerous trailing space in query s...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2619#issuecomment-57527475 > (Really should add "no trailing space" to our coding style guidelines!) FWIW, `pep8` is enforcing this rule for us for our Python codebase.

[GitHub] spark pull request: [SPARK-3479] [Build] [WIP] Report failed test ...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57525770 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3479] [Build] [WIP] Report failed test ...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57525681 Another checkout timeout. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3479] [Build] [WIP] Report failed test ...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57522641 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3479] [Build] [WIP] Report failed test ...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57522633 Hmm, I expected a failure in the PySpark doctests, not here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-3479] [Build] [WIP] Report failed test ...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57514556 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3479] [Build] [WIP] Report failed test ...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57514540 _sigh_... more checkout troubles. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-3479] [Build] [WIP] Report failed test ...

2014-10-01 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57509410 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57415010 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57414629 Grr, Jenkins flakiness checking out stuff from git... Oi Jenkins! Retest this please. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57412809 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57411918 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57410391 Jenkins failed to checkout the patch. Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-57409407 Dunno where this most recent failure came from. The build output just shows a failure to checkout the patch. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2606#issuecomment-57394445 I'm gonna post several test commits here to try out the various failure modes and confirm that they are reported correctly. --- If your project is set up for it

[GitHub] spark pull request: [SPARK-3479] [Build] Report failed test catego...

2014-09-30 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/2606 [SPARK-3479] [Build] Report failed test category This PR allows SparkQA (i.e. Jenkins) to report in its posts to GitHub what category of test failed, if any. The failure categories are

[GitHub] spark pull request: [Build] Post commit hash with timeout messages

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2597#issuecomment-57377003 cc @rxin [Here is an example](https://github.com/apache/spark/pull/2597#issuecomment-57363092) of the new timeout message. --- If your project is set up for

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-57365013 BTW @shivaram I took your suggestion of [checking the status checks](https://github.com/apache/spark/pull/2339/files#diff-ada66bbeb2f1327b508232ef6c3805a5R656) before

[GitHub] spark pull request: [Build] Post commit hash with timeout messages

2014-09-30 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/2597 [Build] Post commit hash with timeout messages [By request](https://github.com/apache/spark/pull/2588#issuecomment-57266871), and because it also makes sense. You can merge this pull request

[GitHub] spark pull request: [SPARK-3709] Executors don't always report bro...

2014-09-30 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2588#issuecomment-57285986 > @nchammas - would you be interested in submitting a pr to change the qa script so that the timeout and failure message already prints the commit hash? @r

[GitHub] spark pull request: Minor cleanup of code.

2014-09-29 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2581#issuecomment-57215864 > Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuil

[GitHub] spark pull request: [EC2] Sort long, manually-inputted dictionarie...

2014-09-29 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/2578 [EC2] Sort long, manually-inputted dictionaries Similar to the work done in #2571, this PR just sorts the remaining manually-inputted dicts in the EC2 script so they are easier to maintain. You

[GitHub] spark pull request: [EC2] Sort long, manually-inputted dictionarie...

2014-09-29 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2578#issuecomment-57182868 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [EC2] Cleanup Python parens and disk dict

2014-09-28 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2571#issuecomment-57114433 cc @rxin @pwendell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [EC2] Cleanup Python parens and disk dict

2014-09-28 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2571#discussion_r18136353 --- Diff: ec2/spark_ec2.py --- @@ -820,18 +820,18 @@ def ssh_read(host, opts, command): ssh_command(opts) + ['%s@%s' % (opts.

[GitHub] spark pull request: [EC2] Cleanup Python parens and disk dict

2014-09-28 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2571#discussion_r18136340 --- Diff: ec2/spark_ec2.py --- @@ -626,43 +626,43 @@ def wait_for_cluster(conn, wait_secs, master_nodes, slave_nodes): def get_num_disks(instance_type

[GitHub] spark pull request: [EC2] Cleanup Python parens and disk dict

2014-09-28 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2571#discussion_r18136317 --- Diff: ec2/spark_ec2.py --- @@ -508,7 +508,7 @@ def tag_instance(instance, name): break except: print

[GitHub] spark pull request: [EC2] Cleanup Python parens and disk dict

2014-09-28 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/2571 [EC2] Cleanup Python parens and disk dict Minor fixes: * Remove unnecessary parens (Python style) * Sort `disks_by_instance` dict and remove duplicate `t1.micro` key You can merge this

[GitHub] spark pull request: [SPARK-3543] Clean up Java TaskContext impleme...

2014-09-27 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2557#issuecomment-57067407 @rxin The [block that sets this message](https://github.com/apache/spark/blob/5b922bb458e863f5be0ae68167de882743f70b86/dev/run-tests-jenkins#L89) is driven by an

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-26 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-57003193 Hey @pwendell, is there anything else you'd like to change about this PR? --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-25 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-56905825 > I might remove some of the extra linbreaks/whitespace here. Ah, OK, I just took care of that. An earlier version of that text would've put the line

[GitHub] spark pull request: [Build] Diff from branch point

2014-09-25 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2512#issuecomment-56850544 @pwendell Should this also be merged into `branch-1.1`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [Build] Diff from branch point

2014-09-23 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2512#discussion_r17949248 --- Diff: dev/run-tests-jenkins --- @@ -92,13 +92,13 @@ function post_message () { merge_note=" * This patch merges cl

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-56598615 This PR is ready for another review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: diff from branch point

2014-09-23 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/2512 diff from branch point Sometimes Jenkins posts [spurious reports of new classes being added](https://github.com/apache/spark/pull/2339#issuecomment-56570170). I believe this stems from diffing

[GitHub] spark pull request: [Build] Diff from branch point

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2512#issuecomment-56584828 cc @pwendell @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-56571373 > This patch adds the following public classes _(experimental)_: `logInfo("Interrupting user class to stop.")` This shows up because it was [add

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-56568264 > the easiest fix may be for us to completely move the build triggering to our own spark-prs.appspot.com platform Oh, interesting! How would that cha

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-56559355 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-56559339 > Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuil

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-56556454 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3398] [EC2] Have spark-ec2 intelligentl...

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2339#issuecomment-56552890 @pwendell @JoshRosen @shivaram I've updated the `wait_for_cluster_state()` function to check the instance state, then the status checks, and then finally SSH i

[GitHub] spark pull request: SPARK-1143: refactor TaskSchedulerImplSuite

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/339#issuecomment-56526582 > Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20

[GitHub] spark pull request: Merge pull request #1 from apache/master

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2502#issuecomment-56484239 Hi @ceys. What is the purpose of this PR? It looks like it was perhaps a mistake? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: SPARK-1143: refactor TaskSchedulerImplSuite

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/339#issuecomment-56483779 @kayousterhout @CodingCat What is the status of this PR? It hasn't been updated in a while. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: SPARK-1380: Add sort-merge based cogroup/joins...

2014-09-23 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/283#issuecomment-56483665 @pwendell @rxin @mateiz What is the status of this PR? It looks pretty substantial, but it hasn't been updated in a while. --- If your project is set up for it, yo

[GitHub] spark pull request: stop, start and destroy require the EC2_REGION

2014-09-22 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2473#discussion_r17889871 --- Diff: docs/ec2-scripts.md --- @@ -137,11 +146,11 @@ cost you any EC2 cycles, but ***will*** continue to cost money for EBS storage. - To

[GitHub] spark pull request: stop, start and destroy require the EC2_REGION

2014-09-22 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2473#discussion_r17889468 --- Diff: docs/ec2-scripts.md --- @@ -48,6 +48,15 @@ by looking for the "Name" tag of the instance in the Amazon EC2 Console. key pair,

[GitHub] spark pull request: stop, start and destroy require the EC2_REGION

2014-09-22 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/2473#discussion_r17889459 --- Diff: docs/ec2-scripts.md --- @@ -137,11 +146,11 @@ cost you any EC2 cycles, but ***will*** continue to cost money for EBS storage. - To

[GitHub] spark pull request: Update docs to use jsonRDD instead of wrong js...

2014-09-22 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2479#issuecomment-56401607 cc @marmbrus @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3218, SPARK-3219, SPARK-3261, SPARK-342...

2014-09-19 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2419#issuecomment-56248490 FYI, I noticed this towards the beginning of the build output: > fatal: ambiguous argument 'mllib/src/main/scala/org/apache/spark/mllib/cl

[GitHub] spark pull request: [Build] Fix passing of args to sbt

2014-09-19 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2462#issuecomment-56235058 @pwendell I think this is good to merge in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [Build] Fix passing of args to sbt

2014-09-19 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2462#issuecomment-56225026 Great. This latest failure replicates the ones we've been getting in the master build. I'm going to revert 061600c and b2de56c and we should be good to go

[GitHub] spark pull request: [Build] Fix passing of args to sbt

2014-09-19 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2462#issuecomment-56223517 PSA: Don't merge this in until I back out a few hacks for testing! --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: [Build] Fix passing of args to sbt

2014-09-19 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/2462 [Build] Fix passing of args to sbt Simple mistake, simple fix: ```shell args="arg1 arg2 arg3" sbt $args# sbt sees 3 arguments sbt "$args" #

[GitHub] spark pull request: [Build] Test selective testing

2014-09-17 Thread nchammas
Github user nchammas closed the pull request at: https://github.com/apache/spark/pull/2441 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [Build] Test selective testing

2014-09-17 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2441#issuecomment-55992062 > Just add `hive-thriftserver/test` to the sbt command for SQL. Done in #2442. --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark pull request: [SPARK-3534] Add hive-thriftserver to SQL test...

2014-09-17 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/2442 [SPARK-3534] Add hive-thriftserver to SQL tests Addresses the problem pointed out in [this comment](https://github.com/apache/spark/pull/2441#issuecomment-55990116). You can merge this pull

[GitHub] spark pull request: [Build] Test selective testing

2014-09-17 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2441#issuecomment-55990371 > One minor issue is that `hive-thriftserver` is missing from the SQL tests No problem. Just give me an updated list of arguments to pass to `sbt` when you j

[GitHub] spark pull request: [Build] Test selective testing

2014-09-17 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2441#issuecomment-55986600 @marmbrus @pwendell @rxin Here are the Jenkins outputs for when: * [Only SQL files](https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders

[GitHub] spark pull request: [Build] Test selective testing

2014-09-17 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2441#issuecomment-55986032 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3218, SPARK-3219, SPARK-3261, SPARK-342...

2014-09-17 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2419#issuecomment-55983870 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

<    2   3   4   5   6   7   8   9   10   >