Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73470437
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73470429
[Test build #27089 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27089/consoleFull)
for PR 4385 at commit
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73566658
Thanks @florianverhein - I just tested this locally and it worked fine.
Lets wait for @nchammas to sign off and we can merge it after that (also
Jenkins passed !)
---
Github user florianverhein commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73615976
Thanks. No problem.
I also have [SPARK-5641] ready to go once this is merged.
---
If your project is set up for it, you can reply to this email and have
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73616133
Looks like 3 users in agreement and to my moderately trained eye looks good
too. Jenkins says aye, so let's merge it
---
If your project is set up for it, you can reply
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4385
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user florianverhein commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73611158
Great. Thanks for that @shivaram.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user nchammas commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73612751
LGTM. Thanks for working on this @florianverhein.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user nchammas commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73390259
I agree we don't want to put much effort into this feature since the use
case isn't standard.
Maybe instead we can force the repo to be named `spark-ec2` by
Github user florianverhein commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73332095
Thanks @nchammas. I've tested only with a fork named `spark-ec2`. I relied
on your comment/request re cloning from a different name (though I agree
there's no
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24278389
--- Diff: ec2/spark_ec2.py ---
@@ -1007,6 +1023,11 @@ def real_main():
print stderr, ebs-vol-num cannot be greater than 8
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73409569
[Test build #27036 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27036/consoleFull)
for PR 4385 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73409574
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73406716
[Test build #27036 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27036/consoleFull)
for PR 4385 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73464449
[Test build #27089 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27089/consoleFull)
for PR 4385 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73460952
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73460944
[Test build #27079 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27079/consoleFull)
for PR 4385 at commit
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24311076
--- Diff: ec2/spark_ec2.py ---
@@ -1026,6 +1042,17 @@ def real_main():
print stderr, ebs-vol-num cannot be greater than 8
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24311190
--- Diff: ec2/spark_ec2.py ---
@@ -1026,6 +1042,17 @@ def real_main():
print stderr, ebs-vol-num cannot be greater than 8
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-7344
There may be flakiness in a test or in Jenkins. I think you can ignore
failures from outside Python for now. The CI says it doesn't add classes, good.
But the bigger
Github user florianverhein commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73442070
@srowen any feedback/tips about this?
I expect the merge issue, as there's been a commit to this file since my
last commit on this fork.
When I run
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24308635
--- Diff: ec2/spark_ec2.py ---
@@ -643,12 +654,14 @@ def setup_cluster(conn, master_nodes, slave_nodes,
opts, deploy_ssh_key):
# NOTE:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73457137
[Test build #27079 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27079/consoleFull)
for PR 4385 at commit
Github user florianverhein commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73457613
@srowen @nchammas Ok, I've rebased on top of current master and added all
requested changes as best I understand them (also minor change to a log line to
as per
Github user florianverhein commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73442633
@srowen The proposed change is definitely worth making. There was a small
extension/generalisation suggested/requested by @nchammas, which as you saw we
were
Github user florianverhein commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73443768
@nchammas I can revert the input check back to what I had earlier to
enforce `spark-ec2` only. We can remove the check later if needed.
Given the help string
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73443784
OK, rebase your branch and push any other changes to it, and let's see that
the python style checks and tests pass, and hopefully Jenkins does too. If it
looks good and
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24264051
--- Diff: ec2/spark_ec2.py ---
@@ -1007,6 +1023,11 @@ def real_main():
print stderr, ebs-vol-num cannot be greater than 8
Github user nchammas commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73295741
@florianverhein This is looking good. Have you tested this against a fork
named `spark-ec2` as well as a fork named something else?
---
If your project is set up for
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24221789
--- Diff: ec2/spark_ec2.py ---
@@ -145,6 +145,14 @@ def parse_args():
default=DEFAULT_SPARK_GITHUB_REPO,
help=Github repo
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73185820
Thanks @florianverhein for the change - This is a pretty useful change as I
often modify these variables inline for my experiments.
@nchammas @JoshRosen could
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24224925
--- Diff: ec2/spark_ec2.py ---
@@ -1007,6 +1022,14 @@ def real_main():
print stderr, ebs-vol-num cannot be greater than 8
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24222907
--- Diff: ec2/spark_ec2.py ---
@@ -643,12 +654,14 @@ def setup_cluster(conn, master_nodes, slave_nodes,
opts, deploy_ssh_key):
# NOTE:
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24222849
--- Diff: ec2/spark_ec2.py ---
@@ -643,12 +654,14 @@ def setup_cluster(conn, master_nodes, slave_nodes,
opts, deploy_ssh_key):
# NOTE:
Github user florianverhein commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73185215
Thanks for prompt feedback @nchammas. Much appreciated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24224602
--- Diff: ec2/spark_ec2.py ---
@@ -1007,6 +1022,14 @@ def real_main():
print stderr, ebs-vol-num cannot be greater than 8
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24207640
--- Diff: ec2/spark_ec2.py ---
@@ -145,6 +145,14 @@ def parse_args():
default=DEFAULT_SPARK_GITHUB_REPO,
help=Github repo from
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24208073
--- Diff: ec2/spark_ec2.py ---
@@ -643,12 +654,14 @@ def setup_cluster(conn, master_nodes, slave_nodes,
opts, deploy_ssh_key):
# NOTE: We
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24207944
--- Diff: ec2/spark_ec2.py ---
@@ -330,7 +338,10 @@ def get_spark_ami(opts):
print stderr,\
Don't recognize %s, assuming type
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24208401
--- Diff: ec2/spark_ec2.py ---
@@ -643,12 +654,14 @@ def setup_cluster(conn, master_nodes, slave_nodes,
opts, deploy_ssh_key):
# NOTE: We
Github user nchammas commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-73156805
Thank @florianverhein for submitting this. It should be useful. I know I've
manually edited `spark_ec2.py` several times to point it to different places.
cc
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24218617
--- Diff: ec2/spark_ec2.py ---
@@ -145,6 +145,14 @@ def parse_args():
default=DEFAULT_SPARK_GITHUB_REPO,
help=Github repo
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24218680
--- Diff: ec2/spark_ec2.py ---
@@ -330,7 +338,10 @@ def get_spark_ami(opts):
print stderr,\
Don't recognize %s, assuming
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24218731
--- Diff: ec2/spark_ec2.py ---
@@ -643,12 +654,14 @@ def setup_cluster(conn, master_nodes, slave_nodes,
opts, deploy_ssh_key):
# NOTE:
Github user florianverhein commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24218853
--- Diff: ec2/spark_ec2.py ---
@@ -643,12 +654,14 @@ def setup_cluster(conn, master_nodes, slave_nodes,
opts, deploy_ssh_key):
# NOTE:
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24219625
--- Diff: ec2/spark_ec2.py ---
@@ -145,6 +145,14 @@ def parse_args():
default=DEFAULT_SPARK_GITHUB_REPO,
help=Github repo from
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24219572
--- Diff: ec2/spark_ec2.py ---
@@ -145,6 +145,14 @@ def parse_args():
default=DEFAULT_SPARK_GITHUB_REPO,
help=Github repo from
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24219713
--- Diff: ec2/spark_ec2.py ---
@@ -330,7 +338,10 @@ def get_spark_ami(opts):
print stderr,\
Don't recognize %s, assuming type
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24219781
--- Diff: ec2/spark_ec2.py ---
@@ -643,12 +654,14 @@ def setup_cluster(conn, master_nodes, slave_nodes,
opts, deploy_ssh_key):
# NOTE: We
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4385#discussion_r24219900
--- Diff: ec2/spark_ec2.py ---
@@ -643,12 +654,14 @@ def setup_cluster(conn, master_nodes, slave_nodes,
opts, deploy_ssh_key):
# NOTE: We
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4385#issuecomment-72996393
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user florianverhein opened a pull request:
https://github.com/apache/spark/pull/4385
[SPARK-5611] [EC2] Allow spark-ec2 repo and branch to be set on CLI of
spark_ec2.py
and by extension, the ami-list
Useful for using alternate spark-ec2 repos or branches.
You can
52 matches
Mail list logo
