[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/9135#issuecomment-148957580
  
LGTM as well, so I'm going to merge this into master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11000][YARN]Load `metadata.Hive` class ...

[vanzin]

Github user vanzin commented on the pull request:

https://github.com/apache/spark/pull/9026#issuecomment-148959597
  
The description still mentions the old approach, just the title was 
changed. I can delete the description while merging though.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9695] [ML] Add random seed Param to ML ...

[yanboliang]

GitHub user yanboliang opened a pull request:

https://github.com/apache/spark/pull/9158

[SPARK-9695] [ML] Add random seed Param to ML Pipeline

Add random seed Param to ML Pipeline

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yanboliang/spark spark-9695

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/9158.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #9158


commit 21560157545f28abf614213b6fa076148dfbbd90
Author: Yanbo Liang 
Date:   2015-10-18T02:44:04Z

Add random seed Param to ML Pipeline




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9695] [ML] Add random seed Param to ML ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9158#issuecomment-148971468
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10477][SQL] using DSL in ColumnPruningS...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/8645#issuecomment-148958094
  
Jenkins, retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10477][SQL] using DSL in ColumnPruningS...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8645#issuecomment-148958165
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/8995#issuecomment-148958174
  
Jenkins, retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10532][EC2]Added --profile option to sp...

[JoshRosen]

Github user JoshRosen commented on a diff in the pull request:

https://github.com/apache/spark/pull/8696#discussion_r42314994
  
--- Diff: ec2/spark_ec2.py ---
@@ -178,6 +178,9 @@ def parse_args():
 "-i", "--identity-file",
 help="SSH private key file to use for logging into instances")
 parser.add_option(
+"-p", "--profile", default="default",
+help=" If you have multiple profiles(aws or boto config), you can 
configure additional, named profiles by using this option (default: %default)")
--- End diff --

Space before `(`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8995#issuecomment-148958362
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11127] [Streaming] upgrade AWS SDK and ...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/9153#issuecomment-148958325
  
@mengxr, I think this might be pulling in a conflicting version of Guava or 
something like that. Can you diff the dependency tree to check?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11000][YARN]Load `metadata.Hive` class ...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/9026#issuecomment-148958352
  
@vanzin, looks like the description has now been updated, so this should be 
ready for another look I think.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10532][EC2]Added --profile option to sp...

[JoshRosen]

Github user JoshRosen commented on a diff in the pull request:

https://github.com/apache/spark/pull/8696#discussion_r42314991
  
--- Diff: ec2/spark_ec2.py ---
@@ -1311,7 +1314,10 @@ def real_main():
 sys.exit(1)
 
 try:
-conn = ec2.connect_to_region(opts.region)
+if opts.profile != "default":
--- End diff --

What if I wanted to have a profile that's named "default"? I think it would 
better to have `None` as the default and to replace this with an `is not None` 
check.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8995#issuecomment-148958357
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11169] Remove the extra spaces in merge...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9156#issuecomment-148958534
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11169] Remove the extra spaces in merge...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9156#issuecomment-148958537
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8995#issuecomment-148958855
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43890/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8995#issuecomment-148958851
  
  [Test build #43890 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43890/console)
 for   PR 8995 at commit 
[`3d650c8`](https://github.com/apache/spark/commit/3d650c82eda1c6a3b826be8c8b052db5e3b95fc3).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8995#issuecomment-148958854
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10271][Pyspark][MLLib] Added @since tag...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/8627#issuecomment-148959799
  
/bump @yu-iskw @mengxr 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-8510] [CORE] [PYSPARK] NumPy matrices a...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/8384#issuecomment-148959954
  
@mengxr @jkbradley, could one of you help to triage to see if we should add 
this feature? My layperson's gut instinct says that it might be too narrow to 
want to put into Spark, but I'd like your opinion.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10272][Pyspark][MLLib] Added @since tag...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/8628#issuecomment-148959803
  
/bump @yu-iskw @mengxr 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10269][Pyspark][MLLib] Add @since annot...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/8626#issuecomment-148959795
  
/bump @yu-iskw @mengxr 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: SPARK-11172 Close JsonParser/Generator in test

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9157#issuecomment-148965706
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43892/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: SPARK-11172 Close JsonParser/Generator in test

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9157#issuecomment-148965672
  
  [Test build #43892 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43892/console)
 for   PR 9157 at commit 
[`e7e71c9`](https://github.com/apache/spark/commit/e7e71c9b9461a97f49c626d05b1125cf403bae8f).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: SPARK-11172 Close JsonParser/Generator in test

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9157#issuecomment-148965705
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10906][MLlib] More efficient SparseMatr...

[rahulpalamuttam]

Github user rahulpalamuttam commented on the pull request:

https://github.com/apache/spark/pull/8960#issuecomment-148956288
  
@jkbradley 
I have merged you're PR to my branch.
Both of our tests for explicit zeros pass. (I have included both).
Also I have taken out the unnecessary test involving the 10x10 identity 
matrix.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/9135


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10477][SQL] using DSL in ColumnPruningS...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8645#issuecomment-148958482
  
  [Test build #43889 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43889/consoleFull)
 for   PR 8645 at commit 
[`861cf94`](https://github.com/apache/spark/commit/861cf94860814bcdb4701c6d0c93aba952d469b1).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11169] Remove the extra spaces in merge...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/9156#issuecomment-148958433
  
Jenkins, retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10608][CORE] disable reduce locality as...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/8765#issuecomment-148958497
  
/cc @shivaram @mateiz


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9819][Streaming][Documentation] Clarify...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/8103#issuecomment-148958464
  
@tdas, what's the status on this PR? I'm not familiar with the 
documentation being changed here, but this looks like a small PR that might be 
very close to being merged, so I just wanted to bump it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10532][EC2]Added --profile option to sp...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/8696#issuecomment-148964651
  
This seems reasonable to me. Might be nice to have @nchammas take a quick 
peek. If I don't hear otherwise, I'll merge this by Monday.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11169] Remove the extra spaces in merge...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9156#issuecomment-148964917
  
  [Test build #43891 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43891/console)
 for   PR 9156 at commit 
[`ccf9e54`](https://github.com/apache/spark/commit/ccf9e545db4fbb375e04a8ae9c12c0572f5ff7e3).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10477][SQL] using DSL in ColumnPruningS...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8645#issuecomment-148964932
  
  [Test build #43889 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43889/console)
 for   PR 8645 at commit 
[`861cf94`](https://github.com/apache/spark/commit/861cf94860814bcdb4701c6d0c93aba952d469b1).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10477][SQL] using DSL in ColumnPruningS...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8645#issuecomment-148964996
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43889/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10708] Consolidate sort shuffle impleme...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8829#issuecomment-148966297
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10708] Consolidate sort shuffle impleme...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8829#issuecomment-148966300
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10477][SQL] using DSL in ColumnPruningS...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8645#issuecomment-148958170
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10533] [SQL] handle scientific notation...

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/9085#issuecomment-148958241
  
Just to clarify, do HiveSQL and other SQL dialects support this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10949] Update Snappy version to 1.1.2

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8995#issuecomment-148958776
  
  [Test build #43890 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43890/consoleFull)
 for   PR 8995 at commit 
[`3d650c8`](https://github.com/apache/spark/commit/3d650c82eda1c6a3b826be8c8b052db5e3b95fc3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11169] Remove the extra spaces in merge...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9156#issuecomment-148958857
  
  [Test build #43891 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43891/consoleFull)
 for   PR 9156 at commit 
[`ccf9e54`](https://github.com/apache/spark/commit/ccf9e545db4fbb375e04a8ae9c12c0572f5ff7e3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9695] [ML] Add random seed Param to ML ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9158#issuecomment-148974932
  
  [Test build #43894 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43894/console)
 for   PR 9158 at commit 
[`2156015`](https://github.com/apache/spark/commit/21560157545f28abf614213b6fa076148dfbbd90).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `class Pipeline(override val uid: String) extends 
Estimator[PipelineModel] with HasSeed `



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-7634][SQL] thrift server UI optimizatio...

[luogankun]

Github user luogankun closed the pull request at:

https://github.com/apache/spark/pull/6148


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9695] [ML] Add random seed Param to ML ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9158#issuecomment-148974956
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43894/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9695] [ML] Add random seed Param to ML ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9158#issuecomment-148974955
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [WIP][SPARK-7018][Build]: Refactor dev/run-tes...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/7401#issuecomment-148958664
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43887/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [WIP][SPARK-7018][Build]: Refactor dev/run-tes...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/7401#issuecomment-148958663
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Close JsonParser/Generator in test

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9157#issuecomment-148959655
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Close JsonParser/Generator in test

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9157#issuecomment-148959652
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-1061] assumePartitioned

[JoshRosen]

Github user JoshRosen commented on the pull request:

https://github.com/apache/spark/pull/4449#issuecomment-148960328
  
What's the final verdict on this? Can we do the standalone package approach 
for now, then close this out?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [WIP][SPARK-7018][Build]: Refactor dev/run-tes...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/7401#issuecomment-148958603
  
**[Test build #43887 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43887/console)**
 for PR 7401 at commit 
[`206930e`](https://github.com/apache/spark/commit/206930e5f239485c092d0fb4d418fbbb0178).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: SPARK-11172 Close JsonParser/Generator in test

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9157#issuecomment-148960186
  
  [Test build #43892 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43892/consoleFull)
 for   PR 9157 at commit 
[`e7e71c9`](https://github.com/apache/spark/commit/e7e71c9b9461a97f49c626d05b1125cf403bae8f).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10532][EC2]Added --profile option to sp...

[teramonagi]

Github user teramonagi commented on a diff in the pull request:

https://github.com/apache/spark/pull/8696#discussion_r42315781
  
--- Diff: ec2/spark_ec2.py ---
@@ -178,6 +178,9 @@ def parse_args():
 "-i", "--identity-file",
 help="SSH private key file to use for logging into instances")
 parser.add_option(
+"-p", "--profile", default="default",
+help=" If you have multiple profiles(aws or boto config), you can 
configure additional, named profiles by using this option (default: %default)")
--- End diff --

Fixed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10708] Consolidate sort shuffle impleme...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8829#issuecomment-148966823
  
  [Test build #43893 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43893/consoleFull)
 for   PR 8829 at commit 
[`b3af7a8`](https://github.com/apache/spark/commit/b3af7a8168ca496bc53c6b78d72b8a766587d5fe).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9695] [ML] Add random seed Param to ML ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9158#issuecomment-148971474
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9695] [ML] Add random seed Param to ML ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9158#issuecomment-148972893
  
  [Test build #43894 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43894/consoleFull)
 for   PR 9158 at commit 
[`2156015`](https://github.com/apache/spark/commit/21560157545f28abf614213b6fa076148dfbbd90).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10979] [SparkR] Sparkrmerge: Add merge ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9012#issuecomment-148977328
  
  [Test build #43895 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43895/consoleFull)
 for   PR 9012 at commit 
[`8151bb8`](https://github.com/apache/spark/commit/8151bb8209555697f8f85ea4af1d4ba82a0eb366).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10708] Consolidate sort shuffle impleme...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8829#issuecomment-148979286
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43893/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10708] Consolidate sort shuffle impleme...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8829#issuecomment-148979261
  
  [Test build #43893 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43893/console)
 for   PR 8829 at commit 
[`b3af7a8`](https://github.com/apache/spark/commit/b3af7a8168ca496bc53c6b78d72b8a766587d5fe).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10979] [SparkR] Sparkrmerge: Add merge ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9012#issuecomment-148977368
  
  [Test build #43895 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43895/console)
 for   PR 9012 at commit 
[`8151bb8`](https://github.com/apache/spark/commit/8151bb8209555697f8f85ea4af1d4ba82a0eb366).
 * This patch **fails R style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10979] [SparkR] Sparkrmerge: Add merge ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9012#issuecomment-148977369
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10979] [SparkR] Sparkrmerge: Add merge ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9012#issuecomment-148977060
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10979] [SparkR] Sparkrmerge: Add merge ...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9012#issuecomment-148977063
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10708] Consolidate sort shuffle impleme...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8829#issuecomment-148979285
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10895][SPARK-11164][SQL] Push down InSe...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/8956#issuecomment-148891462
  
Are there any performance improvements by pushing this down?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11165] Logging trait should be private ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9155#issuecomment-14257
  
  [Test build #1921 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1921/consoleFull)
 for   PR 9155 at commit 
[`5487f3e`](https://github.com/apache/spark/commit/5487f3ead4a72d5f3eb250be6e536441c9f3a63b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11126][SQL]Fix a memory leak in SQLList...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9132#issuecomment-148890928
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43879/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11126][SQL]Fix a memory leak in SQLList...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9132#issuecomment-148890927
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11126][SQL]Fix a memory leak in SQLList...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9132#issuecomment-148890763
  
  [Test build #43879 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43879/console)
 for   PR 9132 at commit 
[`da33fb6`](https://github.com/apache/spark/commit/da33fb6e4e6728105e590821c720cb69cd5ad292).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10895][SPARK-11164][SQL] Push down InSe...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8956#issuecomment-148891158
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43880/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10895][SPARK-11164][SQL] Push down InSe...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8956#issuecomment-148891157
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11165] Logging trait should be private ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9155#issuecomment-14128
  
  [Test build #1920 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1920/console)
 for   PR 9155 at commit 
[`5487f3e`](https://github.com/apache/spark/commit/5487f3ead4a72d5f3eb250be6e536441c9f3a63b).
 * This patch **fails MiMa tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10895][SPARK-11164][SQL] Push down InSe...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8956#issuecomment-148891128
  
  [Test build #43880 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43880/console)
 for   PR 8956 at commit 
[`02bbab8`](https://github.com/apache/spark/commit/02bbab84a82d645250b2afe9c58bed194523da38).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `  case class InSetFilter[T <: Comparable[T]](valueSet: Set[T])`
  * `  abstract class StringFilter extends UserDefinedPredicate[Binary] `
  * `  case class StringStartsWithFilter(prefix: String) extends 
StringFilter `
  * `  case class StringEndsWithFilter(suffix: String) extends StringFilter 
`
  * `  case class StringContainsFilter(str: String) extends StringFilter `



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10447][WIP][PYSPARK] upgrade pyspark to...

[kevincox]

Github user kevincox commented on the pull request:

https://github.com/apache/spark/pull/8615#issuecomment-148933488
  
Of course, I just wanted to encourage the work on this :smiley: 

@holdenk I'm quite busy at the moment but if I find time i will definitely 
try to help you out.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11029] [ML] Add computeCost to KMeansMo...

[jkbradley]

Github user jkbradley commented on the pull request:

https://github.com/apache/spark/pull/9073#issuecomment-148934287
  
Merging with master.  Thanks for the PR!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9963] [ML] RandomForest cleanup: replac...

[jkbradley]

Github user jkbradley commented on the pull request:

https://github.com/apache/spark/pull/8609#issuecomment-148934618
  
LGTM
Merging with master.
Thanks for the PR!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11029] [ML] Add computeCost to KMeansMo...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/9073


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-9963] [ML] RandomForest cleanup: replac...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/8609


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11131] [core] Fix race in worker regist...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9138#issuecomment-148935387
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11131] [core] Fix race in worker regist...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9138#issuecomment-148935378
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11086][SPARKR] Use dropFactors column-w...

[zero323]

Github user zero323 commented on the pull request:

https://github.com/apache/spark/pull/9099#issuecomment-148935477
  
I've opened [a JIRA for the first 
issue](https://issues.apache.org/jira/browse/SPARK-11167) but it looks like the 
is a regression. It can be easily fixed [by reusing type 
mapping](https://github.com/zero323/spark/blob/SPARK-11086/R/pkg/R/SQLContext.R#L27)
 , but I am starting to think it is not such a good idea after all. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-2960][Deploy] Support executing Spark f...

[vanzin]

Github user vanzin commented on a diff in the pull request:

https://github.com/apache/spark/pull/8669#discussion_r42311622
  
--- Diff: bin/load-spark-env.sh ---
@@ -20,7 +20,29 @@
 # This script loads spark-env.sh if it exists, and ensures it is only 
loaded once.
 # spark-env.sh is loaded from SPARK_CONF_DIR if set, or within the current 
directory's
 # conf/ subdirectory.
-FWDIR="$(cd "`dirname "$0"`"/..; pwd)"
+realpath () {
--- End diff --

Can this function be replaced by:

DIR="$(dirname "$(readlink -f "$0")")"

I see you're recursively resolving the symlink and putting a limit in the 
depth of recursion, but it sounds like `readlink -f` does just that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-2960][Deploy] Support executing Spark f...

[vanzin]

Github user vanzin commented on a diff in the pull request:

https://github.com/apache/spark/pull/8669#discussion_r42311628
  
--- Diff: bin/load-spark-env.sh ---
@@ -20,7 +20,29 @@
 # This script loads spark-env.sh if it exists, and ensures it is only 
loaded once.
 # spark-env.sh is loaded from SPARK_CONF_DIR if set, or within the current 
directory's
 # conf/ subdirectory.
-FWDIR="$(cd "`dirname "$0"`"/..; pwd)"
+realpath () {
--- End diff --

Nevermind, I just saw the comment about Macs not supporting that. There's 
no alternative on Macs? (If no, that sucks.)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11131] [core] Fix race in worker regist...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9138#issuecomment-148936605
  
  [Test build #43884 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43884/consoleFull)
 for   PR 9138 at commit 
[`2d35024`](https://github.com/apache/spark/commit/2d350249bbf40baa7677b150e4481c675784e9f2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10382] [POC] Make example code in user ...

[yinxusen]

Github user yinxusen commented on a diff in the pull request:

https://github.com/apache/spark/pull/9109#discussion_r42306903
  
--- Diff: docs/_plugins/include_example.rb ---
@@ -0,0 +1,70 @@
+require 'octopress-code-highlighter'
+require 'liquid'
+
+module Octopress
+  module IncludeExample
+class Tag < Liquid::Tag
+  
+  # Pattern for {% include_example /path/to/a/file.lan %}
+  FileOnly = /^(\S+)$/
+  # Pattern for {% include_example /path/to/a/file.lan title %}
+  FileTitle = /(\S+)\s+(\S.*?)$/i
+
+  def initialize(tag_name, markup, tokens)
+@markup = markup
+super
+  end
+
+  def render(context)
+site = context.registers[:site]
+config_dir = (site.config['code_dir'] || 
'../examples/src/main').sub(/^\//,'')
+@code_dir = File.join(site.source, config_dir)
+
+begin
+  options = get_options
+  code = File.open(@file).read.encode("UTF-8")
+  code = select_lines(code, options)
+
+  CodeHighlighter.highlight(code, options)
+rescue => e
+  puts "not work"
+end
+  end
+
+  def get_options
+defaults = {}
+clean_markup = CodeHighlighter.clean_markup(@markup).strip
+
+if clean_markup =~ FileOnly
+  @file = File.join(@code_dir, $1)
+elsif clean_markup =~ FileTitle
+  @file = File.join(@code_dir, $1)
+  defaults[:title] = $2
+end
+
+options = CodeHighlighter.parse_markup(@markup, defaults)
+options[:lang] ||= File.extname(@file).delete('.')
+options[:link_text] ||= "Example code"
+options
+  end
+
+  # Select lines according to labels in code. Currently we use "begin 
code" and "end code" as
+  # labels.
+  def select_lines(code, options)
+lines = code.each_line.to_a
+start = lines.each_with_index.select { |l, i| l.include? "begin 
code" }.last.last
--- End diff --

Here is the thing, different language requires different comment signs. So 
in Java and Scala we have `// example on` and `// example off` (or `/* example 
on/off */`) and in Python, it is `# example on/off`. Does your `$example 
on/off$` mean that we use the dollar sign to distinguish our label with common 
code comments?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10382] [POC] Make example code in user ...

[yinxusen]

Github user yinxusen commented on a diff in the pull request:

https://github.com/apache/spark/pull/9109#discussion_r42306906
  
--- Diff: docs/index.md ---
@@ -12,6 +12,8 @@ It also supports a rich set of higher-level tools 
including [Spark SQL](sql-prog
 
 # Downloading
 
+{% include_example scala/org/apache/spark/examples/SparkALS.scala SparkALS 
%}
--- End diff --

Sure I will add it after refactoring the code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10478][ML] Performance, organization, a...

[feynmanliang]

Github user feynmanliang commented on the pull request:

https://github.com/apache/spark/pull/8648#issuecomment-148893887
  
@mengxr added benchmarks, can you make another pass when you have a chance


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11165] Logging trait should be private ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9155#issuecomment-148894828
  
  [Test build #1921 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1921/console)
 for   PR 9155 at commit 
[`5487f3e`](https://github.com/apache/spark/commit/5487f3ead4a72d5f3eb250be6e536441c9f3a63b).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11063][Streaming]Change preferredLocati...

[tdas]

Github user tdas commented on the pull request:

https://github.com/apache/spark/pull/9075#issuecomment-148896953
  
LGTM. Just a clarifying question. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11039][Documentation][Web UI] Document ...

[syepes]

Github user syepes commented on the pull request:

https://github.com/apache/spark/pull/9052#issuecomment-148893986
  
Sorry for the question I just found these settings:
```
spark.ui.retainedJobs
spark.ui.retainedStages
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11039][Documentation][Web UI] Document ...

[syepes]

Github user syepes commented on the pull request:

https://github.com/apache/spark/pull/9052#issuecomment-148893464
  
@pnpritchard I have just found these settings and they are extremely 
useful. Do you know is there exits the same options for the Completed Jobs & 
Stages?
When running a stream jobs for a long time the list gets very big..

Regards,
Sebastian


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10532][EC2]Added --profile option to sp...

[teramonagi]

Github user teramonagi commented on a diff in the pull request:

https://github.com/apache/spark/pull/8696#discussion_r42307104
  
--- Diff: ec2/spark_ec2.py ---
@@ -178,6 +178,9 @@ def parse_args():
 "-i", "--identity-file",
 help="SSH private key file to use for logging into instances")
 parser.add_option(
+"-p", "--profile", default="default",
+help=" If you have multiple profiles, you can configure 
additional, named profiles by using this option (default: %default)")
--- End diff --

Actually speaking, we can use any types of profile(AWS/boto). The remaining 
problem is just the loading order of these "profiles" as [Jan Vlcinsky 
explained](https://stackoverflow.com/questions/11286479/recommended-way-to-manage-credentials-with-multiple-aws-accounts).
 I think that it is little bit long to explain that in commandline help.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10895][SPARK-11164][SQL] Push down InSe...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/8956#issuecomment-148891749
  
Thanks - that'd be great.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-2960][Deploy] Support executing Spark f...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/8669#issuecomment-148891805
  
@JoshRosen yes agree. See the discussion above out the small 
chicken-and-egg problem there about the location of the helper script. Still 
may be a better solution than this much duplication, or else, declare that 
`SPARK_HOME` must be set.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10895][SPARK-11164][SQL] Push down InSe...

[viirya]

Github user viirya commented on the pull request:

https://github.com/apache/spark/pull/8956#issuecomment-148891670
  
I can run some performance tests later.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10382] [POC] Make example code in user ...

[yinxusen]

Github user yinxusen commented on a diff in the pull request:

https://github.com/apache/spark/pull/9109#discussion_r42306886
  
--- Diff: docs/_plugins/include_example.rb ---
@@ -0,0 +1,70 @@
+require 'octopress-code-highlighter'
+require 'liquid'
+
+module Octopress
+  module IncludeExample
+class Tag < Liquid::Tag
+  
+  # Pattern for {% include_example /path/to/a/file.lan %}
+  FileOnly = /^(\S+)$/
+  # Pattern for {% include_example /path/to/a/file.lan title %}
+  FileTitle = /(\S+)\s+(\S.*?)$/i
+
+  def initialize(tag_name, markup, tokens)
+@markup = markup
+super
+  end
+
+  def render(context)
+site = context.registers[:site]
+config_dir = (site.config['code_dir'] || 
'../examples/src/main').sub(/^\//,'')
+@code_dir = File.join(site.source, config_dir)
+
+begin
+  options = get_options
+  code = File.open(@file).read.encode("UTF-8")
+  code = select_lines(code, options)
+
+  CodeHighlighter.highlight(code, options)
--- End diff --

So there is no need to call outside function, just translate it into 
markdown highlight. Then we can avoid to use Octopress in the beginning. I get 
it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11063][Streaming]Change preferredLocati...

[tdas]

Github user tdas commented on a diff in the pull request:

https://github.com/apache/spark/pull/9075#discussion_r42307531
  
--- Diff: 
streaming/src/test/scala/org/apache/spark/streaming/scheduler/ReceiverTrackerSuite.scala
 ---
@@ -80,6 +82,28 @@ class ReceiverTrackerSuite extends TestSuiteBase {
   }
 }
   }
+
+  test("SPARK-11063: TaskSetManager should use Receiver RDD's 
preferredLocations") {
+// Use ManualClock to prevent from starting batches so that we can 
make sure the only task is
+// for starting the Receiver
+val _conf = conf.clone.set("spark.streaming.clock", 
"org.apache.spark.util.ManualClock")
+withStreamingContext(new StreamingContext(_conf, Milliseconds(100))) { 
ssc =>
+  @volatile var receiverTaskLocality: TaskLocality = null
+  ssc.sparkContext.addSparkListener(new SparkListener {
+override def onTaskStart(taskStart: SparkListenerTaskStart): Unit 
= {
+  receiverTaskLocality = taskStart.taskInfo.taskLocality
+}
+  })
+  val input = ssc.receiverStream(new TestReceiver)
+  val output = new TestOutputStream(input)
+  output.register()
+  ssc.start()
+  eventually(timeout(10 seconds), interval(10 millis)) {
+// If preferredLocations is set correctly, receiverTaskLocality 
should be NODE_LOCAL
+assert(receiverTaskLocality === TaskLocality.NODE_LOCAL)
--- End diff --

What will it be if it is not set correctly? That is, without the fix above, 
what would locality be


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

[koertkuipers]

Github user koertkuipers commented on a diff in the pull request:

https://github.com/apache/spark/pull/8416#discussion_r42308755
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ---
@@ -123,6 +124,24 @@ class DataFrameReader private[sql](sqlContext: 
SQLContext) extends Logging {
   }
 
   /**
+   * Loads input in as a [[DataFrame]], for data sources that support 
multiple paths.
+   * Only works if the source is a HadoopFsRelationProvider.
+   *
+   * @since 1.6.0
+   */
+  def load(paths: Seq[String]): DataFrame = {
--- End diff --

That makes sense Since it works for java too that eay
On Oct 17, 2015 7:14 AM, "Michael Armbrust" 
wrote:

> In sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala
> :
>
> > @@ -123,6 +124,24 @@ class DataFrameReader private[sql](sqlContext: 
SQLContext) extends Logging {
> >}
> >
> >/**
> > +   * Loads input in as a [[DataFrame]], for data sources that support 
multiple paths.
> > +   * Only works if the source is a HadoopFsRelationProvider.
> > +   *
> > +   * @since 1.6.0
> > +   */
> > +  def load(paths: Seq[String]): DataFrame = {
>
> This should be just a single load(paths: Array[String]) as in other parts
> of the API.
>
> —
> Reply to this email directly or view it on GitHub
> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: SPARK-4644 blockjoin

[koertkuipers]

Github user koertkuipers closed the pull request at:

https://github.com/apache/spark/pull/6883


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: SPARK-4644 blockjoin

[koertkuipers]

Github user koertkuipers commented on the pull request:

https://github.com/apache/spark/pull/6883#issuecomment-148917448
  
sure


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org