date:20161128

[GitHub] spark issue #16054: dstreaming receiver for twitter

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16054
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15935: [SPARK-18188] add checksum for blocks of broadcast

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15935
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69299/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15935: [SPARK-18188] add checksum for blocks of broadcast

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15935
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16054: dstreaming receiver for twitter

2016-11-28 Thread Aegeaner

GitHub user Aegeaner opened a pull request:

https://github.com/apache/spark/pull/16054

dstreaming receiver for twitter

## What changes were proposed in this pull request?

implement a streaming receiver for twitter

## How was this patch tested?

./dev/run-tests passed

Please review http://spark.apache.org/contributing.html before opening a 
pull request.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Aegeaner/spark spark-streaming-twitter

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/16054.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #16054


commit 80cfd0ed2ea20e89670c3f4dd8cd7a1e413bc19c
Author: Aegeaner 
Date:   2016-11-29T07:35:32Z

streaming receiver for twitter

commit 9aa8da4f15dee6260c8d89825310bb123524807c
Author: Aegeaner 
Date:   2016-11-29T07:46:34Z

cleanup




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16048: [DO_NOT_MERGE]Test kafka deletion

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16048
  
**[Test build #69309 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69309/consoleFull)**
 for PR 16048 at commit 
[`6abd7cf`](https://github.com/apache/spark/commit/6abd7cfba4876bd76c91209066c8a1db8006ccde).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15935: [SPARK-18188] add checksum for blocks of broadcast

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15935
  
**[Test build #69299 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69299/consoleFull)**
 for PR 15935 at commit 
[`5c56974`](https://github.com/apache/spark/commit/5c56974f3e366688fc4c8fa2909a411788b03a1d).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15954
  
**[Test build #69308 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69308/consoleFull)**
 for PR 15954 at commit 
[`d92f4bf`](https://github.com/apache/spark/commit/d92f4bf9bb7a048f9db22750d12886a180e8f2c3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto p...

2016-11-28 Thread rxin

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/16052#discussion_r89954432
  
--- Diff: 
core/src/main/scala/org/apache/spark/serializer/SerializerManager.scala ---
@@ -77,8 +77,8 @@ private[spark] class SerializerManager(defaultSerializer: 
Serializer, conf: Spar
 primitiveAndPrimitiveArrayClassTags.contains(ct) || ct == 
stringClassTag
   }
 
-  def getSerializer(ct: ClassTag[_]): Serializer = {
-if (canUseKryo(ct)) {
+  def getSerializer(ct: ClassTag[_], autoPick: Boolean): Serializer = {
--- End diff --

document what this means


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15877: [SPARK-18429] [SQL] implement a new Aggregate for...

2016-11-28 Thread wzhfy

Github user wzhfy commented on a diff in the pull request:

https://github.com/apache/spark/pull/15877#discussion_r89953368
  
--- Diff: 
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
 ---
@@ -152,6 +153,16 @@ public void add(Object item) {
   public void add(Object item, long count) {
 if (item instanceof String) {
   addString((String) item, count);
+} else if (item instanceof BigDecimal) {
+  addString(((BigDecimal) item).toString(), count);
--- End diff --

Sorry I didn't describe it accurately. It should be "There is a one-to-one 
mapping between the **distinguishable values** and the result of this 
conversion." (from java doc of BigDecimal)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15788: [SPARK-18291][SparkR][ML] SparkR glm predict should outp...

2016-11-28 Thread yanboliang

Github user yanboliang commented on the issue:

https://github.com/apache/spark/pull/15788
  
@jkbradley I agree that exposing probabilities is also useful to R users. 
Think more about this issue, I think we can add a new argument ```type``` to 
```predict```, this argument controls the type of prediction required. This 
will match the behavior of R 
[```glm.predict```](https://stat.ethz.ch/R-manual/R-devel/library/stats/html/predict.glm.html)
 and other native R predict method.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16030
  
**[Test build #69307 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69307/consoleFull)**
 for PR 16030 at commit 
[`43f028d`](https://github.com/apache/spark/commit/43f028d3b495a825cebff39daa12d6a2f25f0110).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16028
  
**[Test build #69305 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69305/consoleFull)**
 for PR 16028 at commit 
[`74cb363`](https://github.com/apache/spark/commit/74cb3639278fa525dfe5b75d11a7a8dcd06f04a3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter methods to...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16017
  
**[Test build #69306 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69306/consoleFull)**
 for PR 16017 at commit 
[`30f5096`](https://github.com/apache/spark/commit/30f5096ce9dce89e3d3a3014bc53164cc2af2788).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KMeans of...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13557
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KMeans of...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/13557
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69302/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KMeans of...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13557
  
**[Test build #69302 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69302/consoleFull)**
 for PR 13557 at commit 
[`01c6da9`](https://github.com/apache/spark/commit/01c6da9a7cdc4ee598cffd69cc3f6a01d59984d6).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15949: [SPARK-18339] [SPARK-18513] [SQL] Don't push down...

2016-11-28 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/15949


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15949: [SPARK-18339] [SPARK-18513] [SQL] Don't push down curren...

2016-11-28 Thread tdas

Github user tdas commented on the issue:

https://github.com/apache/spark/pull/15949
  
Merging this to master and 2.1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15662: [SPARK-18141][SQL] Fix to quote column names in the pred...

2016-11-28 Thread sureshthalamati

Github user sureshthalamati commented on the issue:

https://github.com/apache/spark/pull/15662
  
Thanks, @gatorsmile . Resolved the conflicts, and also added test case for 
empty in clause with mixed case column name.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16045: [SPARK-18553][CORE] Fix leak of TaskSetManager following...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16045
  
**[Test build #69304 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69304/consoleFull)**
 for PR 16045 at commit 
[`9689763`](https://github.com/apache/spark/commit/96897631dfb5efc9cbf94044ea4eb0a8764ae78c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16053
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter met...

2016-11-28 Thread yanboliang

Github user yanboliang commented on a diff in the pull request:

https://github.com/apache/spark/pull/16017#discussion_r89949760
  
--- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeParams.scala ---
@@ -107,25 +107,41 @@ private[ml] trait DecisionTreeParams extends 
PredictorParams
   setDefault(maxDepth -> 5, maxBins -> 32, minInstancesPerNode -> 1, 
minInfoGain -> 0.0,
 maxMemoryInMB -> 256, cacheNodeIds -> false, checkpointInterval -> 10)
 
-  /** @group setParam */
+  /**
+   * @deprecated This method is deprecated and will be removed in 2.2.0.
+   * @group setParam
+   */
+  @deprecated("This method is deprecated and will be removed in 2.2.0.", 
"2.1.0")
--- End diff --

This will not appear in estimator classes, only in models. This can be 
confirmed by the following ways:
The IDE will prompt deprecated methods only for models: 

![image](https://cloud.githubusercontent.com/assets/1962026/20699424/247abfc4-b5bc-11e6-8eb2-a9fe53661cba.png)
We can also confirm it in Scala API docs:
```DecisionTreeClassifier```

![image](https://cloud.githubusercontent.com/assets/1962026/20699474/63b03688-b5bc-11e6-9e8d-b6d83f6bb00a.png)
```DecisionTreeClassificationModel```

![image](https://cloud.githubusercontent.com/assets/1962026/20699522/a0bf3556-b5bc-11e6-9289-d2c893f9428e.png)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16053
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69301/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16053
  
**[Test build #69301 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69301/consoleFull)**
 for PR 16053 at commit 
[`e3cb71e`](https://github.com/apache/spark/commit/e3cb71ea57e135975ce3e28caaba314a9a599b86).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16045: [SPARK-18553][CORE] Fix leak of TaskSetManager following...

2016-11-28 Thread JoshRosen

Github user JoshRosen commented on the issue:

https://github.com/apache/spark/pull/16045
  
Jenkins, retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16052
  
**[Test build #69303 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69303/consoleFull)**
 for PR 16052 at commit 
[`d205ebd`](https://github.com/apache/spark/commit/d205ebd6c44ae37c620c35654fac8632440b00f4).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-11-28 Thread kayousterhout

Github user kayousterhout commented on the issue:

https://github.com/apache/spark/pull/15505
  
@witgo I took a look at this and had an idea for an alternate approach that 
is much simpler.  Can you take a look at #16053 and let me know what you think?

If we decide to go with the approach I proposed, I'll leave the JIRA 
assigned to you so you're noted as fixing this in the release notes!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KMeans of...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/13557
  
**[Test build #69302 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69302/consoleFull)**
 for PR 13557 at commit 
[`01c6da9`](https://github.com/apache/spark/commit/01c6da9a7cdc4ee598cffd69cc3f6a01d59984d6).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16053
  
**[Test build #69301 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69301/consoleFull)**
 for PR 16053 at commit 
[`e3cb71e`](https://github.com/apache/spark/commit/e3cb71ea57e135975ce3e28caaba314a9a599b86).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16053: [SPARK-17931] Eliminate unncessary task (de) seri...

2016-11-28 Thread kayousterhout

GitHub user kayousterhout opened a pull request:

https://github.com/apache/spark/pull/16053

[SPARK-17931] Eliminate unncessary task (de) serialization

## What changes were proposed in this pull request?

In the existing code, there are three layers of serialization
involved in sending a task from the scheduler to an executor:
- A Task object is serialized
- The Task object is copied to a byte buffer that also
  contains serialized information about any additional JARs,
  files, and Properties needed for the task to execute. This
  byte buffer is stored as the member variable serializedTask
  in the TaskDescription class.
- The TaskDescription is serialized (in addition to the serialized
  task + JARs, the TaskDescription class contains the task ID and
  other metadata) and sent in a LaunchTask message.

While it *is* necessary to have two layers of serialization, so that
the JAR, file, and Property info can be deserialized prior to
deserializing the Task object, the third layer of deserialization is
unnecessary.  This commit eliminates a layer of serialization by moving
the JARs, files, and Properties into the TaskDescription class.

## How was this patch tested?

Unit tests

This is a simpler alternative to the approach proposed in #15505.

The biggest difference in functionality from the approach there is that, in 
that code, all of the serialization occurs in one place (in 
CoarseGrainedExecutorBackend), whereas this approach maintains the split of 
serialization (where some happens in TaskSetManager and some in 
CoarseGrainedExecutorBackend) that was present in the existing code.  I do 
think there are some benefits of doing all of the serialization in one place 
(e.g., to time it all, or to enable opportunities for parallelism in the 
future) but given that we don't take advantage of any of those thigns 
currently, it doesn't seem necessary to change (and it's simpler not to change 
it).

cc @shivaram

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kayousterhout/spark-1 SPARK-17931

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/16053.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #16053


commit e3cb71ea57e135975ce3e28caaba314a9a599b86
Author: Kay Ousterhout 
Date:   2016-11-29T06:23:45Z

[SPARK-17931] Eliminate unncessary task (de) serialization

In the existing code, there are three layers of serialization
involved in sending a task from the scheduler to an executor:
- A Task object is serialized
- The Task object is copied to a byte buffer that also
  contains serialized information about any additional JARs,
  files, and Properties needed for the task to execute. This
  byte buffer is stored as the member variable serializedTask
  in the TaskDescription class.
- The TaskDescription is serialized (in addition to the serialized
  task + JARs, the TaskDescription class contains the task ID and
  other metadata) and sent in a LaunchTask message.

While it *is* necessary to have two layers of serialization, so that
the JAR, file, and Property info can be deserialized prior to
deserializing the Task object, the third layer of deserialization is
unnecessary.  This commit eliminates a layer of serialization by moving
the JARs, files, and Properties into the TaskDescription class.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15949: [SPARK-18339] [SPARK-18513] [SQL] Don't push down curren...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15949
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69294/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15949: [SPARK-18339] [SPARK-18513] [SQL] Don't push down curren...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15949
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15949: [SPARK-18339] [SPARK-18513] [SQL] Don't push down curren...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15949
  
**[Test build #69294 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69294/consoleFull)**
 for PR 15949 at commit 
[`0ea5796`](https://github.com/apache/spark/commit/0ea579695050ae9c1dd7f68cc8df2dc7fbfc833e).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15620: [SPARK-18091] [SQL] Deep if expressions cause Generated ...

2016-11-28 Thread kapilsingh5050

Github user kapilsingh5050 commented on the issue:

https://github.com/apache/spark/pull/15620
  
I'm not sure whether I understand the problem completely. It seems 
WholeStage CodeGen wants the complete stage code in a single function. The fix 
I made violates this because it creates separate methods for codegen of If 
expression's condition, if block and then block (of course, when the total code 
is beyond a threshold).

Now, if my assumption about WholeStage CodeGen is correct then how does the 
split expressions logic work? Or maybe the problem is specific to 
initialisation of input attributes of a stage.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16052
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16052
  
**[Test build #69300 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69300/consoleFull)**
 for PR 16052 at commit 
[`e4416bd`](https://github.com/apache/spark/commit/e4416bd155f1b3a973fd37f03a60d61f1a0612f3).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16052
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69300/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16011: [SPARK-18587][ML] Remove handleInvalid from QuantileDisc...

2016-11-28 Thread yanboliang

Github user yanboliang commented on the issue:

https://github.com/apache/spark/pull/16011
  
That's OK. I will improve the docs in another 2.1 QA PR and close this one. 
Thanks for all your clarification. @MLnick @jkbradley 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15848: [SPARK-9487] Use the same num. worker threads in Java/Sc...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15848
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16011: [SPARK-18587][ML] Remove handleInvalid from Quant...

2016-11-28 Thread yanboliang

Github user yanboliang closed the pull request at:

https://github.com/apache/spark/pull/16011


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15848: [SPARK-9487] Use the same num. worker threads in Java/Sc...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15848
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69297/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15848: [SPARK-9487] Use the same num. worker threads in Java/Sc...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15848
  
**[Test build #69297 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69297/consoleFull)**
 for PR 15848 at commit 
[`eb0936b`](https://github.com/apache/spark/commit/eb0936b1491f021393412d0a19f6effad16ccb29).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-28 Thread uncleGen

Github user uncleGen commented on the issue:

https://github.com/apache/spark/pull/16052
  
cc @zsxwing 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16052
  
**[Test build #69300 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69300/consoleFull)**
 for PR 16052 at commit 
[`e4416bd`](https://github.com/apache/spark/commit/e4416bd155f1b3a973fd37f03a60d61f1a0612f3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15255: [SPARK-17680] [SQL] [TEST] Added a Testcase for V...

2016-11-28 Thread cloud-fan

Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/15255#discussion_r89946590
  
--- Diff: 
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala 
---
@@ -148,6 +148,52 @@ class HiveDDLSuite
 }
   }
 
+  test("create Hive-serde table and view with unicode columns and 
comment") {
+val catalog = spark.sessionState.catalog
+val tabName = "tab1"
+val viewName = "view1"
+// scalastyle:off
+// non ascii characters are not allowed in the source code, so we 
disable the scalastyle.
+val colName1 = "å"
+val colName2 = "å°¼"
+val comment = "åº"
+// scalastyle:on
+withTable(tabName) {
+  // non ascii characters are not allowed in the source code, so we 
disable the scalastyle.
--- End diff --

remove this line?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto p...

2016-11-28 Thread uncleGen

GitHub user uncleGen opened a pull request:

https://github.com/apache/spark/pull/16052

[SPARK-18617][CORE][STREAMING] Close "kryo auto pick" feature for Spark 
Streaming

## What changes were proposed in this pull request?

#15992 provided a solution to fix the bug, i.e. **receiver data can not be 
deserialized properly**. As @zsxwing said, it is a critical bug, but we should 
not break APIs between maintenance releases. It may be a rational choice to 
close auto pick kryo serializer for Spark Streaming in the first step.

## How was this patch tested?

existing ut



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/uncleGen/spark SPARK-18617

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/16052.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #16052


commit e4416bd155f1b3a973fd37f03a60d61f1a0612f3
Author: uncleGen 
Date:   2016-11-29T06:11:07Z

SPARK-18617: Close "kryo auto pick" feature for Spark Streaming




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15255: [SPARK-17680] [SQL] [TEST] Added a Testcase for Verifyin...

2016-11-28 Thread gatorsmile

Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/15255
  
cc @cloud-fan Could you review this PR? Thanks!



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching...

2016-11-28 Thread cloud-fan

Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/15975#discussion_r89946424
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala
 ---
@@ -76,9 +76,6 @@ class JDBCOptions(
 
   // the number of partitions
   val numPartitions = parameters.get(JDBC_NUM_PARTITIONS).map(_.toInt)
--- End diff --

what's the behaviour of this config is set to 0 or negative for read path?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15877: [SPARK-18429] [SQL] implement a new Aggregate for...

2016-11-28 Thread rxin

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/15877#discussion_r89946344
  
--- Diff: 
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
 ---
@@ -152,6 +153,16 @@ public void add(Object item) {
   public void add(Object item, long count) {
 if (item instanceof String) {
   addString((String) item, count);
+} else if (item instanceof BigDecimal) {
+  addString(((BigDecimal) item).toString(), count);
--- End diff --

Is this true? 

"1.0" and "1.00" is the same value but not the same string representation.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching...

2016-11-28 Thread cloud-fan

Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/15975#discussion_r89946224
  
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala 
---
@@ -209,6 +209,14 @@ class JDBCSuite extends SparkFunSuite
 conn.close()
   }
 
+  // Check whether the tables are fetched in the expected degree of 
parallelism
+  def checkNumPartitions(df: DataFrame, expectedNumPartitions: Int): Unit 
= {
+val explain = ExplainCommand(df.queryExecution.logical, extended = 
true)
+val plans = spark.sessionState.executePlan(explain).executedPlan
+val expectedMsg = 
s"${JDBCOptions.JDBC_NUM_PARTITIONS}=$expectedNumPartitions"
--- End diff --

instead of matching string, can we call `collect` on the logical plan and 
get the `JDBCRelation` to check the partitions directly?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16000: [SPARK-18537][Web UI]Add a REST api to spark stre...

2016-11-28 Thread ChorPangChan

Github user ChorPangChan commented on a diff in the pull request:

https://github.com/apache/spark/pull/16000#discussion_r89945721
  
--- Diff: 
streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala ---
@@ -145,6 +146,10 @@ private[ui] class StreamingPage(parent: StreamingTab)
   private val listener = parent.listener
   private val startTime = System.currentTimeMillis()
 
+  parent.parent.attachHandler(
+StreamingApiRootResource.getServletHandler(parent.parent, listener, 
startTime)
--- End diff --

all right.
I supose i can make the change
but we need to decide where to record the startTime first
let me trace the code a little bit more.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15983: [SPARK-18544] [SQL] Append with df.saveAsTable wr...

2016-11-28 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/15983


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching...

2016-11-28 Thread cloud-fan

Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/15975#discussion_r89945683
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala
 ---
@@ -667,13 +667,13 @@ object JdbcUtils extends Logging {
 val getConnection: () => Connection = createConnectionFactory(options)
 val batchSize = options.batchSize
 val isolationLevel = options.isolationLevel
-val numPartitions = options.numPartitions
-val repartitionedDF =
-  if (numPartitions.isDefined && numPartitions.get < 
df.rdd.getNumPartitions) {
-df.coalesce(numPartitions.get)
-  } else {
-df
-  }
+val repartitionedDF = options.numPartitions match {
+  case Some(n) if n <= 0 => throw new IllegalArgumentException(
--- End diff --

so this check is only in write path now?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #14517: [SPARK-16931][PYTHON] PySpark APIS for bucketBy a...

2016-11-28 Thread HyukjinKwon

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/14517#discussion_r89945485
  
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -747,16 +800,25 @@ def _test():
 except py4j.protocol.Py4JError:
 spark = SparkSession(sc)
 
+seed = int(time() * 1000)
--- End diff --

@GregBowyer ping


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15983: [SPARK-18544] [SQL] Append with df.saveAsTable writes da...

2016-11-28 Thread rxin

Github user rxin commented on the issue:

https://github.com/apache/spark/pull/15983
  
Merging in master/branch-2.1.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15620: [SPARK-18091] [SQL] Deep if expressions cause Generated ...

2016-11-28 Thread rxin

Github user rxin commented on the issue:

https://github.com/apache/spark/pull/15620
  
@kapilsingh5050 any update?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-28 Thread rxin

Github user rxin commented on the issue:

https://github.com/apache/spark/pull/16014
  
Why is this a blocker thing?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16048: [DO_NOT_MERGE]Test kafka deletion

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16048
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16048: [DO_NOT_MERGE]Test kafka deletion

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16048
  
**[Test build #69284 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69284/consoleFull)**
 for PR 16048 at commit 
[`c90f38e`](https://github.com/apache/spark/commit/c90f38e6b813c6863c42fe6b8194df0bd41a0dce).
 * This patch **fails executing the `dev/run-tests` script**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16048: [DO_NOT_MERGE]Test kafka deletion

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16048
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69284/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16041: [SPARK-18058][SQL][TRIVIAL] Use dataType.sameResu...

2016-11-28 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/16041


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16041: [SPARK-18058][SQL][TRIVIAL] Use dataType.sameResult(...)...

2016-11-28 Thread rxin

Github user rxin commented on the issue:

https://github.com/apache/spark/pull/16041
  
Merging in master/branch-2.1.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16028
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16000: [SPARK-18537][Web UI]Add a REST api to spark stre...

2016-11-28 Thread ajbozarth

Github user ajbozarth commented on a diff in the pull request:

https://github.com/apache/spark/pull/16000#discussion_r89943946
  
--- Diff: 
streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala ---
@@ -145,6 +146,10 @@ private[ui] class StreamingPage(parent: StreamingTab)
   private val listener = parent.listener
   private val startTime = System.currentTimeMillis()
 
+  parent.parent.attachHandler(
+StreamingApiRootResource.getServletHandler(parent.parent, listener, 
startTime)
--- End diff --

I know its small but I'd rather not commit code with the plan to change it 
later.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16028
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69298/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16028
  
**[Test build #69298 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69298/consoleFull)**
 for PR 16028 at commit 
[`5901525`](https://github.com/apache/spark/commit/59015259e35a330f11e8b6b09e4a7372099fe0f1).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15954
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69290/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15954
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15954
  
**[Test build #69290 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69290/consoleFull)**
 for PR 15954 at commit 
[`32ff04e`](https://github.com/apache/spark/commit/32ff04e3b8e3c1985ef1ffdc28039153d72d2f28).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16033: SPARK-18607 get a result on a percent of the task...

2016-11-28 Thread rxin

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/16033#discussion_r89942478
  
--- Diff: 
core/src/main/scala/org/apache/spark/partial/ApproximateActionListener.scala ---
@@ -34,11 +34,13 @@ private[spark] class ApproximateActionListener[T, U, R](
 rdd: RDD[T],
 func: (TaskContext, Iterator[T]) => U,
 evaluator: ApproximateEvaluator[U, R],
-timeout: Long)
+timeout: Long = 1000*60*60*24*30*12,
--- End diff --

@Ru-Xiang that's orthogonal to what @srowen was suggesting. The issue here 
is that the change is breaking binary compatibility for Spark applications.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15935: [SPARK-18188] add checksum for blocks of broadcast

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15935
  
**[Test build #69299 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69299/consoleFull)**
 for PR 15935 at commit 
[`5c56974`](https://github.com/apache/spark/commit/5c56974f3e366688fc4c8fa2909a411788b03a1d).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #15981: [SPARK-18547][core] Propagate I/O encryption key ...

2016-11-28 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/15981


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15935: [SPARK-18188] add checksum for blocks of broadcast

2016-11-28 Thread zsxwing

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/15935
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15981: [SPARK-18547][core] Propagate I/O encryption key when ex...

2016-11-28 Thread zsxwing

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/15981
  
LGTM. The build for the latest commit passed. Merging to master and 2.1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #13690: [SPARK-15767][R][ML] Decision Tree Regression wrapper in...

2016-11-28 Thread HyukjinKwon

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/13690
  
gentle ping @vectorijk 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16051: [SPARK-18588][SS][Kafka]Ignore the flaky kafka te...

2016-11-28 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/16051


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16051: [SPARK-18588][SS][Kafka]Ignore the flaky kafka test

2016-11-28 Thread zsxwing

Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/16051
  
Merging to master and 2.1 to unblock other PRs.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16051: [SPARK-18588][SS][Kafka]Ignore the flaky kafka test

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16051
  
**[Test build #69296 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69296/consoleFull)**
 for PR 16051 at commit 
[`fc63126`](https://github.com/apache/spark/commit/fc63126a43e684fd782dfc7b55242b14de6cffe2).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16051: [SPARK-18588][SS][Kafka]Ignore the flaky kafka test

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16051
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69296/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16051: [SPARK-18588][SS][Kafka]Ignore the flaky kafka test

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16051
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15954
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69288/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15954
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16028
  
**[Test build #69298 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69298/consoleFull)**
 for PR 16028 at commit 
[`5901525`](https://github.com/apache/spark/commit/59015259e35a330f11e8b6b09e4a7372099fe0f1).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15954
  
**[Test build #69288 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69288/consoleFull)**
 for PR 15954 at commit 
[`f6d60df`](https://github.com/apache/spark/commit/f6d60dfce923bae79437925cdf39e3d7e0573f6d).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16050: [SPARK-18615][DOCS] Switch to multi-line doc to avoid a ...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16050
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69285/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16050: [SPARK-18615][DOCS] Switch to multi-line doc to avoid a ...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16050
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16050: [SPARK-18615][DOCS] Switch to multi-line doc to avoid a ...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16050
  
**[Test build #69285 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69285/consoleFull)**
 for PR 16050 at commit 
[`d62ac24`](https://github.com/apache/spark/commit/d62ac242e741a8c2f55b3a4a42090609a56fcd64).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16044: [Spark-18614][SQL] Incorrect predicate pushdown from Exi...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16044
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69287/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16044: [Spark-18614][SQL] Incorrect predicate pushdown from Exi...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16044
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16044: [Spark-18614][SQL] Incorrect predicate pushdown from Exi...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16044
  
**[Test build #69287 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69287/consoleFull)**
 for PR 16044 at commit 
[`7c3bca0`](https://github.com/apache/spark/commit/7c3bca07585980a7d3425d5112de5ab7d67cb076).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15954
  
**[Test build #69293 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69293/consoleFull)**
 for PR 15954 at commit 
[`d6200d1`](https://github.com/apache/spark/commit/d6200d13da13c01b2c177d77b1596ce8bbfeefd7).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15954
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/15954
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69293/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15848: [SPARK-9487] Use the same num. worker threads in Java/Sc...

2016-11-28 Thread skanjila

Github user skanjila commented on the issue:

https://github.com/apache/spark/pull/15848
  
I've changed the underlying trait for LogisticRegressionSuite and 
OneVsRestSuite to use local[2], will add in python changes next , want to get 
this PR across with this set of changes


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16049: [SPARK-16282][SQL] Follow-up: remove "percentile"...

2016-11-28 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/16049


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15848: [SPARK-9487] Use the same num. worker threads in Java/Sc...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/15848
  
**[Test build #69297 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69297/consoleFull)**
 for PR 15848 at commit 
[`eb0936b`](https://github.com/apache/spark/commit/eb0936b1491f021393412d0a19f6effad16ccb29).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16049: [SPARK-16282][SQL] Follow-up: remove "percentile" from t...

2016-11-28 Thread gatorsmile

Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/16049
  
Merging to master/2.1! Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16045: [SPARK-18553][CORE] Fix leak of TaskSetManager following...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16045
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69286/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16045: [SPARK-18553][CORE] Fix leak of TaskSetManager following...

2016-11-28 Thread AmplabJenkins

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/16045
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16045: [SPARK-18553][CORE] Fix leak of TaskSetManager following...

2016-11-28 Thread SparkQA

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/16045
  
**[Test build #69286 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69286/consoleFull)**
 for PR 16045 at commit 
[`9689763`](https://github.com/apache/spark/commit/96897631dfb5efc9cbf94044ea4eb0a8764ae78c).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

1 2 3 4 5 6 7 >

1 - 100 of 676 matches

Mail list logo