[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168339017
  
**[Test build #2289 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2289/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).
 * This patch **fails build dependency tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12600][SQL] Remove deprecated methods i...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10548#issuecomment-168345746
  
**[Test build #48572 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48572/consoleFull)**
 for PR 10548 at commit 
[`439bec5`](https://github.com/apache/spark/commit/439bec5062059bcfe9ee8ffba8e38b89ac77e7ae).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: size RollingPolicy shouldRollover loginfo shou...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10549#issuecomment-168361621
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168367522
  
test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168368188
  
**[Test build #48577 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48577/consoleFull)**
 for PR 10551 at commit 
[`5f67a74`](https://github.com/apache/spark/commit/5f67a7476365828a5eb920b3af3ddad312799509).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168368843
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48579/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168368840
  
**[Test build #48579 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48579/consoleFull)**
 for PR 10550 at commit 
[`797567d`](https://github.com/apache/spark/commit/797567d44e3a27cceff6a98a164f4aef433b8e5e).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168368842
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168367565
  
**[Test build #2295 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2295/consoleFull)**
 for PR 8743 at commit 
[`a489a33`](https://github.com/apache/spark/commit/a489a336b6ef4364f834262738f877596b67297c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168368807
  
**[Test build #48579 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48579/consoleFull)**
 for PR 10550 at commit 
[`797567d`](https://github.com/apache/spark/commit/797567d44e3a27cceff6a98a164f4aef433b8e5e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168368468
  
**[Test build #48578 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48578/consoleFull)**
 for PR 10551 at commit 
[`5f67a74`](https://github.com/apache/spark/commit/5f67a7476365828a5eb920b3af3ddad312799509).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[jegonzal]

Github user jegonzal commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168368506
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168368976
  
OK merging again.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168367746
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48576/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168367738
  
**[Test build #48576 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48576/consoleFull)**
 for PR 10551 at commit 
[`d9b5411`](https://github.com/apache/spark/commit/d9b54115f6dbd3e815963cec702a4a8aae28c154).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168367745
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/10525


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168371292
  
**[Test build #2295 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2295/consoleFull)**
 for PR 8743 at commit 
[`a489a33`](https://github.com/apache/spark/commit/a489a336b6ef4364f834262738f877596b67297c).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12594] [SQL] Outer Join Conversion: Out...

[gatorsmile]

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/10542#discussion_r48680023
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
 ---
@@ -769,6 +770,63 @@ object ReorderJoin extends Rule[LogicalPlan] with 
PredicateHelper {
 }
 
 /**
+ * Conversion of outer joins, if the local predicates can restrict the 
result sets so that
+ * all null-supplying rows are eliminated
+ *
+ * - full outer -> inner if both sides have such local predicates
+ * - left outer -> inner if the right side has such local predicates
+ * - right outer -> inner if the left side has such local predicates
+ * - full outer -> left outer if only the left side has such local 
predicates
+ * - full outer -> right outer if only the right side has such local 
predicates
+ *
+ * This rule should be executed before pushing down the Filter
+ */
+object OuterJoinConversion extends Rule[LogicalPlan] with PredicateHelper {
--- End diff --

You are right. `OuterJoinElimination` might sound better. Let me rename it 
today. 

Since `full outer` is `union distinct` of `left outer` and `right outer`, 
we are removing `right outer` from `full outer` in conversion from `full outer` 
to `left outer`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12600][SQL] Remove deprecated methods i...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10548#issuecomment-168346201
  
**[Test build #48572 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48572/consoleFull)**
 for PR 10548 at commit 
[`439bec5`](https://github.com/apache/spark/commit/439bec5062059bcfe9ee8ffba8e38b89ac77e7ae).
 * This patch **fails MiMa tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168363206
  
**[Test build #48575 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48575/consoleFull)**
 for PR 10551 at commit 
[`d9b5411`](https://github.com/apache/spark/commit/d9b54115f6dbd3e815963cec702a4a8aae28c154).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168364467
  
**[Test build #2294 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2294/consoleFull)**
 for PR 8743 at commit 
[`a489a33`](https://github.com/apache/spark/commit/a489a336b6ef4364f834262738f877596b67297c).
 * This patch **fails PySpark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[gatorsmile]

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168365001
  
Failed cases are not related to the code changes. 
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12600][SQL] Remove deprecated methods i...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10548#issuecomment-168345726
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48571/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12600][SQL] Remove deprecated methods i...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10548#issuecomment-168345725
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12600][SQL] Remove deprecated methods i...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10548#issuecomment-168346207
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48572/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12600][SQL] Remove deprecated methods i...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10548#issuecomment-168346206
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12599][MLlib][SQL] Remove the use of ca...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10547#issuecomment-168357144
  
**[Test build #2292 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2292/consoleFull)**
 for PR 10547 at commit 
[`f6eb35a`](https://github.com/apache/spark/commit/f6eb35a74c5a51951a6b34dcef5793b58e7d00bf).
 * This patch **fails PySpark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168367011
  
**[Test build #48576 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48576/consoleFull)**
 for PR 10551 at commit 
[`d9b5411`](https://github.com/apache/spark/commit/d9b54115f6dbd3e815963cec702a4a8aae28c154).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168338807
  
**[Test build #2289 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2289/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12592][SQL][WIP] Converts resolved logi...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10541#issuecomment-168338815
  
The jira ticket is linked incorrectly I think.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168338761
  
**[Test build #2287 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2287/consoleFull)**
 for PR 8743 at commit 
[`17b6953`](https://github.com/apache/spark/commit/17b695334e07ec9a58e766bd737d1c706f714109).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12592][SQL][TEST] Don't mute Spark logg...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/10540


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SPA...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10511#issuecomment-168339460
  
I took a quick look and I'm going to merge this because I'm working on 
something that conflicts with it. It would be great if @nongli or @JoshRosen 
can review this more carefully post-hoc.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12597] [ML] Use udf to replace callUDF ...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10544#issuecomment-168361036
  
This works too. I have a separate pull request that adds a new API for 
this: https://github.com/apache/spark/pull/10547



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SPA...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/10511


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9608#issuecomment-168351836
  
**[Test build #48573 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48573/consoleFull)**
 for PR 9608 at commit 
[`f88ee99`](https://github.com/apache/spark/commit/f88ee99e36c6b6c6521d1e7d14a13e18114d17fe).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9608#issuecomment-168351853
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9608#issuecomment-168351854
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48573/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168361139
  
**[Test build #2294 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2294/consoleFull)**
 for PR 8743 at commit 
[`a489a33`](https://github.com/apache/spark/commit/a489a336b6ef4364f834262738f877596b67297c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168364748
  
**[Test build #48575 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48575/consoleFull)**
 for PR 10551 at commit 
[`d9b5411`](https://github.com/apache/spark/commit/d9b54115f6dbd3e815963cec702a4a8aae28c154).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168364754
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48575/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Fixed bug in setMinPartitions

[datafarmer]

Github user datafarmer commented on the pull request:

https://github.com/apache/spark/pull/10546#issuecomment-168333823
  
@srowen I guess that I should have created a JIRA ticket first.  I just 
created one: SPARK-12598


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168339224
  
@JoshRosen I pushed a commit directly to disable test-dependencies.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12600][SQL] Remove deprecated methods i...

[rxin]

Github user rxin closed the pull request at:

https://github.com/apache/spark/pull/10548


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168362804
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168362805
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48574/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168362743
  
**[Test build #48574 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48574/consoleFull)**
 for PR 10550 at commit 
[`61512ac`](https://github.com/apache/spark/commit/61512acb2dba276b2bbd1bca5d22ff2474f6def5).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168362802
  
**[Test build #48574 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48574/consoleFull)**
 for PR 10550 at commit 
[`61512ac`](https://github.com/apache/spark/commit/61512acb2dba276b2bbd1bca5d22ff2474f6def5).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12599][MLlib][SQL] Remove the use of ca...

[rxin]

GitHub user rxin opened a pull request:

https://github.com/apache/spark/pull/10547

[SPARK-12599][MLlib][SQL] Remove the use of callUDF in MLlib

callUDF has been deprecated. However, we do not have an alternative for 
users to specify the output data type without type tags. This pull request 
introduced a new API for that, and replaces the invocation of the deprecated 
callUDF with that. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rxin/spark SPARK-12599

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10547.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10547


commit f6eb35a74c5a51951a6b34dcef5793b58e7d00bf
Author: Reynold Xin 
Date:   2016-01-01T22:16:04Z

[SPARK-12599][MLlib][SQL] Remove the use of the deprecated callUDF in MLlib.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9608#issuecomment-168345176
  
**[Test build #48573 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48573/consoleFull)**
 for PR 9608 at commit 
[`f88ee99`](https://github.com/apache/spark/commit/f88ee99e36c6b6c6521d1e7d14a13e18114d17fe).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168346039
  
test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168356745
  
**[Test build #2291 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2291/consoleFull)**
 for PR 8743 at commit 
[`a489a33`](https://github.com/apache/spark/commit/a489a336b6ef4364f834262738f877596b67297c).
 * This patch **fails PySpark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[jegonzal]

Github user jegonzal commented on the pull request:

https://github.com/apache/spark/pull/10550#issuecomment-168362426
  
@davies and @JoshRosen let me know what you think of this design.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Adding zipPartitions to PySpark

[jegonzal]

GitHub user jegonzal opened a pull request:

https://github.com/apache/spark/pull/10550

Adding zipPartitions to PySpark

The following working WIP adds support for `zipPartitions` to PySpark.  
This is accomplished by modifying the PySpark `worker` (in both daemon and 
non-deamon mode) to open a second socket back to the Spark process.  The second 
socket is used to send tuple from the second iterator in `zipPartitions` 
enabling the user defined function to pull tuples from both iterators at 
different rates without requiring a back-and-forth protocol over the primary 
socket.  The single socket protocol design was considered but creates issues 
with the built-in serializers and would require much larger changes.  The 
second socket is always created at the launch of the worker process and is 
simply ignored if it is not needed.  



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jegonzal/spark multi_iterator_pyspark

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10550.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10550


commit 70650ab94ae5dceca2dd6a970035d45dffdce2b1
Author: Joseph Gonzalez 
Date:   2016-01-02T01:40:10Z

compiling prototype

commit 61512acb2dba276b2bbd1bca5d22ff2474f6def5
Author: Joseph Gonzalez 
Date:   2016-01-02T03:51:40Z

addressing a bug where sockets could get created multiple times




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12594] [SQL] Outer Join Elimination: Ou...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10542#issuecomment-168336750
  
**[Test build #48569 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48569/consoleFull)**
 for PR 10542 at commit 
[`c04b53b`](https://github.com/apache/spark/commit/c04b53b0da168bba2a50010d42a13c04efc9a2bd).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168339295
  
**[Test build #2290 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2290/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168339975
  
**[Test build #2287 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2287/consoleFull)**
 for PR 8743 at commit 
[`17b6953`](https://github.com/apache/spark/commit/17b695334e07ec9a58e766bd737d1c706f714109).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12533][SQL] hiveContext.table() throws ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10529#issuecomment-168340743
  
**[Test build #2288 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2288/consoleFull)**
 for PR 10529 at commit 
[`76a9623`](https://github.com/apache/spark/commit/76a9623fed32eeb3c4cd34e6a7ad54ffefca1b69).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

[radekg]

Github user radekg commented on the pull request:

https://github.com/apache/spark/pull/9608#issuecomment-168344020
  
I will need to verify this patch with torrent broadcast.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12599][MLlib][SQL] Remove the use of ca...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10547#issuecomment-168351377
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48570/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12599][MLlib][SQL] Remove the use of ca...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10547#issuecomment-168351376
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: size RollingPolicy shouldRollover loginfo shou...

[ouyangshourui]

GitHub user ouyangshourui opened a pull request:

https://github.com/apache/spark/pull/10549

size RollingPolicy shouldRollover loginfo should change to logDebug

https://issues.apache.org/jira/browse/SPARK-12601
when using size RollingPolicythis code cause worker output a large of log 
.it shoud be changed to logDebug
the code as following:
 /** Should rollover if the next set of bytes is going to exceed the size 
limit */
  def shouldRollover(bytesToBeWritten: Long): Boolean = {
logDebug(s"$bytesToBeWritten + $bytesWrittenSinceRollover > 
$rolloverSizeBytes")
bytesToBeWritten + bytesWrittenSinceRollover > rolloverSizeBytes
  }

-->

 /** Should rollover if the next set of bytes is going to exceed the size 
limit */
  def shouldRollover(bytesToBeWritten: Long): Boolean = {
logDebug(s"$bytesToBeWritten + $bytesWrittenSinceRollover > 
$rolloverSizeBytes")
bytesToBeWritten + bytesWrittenSinceRollover > rolloverSizeBytes
  }


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ouyangshourui/spark rollingsizeloginfo

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10549.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10549


commit ec1fc25f12dfd77146be3ad3931fbdcc1ff680eb
Author: ouyangshourui <494165...@qq.com>
Date:   2016-01-02T03:41:17Z

size RollingPolicy shouldRollover loginfo should change to logDebug




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168364752
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12597] [ML] Use udf to replace callUDF ...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10544#issuecomment-168364756
  
BTW is transformFunc a public API that custom transformers are supposed to 
implement? If it is, this is technically an API breaking change you are making.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[gatorsmile]

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/10551#issuecomment-168366180
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12592][SQL][TEST] Don't mute Spark logg...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10540#issuecomment-168338785
  
I've merged this.
 Thanks.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12533][SQL] hiveContext.table() throws ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10529#issuecomment-168338746
  
**[Test build #2288 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2288/consoleFull)**
 for PR 10529 at commit 
[`76a9623`](https://github.com/apache/spark/commit/76a9623fed32eeb3c4cd34e6a7ad54ffefca1b69).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12594] [SQL] Outer Join Elimination: Ou...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10542#issuecomment-168341219
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48569/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12594] [SQL] Outer Join Elimination: Ou...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10542#issuecomment-168341218
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12594] [SQL] Outer Join Elimination: Ou...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10542#issuecomment-168341197
  
**[Test build #48569 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48569/consoleFull)**
 for PR 10542 at commit 
[`c04b53b`](https://github.com/apache/spark/commit/c04b53b0da168bba2a50010d42a13c04efc9a2bd).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12600][SQL] Remove deprecated methods i...

[rxin]

GitHub user rxin opened a pull request:

https://github.com/apache/spark/pull/10548

[SPARK-12600][SQL] Remove deprecated methods in Spark SQL



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rxin/spark SPARK-12600

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10548.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10548


commit 8dd6fe63f6c3bbcaa1b37d8e1e8c9bfdef6906b8
Author: Reynold Xin 
Date:   2016-01-01T22:16:04Z

[SPARK-12599][MLlib][SQL] Remove the use of the deprecated callUDF in MLlib.

commit 0f4e9a62f78f0dc43a3e191b92f9b01e4c4a4304
Author: Reynold Xin 
Date:   2015-12-29T02:31:22Z

Remove deprecated methods in SQL/DataFrame.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12599][MLlib][SQL] Remove the use of ca...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10547#issuecomment-168343118
  
**[Test build #48570 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48570/consoleFull)**
 for PR 10547 at commit 
[`f6eb35a`](https://github.com/apache/spark/commit/f6eb35a74c5a51951a6b34dcef5793b58e7d00bf).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12599][MLlib][SQL] Remove the use of ca...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10547#issuecomment-168351345
  
**[Test build #48570 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48570/consoleFull)**
 for PR 10547 at commit 
[`f6eb35a`](https://github.com/apache/spark/commit/f6eb35a74c5a51951a6b34dcef5793b58e7d00bf).
 * This patch **fails PySpark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168351298
  
**[Test build #2290 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2290/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168353114
  
**[Test build #2291 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2291/consoleFull)**
 for PR 8743 at commit 
[`a489a33`](https://github.com/apache/spark/commit/a489a336b6ef4364f834262738f877596b67297c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12599][MLlib][SQL] Remove the use of ca...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10547#issuecomment-168353154
  
**[Test build #2292 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2292/consoleFull)**
 for PR 10547 at commit 
[`f6eb35a`](https://github.com/apache/spark/commit/f6eb35a74c5a51951a6b34dcef5793b58e7d00bf).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12599][MLlib][SQL] Remove the use of ca...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10547#issuecomment-168361123
  
**[Test build #2293 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2293/consoleFull)**
 for PR 10547 at commit 
[`f6eb35a`](https://github.com/apache/spark/commit/f6eb35a74c5a51951a6b34dcef5793b58e7d00bf).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12286] [SPARK-12290] [SPARK-12294] [SPA...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10511#issuecomment-168361081
  
Actually I'm going to revert this for now. I've seen two separate pull 
requests that failed python tests.


https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2292/consoleFull

and


https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2291/consoleFull

Neither of them had anything to do with the failure and my hunch is that 
this is the one causing it.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12602] [SQL] Join Reordering: Pushing I...

[gatorsmile]

GitHub user gatorsmile opened a pull request:

https://github.com/apache/spark/pull/10551

[SPARK-12602] [SQL] Join Reordering: Pushing Inner Join Through Left/Right 
Outer Join

This PR is to push `Inner Join` through `Left/Right Outer Join`. This can 
reduce the number of processed rows since the `Inner Join` always can generate 
less rows than `Left/Right Outer Join`. 

This PR can improve the query performance, if applicable.

For example, 

TODO: added test cases into optimizer buckets.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gatorsmile/spark outerInnerReordering

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10551.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10551


commit 41703c2c3570137d67d8ead873fda25ce11075e1
Author: gatorsmile 
Date:   2016-01-02T04:04:27Z

Merge remote-tracking branch 'upstream/master' into outerInnerReordering

commit 7237f618b8f0728a1f1585e1f4db155e6d38519b
Author: gatorsmile 
Date:   2016-01-02T04:04:38Z

reorder outer and inner joins

commit d9b54115f6dbd3e815963cec702a4a8aae28c154
Author: gatorsmile 
Date:   2016-01-02T04:18:32Z

renaming.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12599][MLlib][SQL] Remove the use of ca...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10547#issuecomment-168364250
  
**[Test build #2293 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2293/consoleFull)**
 for PR 10547 at commit 
[`f6eb35a`](https://github.com/apache/spark/commit/f6eb35a74c5a51951a6b34dcef5793b58e7d00bf).
 * This patch **fails PySpark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168294581
  
Sorry I kept merging pull requests that made your life harder. Can you 
bring it up to date again? I promise this is the last one.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-7689][WIP] Remove TTL-based metadata cl...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10534#issuecomment-168294573
  
LGTM. We should let @tdas take a look at this though.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168302341
  
**[Test build #2286 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2286/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168294441
  
I triggered 3 more runs.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168294453
  
**[Test build #2284 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2284/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168294469
  
**[Test build #2286 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2286/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168294465
  
**[Test build #2285 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2285/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10470#issuecomment-168294357
  
Thanks - I've merged this.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12409][SPARK-12387][SPARK-12391][SQL] R...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/10470


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12594] [SQL] Outer Join Conversion: Out...

[rxin]

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/10542#discussion_r48676822
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
 ---
@@ -769,6 +770,63 @@ object ReorderJoin extends Rule[LogicalPlan] with 
PredicateHelper {
 }
 
 /**
+ * Conversion of outer joins, if the local predicates can restrict the 
result sets so that
+ * all null-supplying rows are eliminated
+ *
+ * - full outer -> inner if both sides have such local predicates
+ * - left outer -> inner if the right side has such local predicates
+ * - right outer -> inner if the left side has such local predicates
+ * - full outer -> left outer if only the left side has such local 
predicates
+ * - full outer -> right outer if only the right side has such local 
predicates
+ *
+ * This rule should be executed before pushing down the Filter
+ */
+object OuterJoinConversion extends Rule[LogicalPlan] with PredicateHelper {
--- End diff --

is there a better name for this? this is a form of strength reduction right?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

[rxin]

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/10435#discussion_r48676826
  
--- Diff: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
 ---
@@ -278,6 +278,7 @@ object FunctionRegistry {
 // misc functions
 expression[Crc32]("crc32"),
 expression[Md5]("md5"),
+expression[Murmur3Hash]("murmur3_hash"),
--- End diff --

can you give me a list? i think we should consider just blacklisting them 
...



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168294519
  
cc @JoshRosen dependency failed again


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168294492
  
**[Test build #2285 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2285/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).
 * This patch **fails build dependency tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10525#issuecomment-168301945
  
**[Test build #2284 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2284/consoleFull)**
 for PR 10525 at commit 
[`e3e2f72`](https://github.com/apache/spark/commit/e3e2f72d8080f8e0040be351032c7e3b24289c77).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-10180] [SQL] JDBC datasource are not pr...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/8743#issuecomment-168306690
  
Could anybody please type "test this please" for this PR? I can't trigger a 
test for this PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12597] [ML] Use udf to replace callUDF ...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10544#issuecomment-168312778
  
**[Test build #48565 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48565/consoleFull)**
 for PR 10544 at commit 
[`b4c4329`](https://github.com/apache/spark/commit/b4c4329df5935c8ba4a4fa77ad0a8b2a9487a4af).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12513] [Streaming] SocketReceiver hang ...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/10464#discussion_r48678510
  
--- Diff: 
streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala
 ---
@@ -51,29 +51,44 @@ class SocketReceiver[T: ClassTag](
 storageLevel: StorageLevel
   ) extends Receiver[T](storageLevel) with Logging {
 
+  private var socket: Socket = _
+
   def onStart() {
-// Start the thread that receives data over a connection
-new Thread("Socket Receiver") {
-  setDaemon(true)
-  override def run() { receive() }
-}.start()
+try {
+  logInfo(s"Connecting to $host:$port")
+  socket = new Socket(host, port)
+  logInfo(s"Connected to $host:$port")
+} catch {
+  case NonFatal(e) =>
--- End diff --

Hm, I suppose now you're restarting here on any non-fatal error in 
connecting, not just `ConnectException`. Maybe that's OK. It's simpler to 
return here rather than then have to check if the socket initialized. Also the 
info logs seem like they don't belong in the try-catch; they can't fail.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12513] [Streaming] SocketReceiver hang ...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/10464#discussion_r48678511
  
--- Diff: 
streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala
 ---
@@ -51,29 +51,44 @@ class SocketReceiver[T: ClassTag](
 storageLevel: StorageLevel
   ) extends Receiver[T](storageLevel) with Logging {
 
+  private var socket: Socket = _
+
   def onStart() {
-// Start the thread that receives data over a connection
-new Thread("Socket Receiver") {
-  setDaemon(true)
-  override def run() { receive() }
-}.start()
+try {
+  logInfo(s"Connecting to $host:$port")
+  socket = new Socket(host, port)
+  logInfo(s"Connected to $host:$port")
+} catch {
+  case NonFatal(e) =>
+restart(s"Error connecting to $host:$port", e)
+}
+
+if (socket != null && socket.isConnected) {
+  // Start the thread that receives data over a connection
+  new Thread("Socket Receiver") {
+setDaemon(true)
+override def run() { receive() }
+  }.start()
+}
   }
 
   def onStop() {
-// There is nothing much to do as the thread calling receive()
-// is designed to stop by itself isStopped() returns false
+//in case restart thread close it twice
+synchronized {
+  if (socket != null) {
+socket.close()
+socket = null
+logInfo(s"Closed socket to $host:$port")
+  }
+}
   }
 
   /** Create a socket connection and receive data until receiver is 
stopped */
   def receive() {
-var socket: Socket = null
 try {
-  logInfo("Connecting to " + host + ":" + port)
-  socket = new Socket(host, port)
-  logInfo("Connected to " + host + ":" + port)
-  val iterator = bytesToObjects(socket.getInputStream())
+  val iterator = bytesToObjects(socket.getInputStream)
--- End diff --

Nit: the empty parens were correct here; it has a side effect


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [Spark-10625] [SQL] Spark SQL JDBC read/write ...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/8785#issuecomment-168313849
  
Weird, OK. This LGTM.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org