date:20151219

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-19 Thread gatorsmile

GitHub user gatorsmile opened a pull request:

https://github.com/apache/spark/pull/10393

[SPARK-12441] [SQL] Fixing missingInput in Generate

When explain any plan with Generate, we will see an exclamation mark in the 
plan. Normally, when we see this mark, it means the plan has an error. This PR 
is to correct the `missingInput` in `Generate`.

For example, 
```scala
val df = Seq((1, "a b c"), (2, "a b"), (3, "a")).toDF("number", "letters")
val df2 =
  df.explode('letters) {
case Row(letters: String) => letters.split(" ").map(Tuple1(_)).toSeq
  }

df2.explain(true)
```
Before the fix, the plan is like 
```
== Parsed Logical Plan ==
'Generate UserDefinedGenerator('letters), true, false, None
+- Project [_1#0 AS number#2,_2#1 AS letters#3]
   +- LocalRelation [_1#0,_2#1], [[1,a b c],[2,a b],[3,a]]

== Analyzed Logical Plan ==
number: int, letters: string, _1: string
Generate UserDefinedGenerator(letters#3), true, false, None, [_1#8]
+- Project [_1#0 AS number#2,_2#1 AS letters#3]
   +- LocalRelation [_1#0,_2#1], [[1,a b c],[2,a b],[3,a]]

== Optimized Logical Plan ==
Generate UserDefinedGenerator(letters#3), true, false, None, [_1#8]
+- LocalRelation [number#2,letters#3], [[1,a b c],[2,a b],[3,a]]

== Physical Plan ==
!Generate UserDefinedGenerator(letters#3), true, false, 
[number#2,letters#3,_1#8]
+- LocalTableScan [number#2,letters#3], [[1,a b c],[2,a b],[3,a]]
```

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gatorsmile/spark generateExplain

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10393.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10393


commit 21071d1bc9fe0d5869fa94f21f617ee5dbaae390
Author: gatorsmile 
Date:   2015-12-19T18:33:59Z

correct missingInput.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10393#issuecomment-166017392
  
**[Test build #48056 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48056/consoleFull)**
 for PR 10393 at commit 
[`22d8ba5`](https://github.com/apache/spark/commit/22d8ba5bf1d4ba9e25de02a4619ff3718f257c1b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12413] Fix Mesos ZK persistence

2015-12-19 Thread keithchambers

Github user keithchambers commented on the pull request:

https://github.com/apache/spark/pull/10366#issuecomment-166001787
  
@andrewor14 makes sense. Thank you!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11562][SQL] Provide option to switch Sq...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9553#issuecomment-166017558
  
**[Test build #48057 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48057/consoleFull)**
 for PR 9553 at commit 
[`969008e`](https://github.com/apache/spark/commit/969008ef85723eb35d4ab038c9158eb3ea12b031).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11562][SQL] Provide option to switch Sq...

2015-12-19 Thread yhuai

Github user yhuai commented on a diff in the pull request:

https://github.com/apache/spark/pull/9553#discussion_r48095511
  
--- Diff: 
repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkILoop.scala ---
@@ -132,6 +132,7 @@ class SparkILoop(
   @DeveloperApi
   var sparkContext: SparkContext = _
   var sqlContext: SQLContext = _
+  var useHiveContext: Boolean = _
--- End diff --

Can we make it a local field in `createSQLContext`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11562][SQL] Provide option to switch Sq...

2015-12-19 Thread yhuai

Github user yhuai commented on a diff in the pull request:

https://github.com/apache/spark/pull/9553#discussion_r48095491
  
--- Diff: 
repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkILoop.scala ---
@@ -1026,17 +1027,30 @@ class SparkILoop(
 
   @DeveloperApi
   def createSQLContext(): SQLContext = {
-val name = "org.apache.spark.sql.hive.HiveContext"
+useHiveContext = 
sparkContext.getConf.getBoolean("spark.sql.useHiveContext", true)
+val name = {
+  if (useHiveContext) "org.apache.spark.sql.hive.HiveContext"
+  else "org.apache.spark.sql.SQLContext"
+}
+
 val loader = Utils.getContextOrSparkClassLoader
 try {
   sqlContext = 
loader.loadClass(name).getConstructor(classOf[SparkContext])
 .newInstance(sparkContext).asInstanceOf[SQLContext]
-  logInfo("Created sql context (with Hive support)..")
+  if (useHiveContext) {
+logInfo("Created sql context (with Hive support). To use 
sqlContext (without Hive), " +
+  "set spark.sql.useHiveContext to false before launching 
spark-shell.")
+  }
+  else {
+logInfo("Created sql context.")
+  }
 }
 catch {
-  case _: java.lang.ClassNotFoundException | _: 
java.lang.NoClassDefFoundError =>
+  case _: java.lang.ClassNotFoundException | _: 
java.lang.NoClassDefFoundError
+if useHiveContext =>
 sqlContext = new SQLContext(sparkContext)
-logInfo("Created sql context..")
+logInfo("Created sql context without Hive support, " +
+  "build Spark with -Phive to enable Hive support.")
--- End diff --

`Created sql context without Hive support. To enable Hive support, build 
Spark with -Phive profile.`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10393#issuecomment-166016740
  
**[Test build #48055 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48055/consoleFull)**
 for PR 10393 at commit 
[`21071d1`](https://github.com/apache/spark/commit/21071d1bc9fe0d5869fa94f21f617ee5dbaae390).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11562][SQL] Provide option to switch Sq...

2015-12-19 Thread xguo27

Github user xguo27 commented on a diff in the pull request:

https://github.com/apache/spark/pull/9553#discussion_r48095871
  
--- Diff: 
repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkILoop.scala ---
@@ -1026,17 +1027,30 @@ class SparkILoop(
 
   @DeveloperApi
   def createSQLContext(): SQLContext = {
-val name = "org.apache.spark.sql.hive.HiveContext"
+useHiveContext = 
sparkContext.getConf.getBoolean("spark.sql.useHiveContext", true)
+val name = {
+  if (useHiveContext) "org.apache.spark.sql.hive.HiveContext"
+  else "org.apache.spark.sql.SQLContext"
+}
+
 val loader = Utils.getContextOrSparkClassLoader
 try {
   sqlContext = 
loader.loadClass(name).getConstructor(classOf[SparkContext])
 .newInstance(sparkContext).asInstanceOf[SQLContext]
-  logInfo("Created sql context (with Hive support)..")
+  if (useHiveContext) {
+logInfo("Created sql context (with Hive support). To use 
sqlContext (without Hive), " +
+  "set spark.sql.useHiveContext to false before launching 
spark-shell.")
+  }
+  else {
+logInfo("Created sql context.")
+  }
 }
 catch {
-  case _: java.lang.ClassNotFoundException | _: 
java.lang.NoClassDefFoundError =>
+  case _: java.lang.ClassNotFoundException | _: 
java.lang.NoClassDefFoundError
+if useHiveContext =>
 sqlContext = new SQLContext(sparkContext)
-logInfo("Created sql context..")
+logInfo("Created sql context without Hive support, " +
+  "build Spark with -Phive to enable Hive support.")
--- End diff --

When -Phive is used (which provides necessary hive jars) and an exception 
other than ClassNotFound/NoClassDefFound occured, now how we handle it is to 
let the exception be propagated without creating an alternative SqlContext.

Do you mean by this case, we should catch -> log -> re-throw? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11562][SQL] Provide option to switch Sq...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9553#issuecomment-166025910
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11562][SQL] Provide option to switch Sq...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/9553#issuecomment-166025617
  
**[Test build #48057 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48057/consoleFull)**
 for PR 9553 at commit 
[`969008e`](https://github.com/apache/spark/commit/969008ef85723eb35d4ab038c9158eb3ea12b031).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11562][SQL] Provide option to switch Sq...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/9553#issuecomment-166025914
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48057/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166032102
  
**[Test build #48059 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48059/consoleFull)**
 for PR 10368 at commit 
[`b65aeb6`](https://github.com/apache/spark/commit/b65aeb67ea3a8a8bfabead4a7ba86a2dfeb83fa0).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-165997773
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48053/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-165997757
  
**[Test build #48053 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48053/consoleFull)**
 for PR 10368 at commit 
[`cdd07e2`](https://github.com/apache/spark/commit/cdd07e2f61ba8f1f58954fe1c689c8096ed07985).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-165997771
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10393#issuecomment-166022745
  
**[Test build #48056 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48056/consoleFull)**
 for PR 10393 at commit 
[`22d8ba5`](https://github.com/apache/spark/commit/22d8ba5bf1d4ba9e25de02a4619ff3718f257c1b).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10393#issuecomment-166022788
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10393#issuecomment-166022790
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48056/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11562][SQL] Provide option to switch Sq...

2015-12-19 Thread yhuai

Github user yhuai commented on a diff in the pull request:

https://github.com/apache/spark/pull/9553#discussion_r48095548
  
--- Diff: 
repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkILoop.scala ---
@@ -1026,17 +1027,30 @@ class SparkILoop(
 
   @DeveloperApi
   def createSQLContext(): SQLContext = {
-val name = "org.apache.spark.sql.hive.HiveContext"
+useHiveContext = 
sparkContext.getConf.getBoolean("spark.sql.useHiveContext", true)
+val name = {
+  if (useHiveContext) "org.apache.spark.sql.hive.HiveContext"
+  else "org.apache.spark.sql.SQLContext"
+}
+
 val loader = Utils.getContextOrSparkClassLoader
 try {
   sqlContext = 
loader.loadClass(name).getConstructor(classOf[SparkContext])
 .newInstance(sparkContext).asInstanceOf[SQLContext]
-  logInfo("Created sql context (with Hive support)..")
+  if (useHiveContext) {
+logInfo("Created sql context (with Hive support). To use 
sqlContext (without Hive), " +
+  "set spark.sql.useHiveContext to false before launching 
spark-shell.")
+  }
+  else {
+logInfo("Created sql context.")
+  }
 }
 catch {
-  case _: java.lang.ClassNotFoundException | _: 
java.lang.NoClassDefFoundError =>
+  case _: java.lang.ClassNotFoundException | _: 
java.lang.NoClassDefFoundError
+if useHiveContext =>
 sqlContext = new SQLContext(sparkContext)
-logInfo("Created sql context..")
+logInfo("Created sql context without Hive support, " +
+  "build Spark with -Phive to enable Hive support.")
--- End diff --

It is possible that `-Phive` is used but there was an exception. Can we 
also update the log message for this case?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166028352
  
**[Test build #48058 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48058/consoleFull)**
 for PR 10368 at commit 
[`422ac1c`](https://github.com/apache/spark/commit/422ac1cfd5369e9a00e3d52693bb868048e30ba2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12440] - [core] - Avoid setCheckpoint w...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10392#issuecomment-166004451
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10156#issuecomment-166010389
  
**[Test build #48054 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48054/consoleFull)**
 for PR 10156 at commit 
[`65de259`](https://github.com/apache/spark/commit/65de25955d609852b7088f754bdece47f47cc470).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10156#issuecomment-166017714
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12349] [ML] Make spark.ml PCAModel load...

2015-12-19 Thread BenFradet

Github user BenFradet commented on a diff in the pull request:

https://github.com/apache/spark/pull/10327#discussion_r48093699
  
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/PCA.scala ---
@@ -167,14 +167,37 @@ object PCAModel extends MLReadable[PCAModel] {
 
 private val className = classOf[PCAModel].getName
 
+/**
+ * Loads a [[PCAModel]] from data the input path. Note that the model 
includes an
--- End diff --

isnt there a word missing like: "Loads a PCAModel from data **located at** 
the input path"?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12369][SQL]DataFrameReader fails on glo...

2015-12-19 Thread yanakad

Github user yanakad commented on the pull request:

https://github.com/apache/spark/pull/10379#issuecomment-166020526
  
@liancheng Would logging the fail paths at WARN or ERROR level be an 
acceptable compromise? I am not sure if you're advising that the fix is not 
good enough or if you're disagreeing that there is an issue.
I think the original behavior *is* a problem -- if you have paths like this 
/root/account=number/date='-mo'/... , you create a DF at the root level and 
you execute 'select * where account=nonexistent' you'd get an empty data frame. 
If you execute a query with where date in(mo1,mo2,mo3) and there is no mo3 
partition, you'd still get data for months1 & 2. On the other hand, if you try 
to create a DF at /root/account=nonexistent you'd get an exception. I have a 
very heavily partitioned space, which is why I am creating dataframes as low as 
possible, running into this problem when a partition path is missing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10393#issuecomment-166022587
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10393#issuecomment-166022588
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48055/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166028405
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166028404
  
**[Test build #48058 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48058/consoleFull)**
 for PR 10368 at commit 
[`422ac1c`](https://github.com/apache/spark/commit/422ac1cfd5369e9a00e3d52693bb868048e30ba2).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166028406
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48058/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12440] - [core] - Avoid setCheckpoint w...

2015-12-19 Thread pierre-borckmans

GitHub user pierre-borckmans opened a pull request:

https://github.com/apache/spark/pull/10392

[SPARK-12440] - [core] - Avoid setCheckpoint warning when directory is not 
local

In SparkContext method `setCheckpointDir`, a warning is issued when spark 
master is not local and the passed directory for the checkpoint dir appears to 
be local.

In practice, when relying on HDFS configuration file and using relative 
path (incomplete URI without HDFS scheme, ...), this warning should not be 
issued and might be confusing.

This PR creates a `FileSystem` with the given directory, and check whether 
it is local or not.
The warning is only issued if the directory is not local.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pierre-borckmans/spark 
SPARK-12440_CheckpointDir_Warning_NonLocal

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10392.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10392


commit bace5b98d20a9f7d6876b65120e6668aa064c948
Author: pierre-borckmans 
Date:   2015-12-19T16:55:00Z

AVOID setCheckpoint warning when file system is not local




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-2331] SparkContext.emptyRDD should retu...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10394#issuecomment-166033288
  
**[Test build #48062 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48062/consoleFull)**
 for PR 10394 at commit 
[`86e64e1`](https://github.com/apache/spark/commit/86e64e1a021947f57049ddbf64aef29f73e8523e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12270][SQL]remove empty space after get...

2015-12-19 Thread huaxingao

Github user huaxingao commented on the pull request:

https://github.com/apache/spark/pull/10262#issuecomment-166033287
  
@yhuai 
JDBCSuite uses H2 database.  It seems that for char(n) data type, either H2 
database doesn't pad, or the H2 JDBC driver already trims the empty space for 
ResultSet.getString.  So H2 database doesn't have this problem.  To show the 
problem, it will need DB2 and DB2 JDBC driver ( I guess Oracle has the same 
problem too) , but I don't think the test system has DB2 JDBC driver.  So I am 
guessing maybe no need to add the test? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166033913
  
**[Test build #2237 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2237/consoleFull)**
 for PR 10395 at commit 
[`39b59f0`](https://github.com/apache/spark/commit/39b59f0c071fcb33dd941b8b50412b33d7ef7c39).
 * This patch **fails Java style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-19 Thread gatorsmile

GitHub user gatorsmile opened a pull request:

https://github.com/apache/spark/pull/10396

[SPARK-12292] [SQL] Support UnsafeRow in Generator.

After the fix, unsafe->safe convertors are not inserted between Generate 
and its Child.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gatorsmile/spark GenerateUnsafe

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10396.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10396


commit a7edcdf15936cda334e0d83853e37e0bcd34d6ce
Author: gatorsmile 
Date:   2015-12-20T00:13:28Z

Support UnsafeRow in Generator.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10396#issuecomment-166046558
  
**[Test build #2238 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2238/consoleFull)**
 for PR 10396 at commit 
[`a7edcdf`](https://github.com/apache/spark/commit/a7edcdf15936cda334e0d83853e37e0bcd34d6ce).
 * This patch **fails executing the `dev/run-tests` script**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:\n  * 
`public class JavaTwitterHashTagJoinSentiments `\n  * `case class 
UnresolvedAlias(child: Expression, aliasName: Option[String] = None)`\n


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166046577
  
**[Test build #2240 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2240/consoleFull)**
 for PR 10395 at commit 
[`39b59f0`](https://github.com/apache/spark/commit/39b59f0c071fcb33dd941b8b50412b33d7ef7c39).
 * This patch **fails executing the `dev/run-tests` script**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10396#issuecomment-166046550
  
**[Test build #2238 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2238/consoleFull)**
 for PR 10396 at commit 
[`a7edcdf`](https://github.com/apache/spark/commit/a7edcdf15936cda334e0d83853e37e0bcd34d6ce).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166046523
  
**[Test build #2239 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2239/consoleFull)**
 for PR 10368 at commit 
[`facf246`](https://github.com/apache/spark/commit/facf2460e184900c66393edd3652607cb4e40b12).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166046574
  
**[Test build #2240 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2240/consoleFull)**
 for PR 10395 at commit 
[`39b59f0`](https://github.com/apache/spark/commit/39b59f0c071fcb33dd941b8b50412b33d7ef7c39).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166046525
  
**[Test build #2239 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2239/consoleFull)**
 for PR 10368 at commit 
[`facf246`](https://github.com/apache/spark/commit/facf2460e184900c66393edd3652607cb4e40b12).
 * This patch **fails executing the `dev/run-tests` script**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166061058
  
**[Test build #48068 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48068/consoleFull)**
 for PR 10368 at commit 
[`facf246`](https://github.com/apache/spark/commit/facf2460e184900c66393edd3652607cb4e40b12).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread rxin

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166068833
  
cc @pwendell 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12349] [ML] Make spark.ml PCAModel load...

2015-12-19 Thread BenFradet

Github user BenFradet commented on the pull request:

https://github.com/apache/spark/pull/10327#issuecomment-165997370
  
lgtm except one minor doc comment.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10156#issuecomment-166017716
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48054/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10156#issuecomment-166017682
  
**[Test build #48054 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48054/consoleFull)**
 for PR 10156 at commit 
[`65de259`](https://github.com/apache/spark/commit/65de25955d609852b7088f754bdece47f47cc470).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10393#issuecomment-166022565
  
**[Test build #48055 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48055/consoleFull)**
 for PR 10393 at commit 
[`21071d1`](https://github.com/apache/spark/commit/21071d1bc9fe0d5869fa94f21f617ee5dbaae390).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12062] [CORE] Change Master to asyc reb...

2015-12-19 Thread tedyu

Github user tedyu commented on the pull request:

https://github.com/apache/spark/pull/10284#issuecomment-166033110
  
I think the following exception seen in unit test run was related to this 
PR:
```
[info] - Simple replay (70 milliseconds)
java.lang.NullPointerException
at 
org.apache.spark.deploy.master.Master$$anonfun$asyncRebuildSparkUI$1.applyOrElse(Master.scala:982)
at 
org.apache.spark.deploy.master.Master$$anonfun$asyncRebuildSparkUI$1.applyOrElse(Master.scala:980)
at scala.concurrent.Future$$anonfun$onSuccess$1.apply(Future.scala:117)
at scala.concurrent.Future$$anonfun$onSuccess$1.apply(Future.scala:115)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at 
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)
at 
scala.concurrent.impl.ExecutionContextImpl$$anon$1.execute(ExecutionContextImpl.scala:133)
at 
scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
at 
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
at scala.concurrent.Promise$class.complete(Promise.scala:55)
at 
scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:153)
at 
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:23)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166059915
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166059903
  
**[Test build #48067 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48067/consoleFull)**
 for PR 10368 at commit 
[`facf246`](https://github.com/apache/spark/commit/facf2460e184900c66393edd3652607cb4e40b12).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-19 Thread gatorsmile

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/10396#issuecomment-166069977
  
retest it please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10398#issuecomment-166070873
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10398#issuecomment-166070849
  
**[Test build #48069 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48069/consoleFull)**
 for PR 10398 at commit 
[`a70e95c`](https://github.com/apache/spark/commit/a70e95c9109a753dd2528c5544ddd9ff86722397).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10398#issuecomment-166070874
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48069/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12443][SQL] encoderFor should support D...

2015-12-19 Thread viirya

GitHub user viirya opened a pull request:

https://github.com/apache/spark/pull/10399

[SPARK-12443][SQL] encoderFor should support Decimal

JIRA: https://issues.apache.org/jira/browse/SPARK-12443

`constructorFor` will call `dataTypeFor` to determine if a type is 
`ObjectType` or not. If there is not case for `Decimal`, it will be recognized 
as `ObjectType` and causes the bug.




You can merge this pull request into a Git repository by running:

$ git pull https://github.com/viirya/spark-1 fix-encoder-decimal

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10399.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10399


commit c8cdca50e725d4a5ed4926754e9ef04126cb2f34
Author: Liang-Chi Hsieh 
Date:   2015-12-20T07:30:38Z

Add Decimal support for encoder.

commit 6afb1a40f8bd2528a38e471c8f6d32fdb1aa91d3
Author: Liang-Chi Hsieh 
Date:   2015-12-20T07:44:05Z

Move test case.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Bump master version to 2.0.0-SNAPSHOT.

2015-12-19 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/10387


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread rxin

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166033075
  
cc @ankurdave 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread rxin

GitHub user rxin opened a pull request:

https://github.com/apache/spark/pull/10395

[SPARK-11808] Remove Bagel.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rxin/spark SPARK-11808

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10395.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10395


commit 39b59f0c071fcb33dd941b8b50412b33d7ef7c39
Author: Reynold Xin 
Date:   2015-11-18T06:24:30Z

[SPARK-11808] Remove Bagel.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-2331] SparkContext.emptyRDD should retu...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10394#issuecomment-166033995
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-2331] SparkContext.emptyRDD should retu...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10394#issuecomment-166033988
  
**[Test build #48062 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48062/consoleFull)**
 for PR 10394 at commit 
[`86e64e1`](https://github.com/apache/spark/commit/86e64e1a021947f57049ddbf64aef29f73e8523e).
 * This patch **fails MiMa tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-2331] SparkContext.emptyRDD should retu...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10394#issuecomment-166033996
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48062/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: HOTFIX: Build testing failed.

2015-12-19 Thread gatorsmile

GitHub user gatorsmile opened a pull request:

https://github.com/apache/spark/pull/10397

HOTFIX: Build testing failed. 

```
./dev/run-tests.py
  File "./dev/run-tests.py", line 532
if not changed_files or any(f.endswith(".py") for f in changed_files):
 ^
```
Unable to run the command: ./dev/run-tests.py

@rxin could you check if my fix works? Thanks!

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gatorsmile/spark disableJavaSyntaxChecking

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10397.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10397


commit 4475706874082d19fcc7acb687689fa8d353a8aa
Author: gatorsmile 
Date:   2015-12-20T00:41:37Z

comment it out.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-19 Thread gatorsmile

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/10396#issuecomment-166047424
  
retest it please



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11944][PYSPARK][MLLIB] python mllib.clu...

2015-12-19 Thread holdenk

Github user holdenk commented on a diff in the pull request:

https://github.com/apache/spark/pull/10150#discussion_r48097828
  
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -38,13 +38,158 @@
 from pyspark.mllib.util import Saveable, Loader, inherit_doc, JavaLoader, 
JavaSaveable
 from pyspark.streaming import DStream
 
-__all__ = ['KMeansModel', 'KMeans', 'GaussianMixtureModel', 
'GaussianMixture',
-   'PowerIterationClusteringModel', 'PowerIterationClustering',
-   'StreamingKMeans', 'StreamingKMeansModel',
+__all__ = ['BisectingKMeansModel', 'BisectingKMeans', 'KMeansModel', 
'KMeans',
+   'GaussianMixtureModel', 'GaussianMixture', 
'PowerIterationClusteringModel',
+   'PowerIterationClustering', 'StreamingKMeans', 
'StreamingKMeansModel',
'LDA', 'LDAModel']
 
 
 @inherit_doc
+class BisectingKMeansModel(JavaModelWrapper):
+"""
+.. note:: Experimental
+
+A clustering model derived from the bisecting k-means method.
+
+>>> data = array([0.0,0.0, 1.0,1.0, 9.0,8.0, 8.0,9.0]).reshape(4, 2)
+>>> bskm = BisectingKMeans()
+>>> model = bskm.run(sc.parallelize(data))
+>>> model.predict(array([0.0, 0.0])) == model.predict(array([0.0, 
0.0]))
+True
+>>> model.k
+4
+>>> model.computeCost(array([0.0, 0.0]))
+0.0
+>>> model.k == len(model.clusterCenters)
+True
+>>> model = bskm.setK(2).run(sc.parallelize(data))
+>>> model.predict(array([0.0, 0.0])) == model.predict(array([1.0, 
1.0]))
+True
+>>> model.k
+2
+"""
+
+@property
+def clusterCenters(self):
+"""Get the cluster centers, represented as a list of NumPy 
arrays."""
+return [c.toArray() for c in self.call("clusterCenters")]
+
+@property
+def k(self):
+"""Get the number of clusters"""
+return self.call("k")
+
+def predict(self, x):
+"""
+Find the cluster to which x belongs in this model.
+
+:param x: Either the point to determine the cluster for or an RDD 
of points to determine
+the clusters for.
+"""
+if isinstance(x, RDD):
+return x.map(self.predict(x))
+
+x = _convert_to_vector(x)
+return self.call("predict", x)
+
+def computeCost(self, point):
+"""
+Return the Bisecting K-means cost (sum of squared distances of 
points to
+their nearest center) for this model on the given data.
+
+:param point: the point to compute the cost to
+"""
+return self.call("computeCost", _convert_to_vector(point))
+
+
+class BisectingKMeans:
+"""
+A bisecting k-means algorithm based on the paper "A comparison of 
document clustering
+techniques" by Steinbach, Karypis, and Kumar, with modification to fit 
Spark.
+The algorithm starts from a single cluster that contains all points.
+Iteratively it finds divisible clusters on the bottom level and 
bisects each of them using
+k-means, until there are `k` leaf clusters in total or no leaf 
clusters are divisible.
+The bisecting steps of clusters on the same level are grouped together 
to increase parallelism.
+If bisecting all divisible clusters on the bottom level would result 
more than `k` leaf
+clusters, larger clusters get higher priority.
+
+Based on 
[[http://glaros.dtc.umn.edu/gkhome/fetch/papers/docclusterKDDTMW00.pdf
+Steinbach, Karypis, and Kumar, A comparison of document clustering 
techniques,
+KDD Workshop on Text Mining, 2000.]]
+"""
+def __init__(self):
+self.k = 4
+self.maxIterations = 20
+self.minDivisibleClusterSize = 1.0
+self.seed = -1888008604  # classOf[BisectingKMeans].getName.##
+
+def setK(self, k):
+"""
+Set the number of leaf clusters.
+
+:param k: the desired number of leaf clusters (default: 4). The 
actual number could be
+smaller if there are no divisible leaf clusters.
+"""
+self.k = k
+return self
+
+def getK(self):
+"""Return the desired number of leaf clusters."""
+return self.k
+
+def setMaxIterations(self, maxIterations):
+"""
+Set the maximum number of iterations.
+
+:param maxIterations: the max number of k-means iterations to 
split clusters (default: 20)
+"""
+self.maxIterations = maxIterations
+return self
+
+def getMaxIterations(self):
+"""Return the maximum number of iterations."""
+

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166033822
  
**[Test build #2237 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2237/consoleFull)**
 for PR 10395 at commit 
[`39b59f0`](https://github.com/apache/spark/commit/39b59f0c071fcb33dd941b8b50412b33d7ef7c39).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: Bump master version to 2.0.0-SNAPSHOT.

2015-12-19 Thread rxin

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10387#issuecomment-166033666
  
cc @JoshRosen I had to disable java style checker using a hotfix commit.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread tedyu

Github user tedyu commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166033780
  
Jenkins, test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-19 Thread gatorsmile

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/10396#issuecomment-166040219
  
retest it please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-2331] SparkContext.emptyRDD should retu...

2015-12-19 Thread rxin

GitHub user rxin opened a pull request:

https://github.com/apache/spark/pull/10394

[SPARK-2331] SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rxin/spark SPARK-2331

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10394.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10394


commit 86e64e1a021947f57049ddbf64aef29f73e8523e
Author: Reynold Xin 
Date:   2015-12-19T23:14:16Z

[SPARK-2331] SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166034111
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166034112
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48063/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166034107
  
**[Test build #48063 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48063/consoleFull)**
 for PR 10368 at commit 
[`facf246`](https://github.com/apache/spark/commit/facf2460e184900c66393edd3652607cb4e40b12).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166034110
  
**[Test build #48063 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48063/consoleFull)**
 for PR 10368 at commit 
[`facf246`](https://github.com/apache/spark/commit/facf2460e184900c66393edd3652607cb4e40b12).
 * This patch **fails executing the `dev/run-tests` script**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: SPARK-11295 Add packages to JUnit output for P...

2015-12-19 Thread gliptak

Github user gliptak commented on the pull request:

https://github.com/apache/spark/pull/9263#issuecomment-166042934
  
@zsxwing Are there some changes you would like to see to this pull request?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166050276
  
**[Test build #48066 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48066/consoleFull)**
 for PR 10395 at commit 
[`1b9486c`](https://github.com/apache/spark/commit/1b9486c859e7ca345a7fa4b37314509402655e91).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-19 Thread gatorsmile

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/10396#issuecomment-166055247
  
Jenkins, test this please



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166059382
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166059383
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48066/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166059917
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48067/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12287] [SQL] Support Unsafe Row in MapP...

2015-12-19 Thread gatorsmile

GitHub user gatorsmile opened a pull request:

https://github.com/apache/spark/pull/10398

[SPARK-12287] [SQL] Support Unsafe Row in MapPartitions/MapGroups/CoGroup

Support Unsafe Row in MapPartitions/MapGroups/CoGroup. 

Added a test case for MapPartitions. Since MapGroups and CoGroup are built 
on AppendColumns, all the related dataset test cases already can verify the 
correctness when MapGroups and CoGroup processing unsafe rows. 

@davies @cloud-fan Not sure if my understanding is right, please correct 
me. Thank you!

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gatorsmile/spark unsafeRowMapGroup

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/10398.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #10398


commit 4475706874082d19fcc7acb687689fa8d353a8aa
Author: gatorsmile 
Date:   2015-12-20T00:41:37Z

comment it out.

commit 969f6640f6c5636cd32600d1bfd290bf73a386ea
Author: gatorsmile 
Date:   2015-12-20T04:44:26Z

Support UnsafeRow in MapPartitions/MapGroups/CoGroup.

commit 6d0c8f50241fc2cdbd110e77c90733f4595320ef
Author: gatorsmile 
Date:   2015-12-20T04:47:40Z

Merge remote-tracking branch 'upstream/master' into unsafeRowMapGroup

commit a70e95c9109a753dd2528c5544ddd9ff86722397
Author: gatorsmile 
Date:   2015-12-20T04:49:55Z

converge.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread pwendell

Github user pwendell commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166069156
  
LGTM (I downloaded your PR and did some grepping to make sure there are no 
references). One other thing that occured to me is someone could easily create 
a package with this if they want to continue using in in Spark 2.0+, or just 
copy-paste the source code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166070595
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48070/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166033472
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166033473
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48061/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-11808] Remove Bagel.

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10395#issuecomment-166033471
  
**[Test build #48061 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48061/consoleFull)**
 for PR 10395 at commit 
[`39b59f0`](https://github.com/apache/spark/commit/39b59f0c071fcb33dd941b8b50412b33d7ef7c39).
 * This patch **fails Java style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-19 Thread gatorsmile

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/10396#issuecomment-166041205
  
@rxin Could you also comment it out the first line? Thanks!
```python
if not changed_files or any(f.endswith(".java") for f in changed_files):
# run_java_style_checks()
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX] [Build] [SPARK-12442] Build testing f...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10397#issuecomment-166046107
  
**[Test build #48065 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48065/consoleFull)**
 for PR 10397 at commit 
[`4475706`](https://github.com/apache/spark/commit/4475706874082d19fcc7acb687689fa8d353a8aa).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX] [Build] [SPARK-12442] Build testing f...

2015-12-19 Thread rxin

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/10397#issuecomment-166046053
  
I pushed directly to master and fixed it. Thanks.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: SPARK-11741 Process doctests using TextTestRun...

2015-12-19 Thread gliptak

Github user gliptak commented on the pull request:

https://github.com/apache/spark/pull/9710#issuecomment-166046164
  
@JoshRosen Please review.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX] [Build] [SPARK-12442] Build testing f...

2015-12-19 Thread gatorsmile

Github user gatorsmile closed the pull request at:

https://github.com/apache/spark/pull/10397


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX] [Build] [SPARK-12442] Build testing f...

2015-12-19 Thread gatorsmile

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/10397#issuecomment-166046802
  
Thanks! 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread tedyu

Github user tedyu commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166052576
  
Jenkins, test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-19 Thread gatorsmile

Github user gatorsmile commented on the pull request:

https://github.com/apache/spark/pull/10396#issuecomment-166052820
  
retest it please



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166055164
  
**[Test build #48067 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48067/consoleFull)**
 for PR 10368 at commit 
[`facf246`](https://github.com/apache/spark/commit/facf2460e184900c66393edd3652607cb4e40b12).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX] [Build] [SPARK-12442] Build testing f...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10397#issuecomment-166058780
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX] [Build] [SPARK-12442] Build testing f...

2015-12-19 Thread AmplabJenkins

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/10397#issuecomment-166058781
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48065/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [HOTFIX] [Build] [SPARK-12442] Build testing f...

2015-12-19 Thread SparkQA

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/10397#issuecomment-166058753
  
**[Test build #48065 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48065/consoleFull)**
 for PR 10397 at commit 
[`4475706`](https://github.com/apache/spark/commit/4475706874082d19fcc7acb687689fa8d353a8aa).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12270][SQL]remove empty space after get...

2015-12-19 Thread yhuai

Github user yhuai commented on the pull request:

https://github.com/apache/spark/pull/10262#issuecomment-166058803
  
I guess we can try to add a docker test. Can you try to add one in 
https://github.com/apache/spark/tree/master/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread tedyu

Github user tedyu commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166060830
  
Jenkins, test this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-19 Thread tedyu

Github user tedyu commented on the pull request:

https://github.com/apache/spark/pull/10368#issuecomment-166060825
  
HeartbeatReceiverSuite.'expire dead hosts' failed

Doesn't seem to be related to the PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

1 2 >

1 - 100 of 152 matches

Mail list logo