[GitHub] spark issue #19919: [SPARK-22727] spark.executor.instances's default value s...

2017-12-07 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/spark/pull/19919 @srowen Where does the default value "2" of spark.executor.instances used? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19927: [WIP] OVR transform optimization

2017-12-07 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/19927 [WIP] OVR transform optimization ## What changes were proposed in this pull request? optimize OVR transform ## How was this patch tested? existing tests You can merge this

[GitHub] spark issue #19926: [SPARK-22733] Split StreamExecution into MicroBatchExecu...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19926 **[Test build #84635 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84635/testReport)** for PR 19926 at commit

[GitHub] spark pull request #19783: [SPARK-21322][SQL] support histogram in filter ca...

2017-12-07 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19783#discussion_r155691788 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -529,6 +570,56 @@ case class

[GitHub] spark issue #19769: [SPARK-12297][SQL] Adjust timezone for int96 data from i...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19769 **[Test build #84634 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84634/testReport)** for PR 19769 at commit

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19925 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19925 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84624/ Test PASSed. ---

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19925 **[Test build #84624 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84624/testReport)** for PR 19925 at commit

[GitHub] spark pull request #19769: [SPARK-12297][SQL] Adjust timezone for int96 data...

2017-12-07 Thread henryr
Github user henryr commented on a diff in the pull request: https://github.com/apache/spark/pull/19769#discussion_r155690933 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedColumnReader.java --- @@ -294,11 +304,22 @@ private void

[GitHub] spark pull request #19783: [SPARK-21322][SQL] support histogram in filter ca...

2017-12-07 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19783#discussion_r155690722 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -332,8 +332,41 @@ case class

[GitHub] spark issue #19921: [SPARK-22452][SQL] Add getDouble to DataSourceV2Options

2017-12-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19921 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18995: [SPARK-21787][SQL] Support for pushing down filte...

2017-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18995 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19926: [SPARK-22733] Split StreamExecution into MicroBatchExecu...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19926 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19926: [SPARK-22733] Split StreamExecution into MicroBatchExecu...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19926 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84625/ Test FAILed. ---

[GitHub] spark issue #19926: [SPARK-22733] Split StreamExecution into MicroBatchExecu...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19926 **[Test build #84625 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84625/testReport)** for PR 19926 at commit

[GitHub] spark issue #18995: [SPARK-21787][SQL] Support for pushing down filters for ...

2017-12-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18995 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18113: [SPARK-20890][SQL] Added min and max typed aggreg...

2017-12-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18113#discussion_r155688313 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala --- @@ -263,6 +262,25 @@ class DatasetAggregatorSuite extends

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-12-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r155688125 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/ExpressionCodegen.scala --- @@ -0,0 +1,237 @@ +/* + *

[GitHub] spark issue #19852: [SPARK-22655][PySpark] Throw exception rather than exit ...

2017-12-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19852 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19920 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84620/ Test PASSed. ---

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19920 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19920 **[Test build #84620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84620/testReport)** for PR 19920 at commit

[GitHub] spark issue #19852: [SPARK-22655][PySpark] Throw exception rather than exit ...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19852 **[Test build #84633 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84633/testReport)** for PR 19852 at commit

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/19717 I understand the concern here. I think we can guard against accidental use in client mode to post a nicer error message. @liyinan926, how about testing if `kubernetes.default.svc` is accessible?

[GitHub] spark pull request #19852: [SPARK-22655][PySpark] Throw exception rather tha...

2017-12-07 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/19852#discussion_r155686222 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -319,7 +319,7 @@ private[spark] abstract class BasePythonRunner[IN,

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19717 All existing clusters work with client mode, so you're on your own here... maybe setting an environment variable when starting the driver and checking that it's set in your code. Or something. ---

[GitHub] spark issue #19924: [SPARK-22187][SS][REVERT] Revert change in state row for...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19924 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84621/ Test PASSed. ---

[GitHub] spark issue #19924: [SPARK-22187][SS][REVERT] Revert change in state row for...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19924 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19924: [SPARK-22187][SS][REVERT] Revert change in state row for...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19924 **[Test build #84621 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84621/testReport)** for PR 19924 at commit

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #84632 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84632/testReport)** for PR 19872 at commit

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84632/ Test FAILed. ---

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-12-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r155684442 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/ExpressionCodegen.scala --- @@ -0,0 +1,237 @@ +/* + *

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #84632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84632/testReport)** for PR 19872 at commit

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19783: [SPARK-21322][SQL] support histogram in filter ca...

2017-12-07 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/19783#discussion_r155683828 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/EstimationUtils.scala --- @@ -114,4 +114,144 @@ object

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84631/ Test FAILed. ---

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #84631 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84631/testReport)** for PR 19872 at commit

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #84631 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84631/testReport)** for PR 19872 at commit

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/19717 Actually given that in our implementations the executors talk to the driver through a Kubernetes headless service that is only created by the submission client. I don't think any of the two

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #84630 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84630/testReport)** for PR 19872 at commit

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84630/ Test FAILed. ---

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #84630 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84630/testReport)** for PR 19872 at commit

[GitHub] spark issue #19591: [SPARK-11035][core] Add in-process Spark app launcher.

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19591 **[Test build #84629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84629/testReport)** for PR 19591 at commit

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/19872 And to @holdenk 's question. Pandas group_agg udf fundamentally uses different physical plan than the existing java/scala udf and therefore it's hard to combine them together. I don't know a

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/19717 There are two cases to consider here. 1) If the client that creates the `SparkContext` is running in a Kubernetes pod, it should work the same way as the driver pod of an application submitted

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/19872 I end up removing `UDFColumn` class and using the existing `Aggregate` logical plan for pandas group_agg UDF. This reuses a lot of code being to existing `Aggregate` and minimize the code

[GitHub] spark pull request #19922: [SPARK-22688][SQL][HOTFIX] Upgrade Janino version...

2017-12-07 Thread srowen
Github user srowen closed the pull request at: https://github.com/apache/spark/pull/19922 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84628/ Test FAILed. ---

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #84628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84628/testReport)** for PR 19872 at commit

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19920 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84618/ Test PASSed. ---

[GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #84628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84628/testReport)** for PR 19872 at commit

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19920 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-12-07 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19527 > For example with 5 categories, we don't know [0.0, 0.0, 0.0, 0.0, 0.0] means last category or invalid value. For the semantics I described ("OPTION 1"), I think it is clear: * Last

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19920 **[Test build #84618 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84618/testReport)** for PR 19920 at commit

[GitHub] spark issue #19721: [SPARK-22496][SQL]thrift server adds operation logs

2017-12-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19721 cc @liufengdb --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/19717 We do enforce that by: ``` case (KUBERNETES, CLIENT) => printErrorAndExit("Client mode is currently not supported for Kubernetes.") ``` ---

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19717 It's more than a docs issue though, it should be enforced in the code, and as far as I've seen the code path I mentioned does not generate an error (or at least not an obvious one). ---

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19893 Yeah, I don't think this can be added before figuring out SQL tests. ``` $ grep 'POSSIBLE THREAD LEAK' unit-tests.log | wc -l 158 ``` ---

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19893 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/19717 @vanzin Sorry I didn't see your question. We don't yet support client mode and no one has ever tried a true client mode, i.e., the driver runs on the client machine outside a Kubernetes cluster.

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19893 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84617/ Test PASSed. ---

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19893 **[Test build #84617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84617/testReport)** for PR 19893 at commit

[GitHub] spark issue #19923: [SPARK-22721] BytesToBytesMap peak memory not updated.

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19923 **[Test build #84627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84627/testReport)** for PR 19923 at commit

[GitHub] spark issue #19923: [SPARK-22721] BytesToBytesMap peak memory not updated.

2017-12-07 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/19923 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19921: [SPARK-22452][SQL] Add getDouble to DataSourceV2Options

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19921 **[Test build #84626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84626/testReport)** for PR 19921 at commit

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19717 I don't think anyone answered my previous question: > One question though: what happens if someone does new SparkContext(myConf) in their code with a > k8s URL? That's a valid way of

[GitHub] spark issue #19923: [SPARK-22721] BytesToBytesMap peak memory not updated.

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19923 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19923: [SPARK-22721] BytesToBytesMap peak memory not updated.

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84619/ Test FAILed. ---

[GitHub] spark issue #19923: [SPARK-22721] BytesToBytesMap peak memory not updated.

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19923 **[Test build #84619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84619/testReport)** for PR 19923 at commit

[GitHub] spark issue #19921: [SPARK-22452][SQL] Add getDouble to DataSourceV2Options

2017-12-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19921 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19926: [SPARK-22733] Split StreamExecution into MicroBatchExecu...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19926 **[Test build #84625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84625/testReport)** for PR 19926 at commit

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-12-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r155673528 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/ExpressionCodegen.scala --- @@ -0,0 +1,237 @@ +/* + *

[GitHub] spark pull request #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nest...

2017-12-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19813#discussion_r155673475 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/ExpressionCodegen.scala --- @@ -0,0 +1,237 @@ +/* + *

[GitHub] spark pull request #19926: [SPARK-22733] Split StreamExecution into MicroBat...

2017-12-07 Thread joseph-torres
GitHub user joseph-torres opened a pull request: https://github.com/apache/spark/pull/19926 [SPARK-22733] Split StreamExecution into MicroBatchExecution and StreamExecution. ## What changes were proposed in this pull request? StreamExecution is now an abstract base class,

[GitHub] spark pull request #19499: [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2017-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19499 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19499: [SPARK-22279][SQL] Turn on spark.sql.hive.convertMetasto...

2017-12-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19499 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19925 **[Test build #84624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84624/testReport)** for PR 19925 at commit

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155668850 --- Diff: python/pyspark/sql/types.py --- @@ -1658,13 +1657,13 @@ def from_arrow_type(at): spark_type = FloatType() elif at ==

[GitHub] spark pull request #18113: [SPARK-20890][SQL] Added min and max typed aggreg...

2017-12-07 Thread setjet
Github user setjet commented on a diff in the pull request: https://github.com/apache/spark/pull/18113#discussion_r155667834 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala --- @@ -263,6 +262,25 @@ class DatasetAggregatorSuite extends QueryTest

[GitHub] spark issue #19843: [SPARK-22644][ML][TEST][WIP] Make ML testsuite support S...

2017-12-07 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19843 I'll make a call: Given that the SQL tests do not use clearActive, let's not bother with it. If we see flakiness, then we can try adding clearActive as a fix. ---

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19925 **[Test build #84623 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84623/testReport)** for PR 19925 at commit

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19925 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84623/ Test FAILed. ---

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19925 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19717 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84613/ Test PASSed. ---

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19717 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19717 **[Test build #84613 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84613/testReport)** for PR 19717 at commit

[GitHub] spark issue #19922: [SPARK-22688][SQL][HOTFIX] Upgrade Janino version to 3.0...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19922 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19922: [SPARK-22688][SQL][HOTFIX] Upgrade Janino version to 3.0...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19922 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84615/ Test PASSed. ---

[GitHub] spark issue #19922: [SPARK-22688][SQL][HOTFIX] Upgrade Janino version to 3.0...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19922 **[Test build #84615 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84615/testReport)** for PR 19922 at commit

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19925 **[Test build #84623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84623/testReport)** for PR 19925 at commit

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19925 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19925 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84622/ Test FAILed. ---

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19925 **[Test build #84622 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84622/testReport)** for PR 19925 at commit

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19925 **[Test build #84622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84622/testReport)** for PR 19925 at commit

[GitHub] spark pull request #19925: [SPARK-22732] Add Structured Streaming APIs to Da...

2017-12-07 Thread joseph-torres
GitHub user joseph-torres opened a pull request: https://github.com/apache/spark/pull/19925 [SPARK-22732] Add Structured Streaming APIs to DataSourceV2 ## What changes were proposed in this pull request? This PR provides DataSourceV2 API support for structured streaming,

[GitHub] spark issue #19924: [SPARK-22187][SS][REVERT] Revert change in state row for...

2017-12-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19924 LGTM pending tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19924: [SPARK-22187][SS][REVERT] Revert change in state row for...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19924 **[Test build #84621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84621/testReport)** for PR 19924 at commit

<    1   2   3   4   5   >