[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-25 Thread mgyucht
Github user mgyucht commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r190842619 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/not-in-unit-tests-single-column.sql --- @@ -0,0 +1,123 @@ +-- Unit tests

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #91152 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91152/testReport)** for PR 19691 at commit

[GitHub] spark issue #21429: [minor] Add port SSL config in toString and scaladoc

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21429 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21331 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21331 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3584/

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-25 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21317 @liyinan926 done. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN subquer...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21425 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21378: [SPARK-24326][Mesos] add support for local:// scheme for...

2018-05-25 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21378 @felixcheung could you review pls? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN subquer...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21425 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91148/ Test PASSed. ---

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91147/ Test PASSed. ---

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21416 **[Test build #91147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91147/testReport)** for PR 21416 at commit

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21410 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21331 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21410 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91151/ Test PASSed. ---

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21331 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91153/ Test PASSed. ---

[GitHub] spark issue #21429: [minor] Add port SSL config in toString and scaladoc

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21429 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91149/ Test PASSed. ---

[GitHub] spark issue #21429: [minor] Add port SSL config in toString and scaladoc

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21429 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21422: [Spark-24376][doc]Summary:compiling spark with scala-2.1...

2018-05-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21422 Seems this paragraph is already removed in master branch (https://issues.apache.org/jira/browse/SPARK-19810). So this is a branch 2.2- issue only. ---

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-05-25 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20640 sure @squito . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91154/ Test PASSed. ---

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3460/ ---

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 **[Test build #91154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91154/testReport)** for PR 21317 at commit

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #91155 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91155/testReport)** for PR 20894 at commit

[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-25 Thread mgyucht
Github user mgyucht commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r190828822 --- Diff: sql/core/src/test/resources/sql-tests/results/typeCoercion/native/concat.sql.out --- @@ -1,5 +1,5 @@ -- Automatically generated by

[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-25 Thread mgyucht
Github user mgyucht commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r190829076 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/not-in-unit-tests-single-column.sql --- @@ -0,0 +1,123 @@ +-- Unit tests

[GitHub] spark pull request #21317: [SPARK-24232][k8s] Add support for secret env var...

2018-05-25 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/21317#discussion_r190867546 --- Diff: docs/running-on-kubernetes.md --- @@ -602,4 +608,20 @@ specific to Spark on Kubernetes.

[GitHub] spark issue #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN subquer...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21425 **[Test build #91148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91148/testReport)** for PR 21425 at commit

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 **[Test build #91154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91154/testReport)** for PR 21317 at commit

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3460/ ---

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21331 **[Test build #91153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91153/testReport)** for PR 21331 at commit

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3585/

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #91152 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91152/testReport)** for PR 19691 at commit

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91152/ Test PASSed. ---

[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-05-25 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r190829341 --- Diff: core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala --- @@ -414,7 +415,106 @@ class ExternalAppendOnlyMapSuite

[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r190841604 --- Diff: sql/core/src/test/resources/sql-tests/results/typeCoercion/native/concat.sql.out --- @@ -1,5 +1,5 @@ -- Automatically generated by

[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r190841865 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/not-in-unit-tests-single-column.sql --- @@ -0,0 +1,123 @@ +-- Unit

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21410 **[Test build #91151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91151/testReport)** for PR 21410 at commit

[GitHub] spark issue #21331: [SPARK-24276][SQL] Order of literals in IN should not af...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21331 **[Test build #91153 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91153/testReport)** for PR 21331 at commit

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21426 **[Test build #91150 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91150/testReport)** for PR 21426 at commit

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91150/ Test PASSed. ---

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21429: [minor] Add port SSL config in toString and scaladoc

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21429 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3582/

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3583/

[GitHub] spark issue #21426: [SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correc...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21426 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21429: [minor] Add port SSL config in toString and scaladoc

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21429 **[Test build #91149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91149/testReport)** for PR 21429 at commit

[GitHub] spark issue #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL write fai...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21430 **[Test build #91156 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91156/testReport)** for PR 21430 at commit

[GitHub] spark issue #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompress...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21431 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3586/

[GitHub] spark issue #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompress...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21431 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompress...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21431 **[Test build #91157 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91157/testReport)** for PR 21431 at commit

[GitHub] spark issue #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompress...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21431 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91157/ Test PASSed. ---

[GitHub] spark issue #21427: [SPARK-24324][PYTHON] Pandas Grouped Map UDF should assi...

2018-05-25 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/21427 @rxin @gatorsmile thanks for joining the discussion! On the configuration side, we have already some mechanism to do so for the "timezone" config:

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21432 **[Test build #91158 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91158/testReport)** for PR 21432 at commit

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21432 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3587/

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-05-25 Thread susanxhuynh
Github user susanxhuynh commented on the issue: https://github.com/apache/spark/pull/20640 I'm in favor of merging this. The hardcoded limit is pretty bad - particularly for streaming jobs; it would be preferable to remove it ASAP even though it may not be a complete solution. ---

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-05-25 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21366 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL write fai...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21430 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91156/ Test PASSed. ---

[GitHub] spark issue #21346: [SPARK-6237][NETWORK] Network-layer changes to allow str...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21346 **[Test build #4189 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4189/testReport)** for PR 21346 at commit

[GitHub] spark pull request #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortC...

2018-05-25 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21431 [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompressionCodecNames to configuration. ## What changes were proposed in this pull request? Spark provides three codecs: `lz4`, `lzf`,

[GitHub] spark issue #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL write fai...

2018-05-25 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/21430 cc @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3464/ ---

[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-25 Thread mgyucht
Github user mgyucht commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r190934130 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/not-in-unit-tests-multi-column-literal.sql.out --- @@ -0,0 +1,97 @@ +--

[GitHub] spark pull request #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN ...

2018-05-25 Thread mgyucht
Github user mgyucht commented on a diff in the pull request: https://github.com/apache/spark/pull/21425#discussion_r190934095 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/not-in-unit-tests-multi-column-literal.sql.out --- @@ -0,0 +1,97 @@ +--

[GitHub] spark issue #21433: [SPARK-23820][CORE] Enable use of long form of callsite ...

2018-05-25 Thread michaelmior
Github user michaelmior commented on the issue: https://github.com/apache/spark/pull/21433 Done! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21045: [SPARK-23931][SQL] Adds zip function to sparksql

2018-05-25 Thread DylanGuedes
Github user DylanGuedes commented on the issue: https://github.com/apache/spark/pull/21045 During the last days I looked for other ways to use `getValue` properly without such repetition, but didn't get success at all :( suggestions are (very) welcome ---

[GitHub] spark issue #21321: [SPARK-24268][SQL] Use datatype.simpleString in error me...

2018-05-25 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21321 kindly ping @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3463/ ---

[GitHub] spark pull request #21433: [SPARK-23820][CORE] Enable use of long form of ca...

2018-05-25 Thread michaelmior
GitHub user michaelmior opened a pull request: https://github.com/apache/spark/pull/21433 [SPARK-23820][CORE] Enable use of long form of callsite in logs You can merge this pull request into a Git repository by running: $ git pull https://github.com/michaelmior/spark

[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #91155 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91155/testReport)** for PR 20894 at commit

[GitHub] spark pull request #21432: [SPARK-24373][SQL] Add AnalysisBarrier to Relatio...

2018-05-25 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/21432 [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGroupedDataset's child ## What changes were proposed in this pull request? When we create a `RelationalGroupedDataset` we set its child

[GitHub] spark issue #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL write fai...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21430 **[Test build #91156 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91156/testReport)** for PR 21430 at commit

[GitHub] spark issue #21433: [SPARK-23820][CORE] Enable use of long form of callsite ...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21433 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL write fai...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21430 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompress...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21431 **[Test build #91157 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91157/testReport)** for PR 21431 at commit

[GitHub] spark issue #21432: [SPARK-24373][SQL] Add AnalysisBarrier to RelationalGrou...

2018-05-25 Thread TomaszGaweda
Github user TomaszGaweda commented on the issue: https://github.com/apache/spark/pull/21432 Probably similar change should be also in KeyValueGroupedDataset. It also uses logicalPlan without AnalysisBarrie --- - To

[GitHub] spark issue #21427: [SPARK-24324][PYTHON] Pandas Grouped Map UDF should assi...

2018-05-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21427 btw, we should really mark this as experimental and allow a bit of behaviour changes really. I guess that's what we meat by Experimental:

[GitHub] spark issue #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL write fai...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21430 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91155/ Test PASSed. ---

[GitHub] spark issue #21405: [SPARK-24361][SQL] Polish code block manipulation API

2018-05-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21405 @cloud-fan I found that the way to collect method parameters in the use case can be simplified. Updated in the PR description. Please take a look when you have time. Thanks. ---

[GitHub] spark pull request #21407: [SPARK-24313][SQL][BACKPORT-2.3] Fix collection o...

2018-05-25 Thread mgaido91
Github user mgaido91 closed the pull request at: https://github.com/apache/spark/pull/21407 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 **[Test build #91159 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91159/testReport)** for PR 21366 at commit

[GitHub] spark issue #21431: [SPARK-19112][CORE][FOLLOW-UP] Add missing shortCompress...

2018-05-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21431 > How was this patch tested? BTW, I also assume you manually tested `zstd` works as documented? --- - To

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3463/ ---

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3588/

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-05-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-05-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21061 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL wr...

2018-05-25 Thread gaborgsomogyi
GitHub user gaborgsomogyi opened a pull request: https://github.com/apache/spark/pull/21430 [SPARK-23991][DSTREAMS] Fix data loss when WAL write fails in allocateBlocksToBatch ## What changes were proposed in this pull request? When blocks tried to get allocated to a batch

[GitHub] spark issue #21425: [SPARK-24381][TESTING] Add unit tests for NOT IN subquer...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21425 **[Test build #91160 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91160/testReport)** for PR 21425 at commit

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-05-25 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r190990593 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -93,6 +96,10 @@ private[spark] class EventLoggingListener(

[GitHub] spark issue #21427: [SPARK-24324][PYTHON] Pandas Grouped Map UDF should assi...

2018-05-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21427 **[Test build #91166 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91166/testReport)** for PR 21427 at commit

[GitHub] spark pull request #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL wr...

2018-05-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21430#discussion_r190993327 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockTrackerSuite.scala --- @@ -308,12 +354,16 @@ class ReceivedBlockTrackerSuite

[GitHub] spark pull request #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL wr...

2018-05-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21430#discussion_r190992559 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceivedBlockTracker.scala --- @@ -112,10 +112,13 @@ private[streaming] class

[GitHub] spark pull request #21430: [SPARK-23991][DSTREAMS] Fix data loss when WAL wr...

2018-05-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21430#discussion_r190992915 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockTrackerSuite.scala --- @@ -115,6 +117,50 @@ class ReceivedBlockTrackerSuite

[GitHub] spark issue #21429: [minor] Add port SSL config in toString and scaladoc

2018-05-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21429 Merged to 2.2 also. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-05-25 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r190995781 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -800,26 +812,50 @@ private[spark] class Executor( } }

  1   2   3   4   5   >