[GitHub] AmplabJenkins removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464304557 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464304555 Merged build finished. Test FAILed.

[GitHub] AmplabJenkins commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464304557 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464304555 Merged build finished. Test FAILed.

[GitHub] SparkQA removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
SparkQA removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464278279 **[Test build #102409 has

[GitHub] SparkQA commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
SparkQA commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464304377 **[Test build #102409 has

[GitHub] HyukjinKwon closed pull request #23800: [SPARK-26673][FollowUp][SQL] File source V2: remove duplicated broadcast object in FileWriterFactory

2019-02-15 Thread GitBox
HyukjinKwon closed pull request #23800: [SPARK-26673][FollowUp][SQL] File source V2: remove duplicated broadcast object in FileWriterFactory URL: https://github.com/apache/spark/pull/23800 This is an automated message from

[GitHub] HyukjinKwon commented on issue #23800: [SPARK-26673][FollowUp][SQL] File source V2: remove duplicated broadcast object in FileWriterFactory

2019-02-15 Thread GitBox
HyukjinKwon commented on issue #23800: [SPARK-26673][FollowUp][SQL] File source V2: remove duplicated broadcast object in FileWriterFactory URL: https://github.com/apache/spark/pull/23800#issuecomment-464301095 Merged to master.

[GitHub] HyukjinKwon commented on a change in pull request #23799: [SPARK-26892]Fix saveAsTextFile throws NullPointerException when null row present

2019-02-15 Thread GitBox
HyukjinKwon commented on a change in pull request #23799: [SPARK-26892]Fix saveAsTextFile throws NullPointerException when null row present URL: https://github.com/apache/spark/pull/23799#discussion_r257449106 ## File path: core/src/main/scala/org/apache/spark/rdd/RDD.scala

[GitHub] HyukjinKwon commented on a change in pull request #23797: [WIP][SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs

2019-02-15 Thread GitBox
HyukjinKwon commented on a change in pull request #23797: [WIP][SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs URL: https://github.com/apache/spark/pull/23797#discussion_r257448655 ## File path: docs/sql-data-sources-avro.md ## @@ -137,6 +137,37 @@

[GitHub] HyukjinKwon commented on a change in pull request #23797: [WIP][SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs

2019-02-15 Thread GitBox
HyukjinKwon commented on a change in pull request #23797: [WIP][SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs URL: https://github.com/apache/spark/pull/23797#discussion_r257448637 ## File path: docs/sql-data-sources-avro.md ## @@ -137,6 +137,37 @@

[GitHub] HyukjinKwon commented on a change in pull request #23797: [WIP][SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs

2019-02-15 Thread GitBox
HyukjinKwon commented on a change in pull request #23797: [WIP][SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs URL: https://github.com/apache/spark/pull/23797#discussion_r257448673 ## File path: python/pyspark/sql/functions.py ## @@ -2402,6 +2402,64

[GitHub] HeartSaVioR commented on issue #23706: [SPARK-26790][CORE] Change approach for retrieving executor logs and attributes: self-retrieve

2019-02-15 Thread GitBox
HeartSaVioR commented on issue #23706: [SPARK-26790][CORE] Change approach for retrieving executor logs and attributes: self-retrieve URL: https://github.com/apache/spark/pull/23706#issuecomment-464294044 Thanks all for reviewing and merging!

[GitHub] BryanCutler commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data.

2019-02-15 Thread GitBox
BryanCutler commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data. URL: https://github.com/apache/spark/pull/23795#discussion_r257447143 ## File path:

[GitHub] AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464291740 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464291739 Build finished. Test FAILed. This is an automated

[GitHub] SparkQA commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
SparkQA commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464291581 **[Test build #102408 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102408/testReport)** for PR

[GitHub] AmplabJenkins commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464291739 Build finished. Test FAILed. This is an automated message from

[GitHub] SparkQA removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
SparkQA removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464271730 **[Test build #102408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102408/testReport)**

[GitHub] AmplabJenkins commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464291740 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] BryanCutler commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data.

2019-02-15 Thread GitBox
BryanCutler commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data. URL: https://github.com/apache/spark/pull/23795#discussion_r257446732 ## File path:

[GitHub] yucai commented on a change in pull request #21149: [SPARK-24076][SQL] Use different seed in HashAggregate to avoid hash conflict

2019-02-15 Thread GitBox
yucai commented on a change in pull request #21149: [SPARK-24076][SQL] Use different seed in HashAggregate to avoid hash conflict URL: https://github.com/apache/spark/pull/21149#discussion_r257445460 ## File path:

[GitHub] SparkQA commented on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc.

2019-02-15 Thread GitBox
SparkQA commented on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc. URL: https://github.com/apache/spark/pull/23750#issuecomment-464281896 **[Test build #102410 has

[GitHub] AmplabJenkins removed a comment on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc.

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc. URL: https://github.com/apache/spark/pull/23750#issuecomment-464281636 Merged build finished. Test PASSed.

[GitHub] AmplabJenkins commented on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc.

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc. URL: https://github.com/apache/spark/pull/23750#issuecomment-464281636 Merged build finished. Test PASSed.

[GitHub] AmplabJenkins removed a comment on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc.

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc. URL: https://github.com/apache/spark/pull/23750#issuecomment-464281638 Test PASSed. Refer to this link for build results (access

[GitHub] AmplabJenkins commented on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc.

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc. URL: https://github.com/apache/spark/pull/23750#issuecomment-464281638 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] dilipbiswal commented on issue #23780: [SPARK-26864][SQL][BACKPORT-2.4] Query may return incorrect result when python udf is used as a join condition and the udf uses attributes from both leg

2019-02-15 Thread GitBox
dilipbiswal commented on issue #23780: [SPARK-26864][SQL][BACKPORT-2.4] Query may return incorrect result when python udf is used as a join condition and the udf uses attributes from both legs of left semi join URL: https://github.com/apache/spark/pull/23780#issuecomment-464281489

[GitHub] dilipbiswal commented on a change in pull request #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc.

2019-02-15 Thread GitBox
dilipbiswal commented on a change in pull request #23750: [SPARK-19712][SQL] Pushing Left Semi and Left Anti joins through Project, Aggregate, Window, Union etc. URL: https://github.com/apache/spark/pull/23750#discussion_r257444587 ## File path:

[GitHub] AmplabJenkins commented on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests URL: https://github.com/apache/spark/pull/23804#issuecomment-464279496 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins removed a comment on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests URL: https://github.com/apache/spark/pull/23804#issuecomment-464279496 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] HyukjinKwon commented on issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hadoop 3 profile

2019-02-15 Thread GitBox
HyukjinKwon commented on issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hadoop 3 profile URL: https://github.com/apache/spark/pull/21588#issuecomment-464279463 ping for what? Hive upgrade is in progress which blocks this PR https://github.com/apache/spark/pull/23788

[GitHub] AmplabJenkins commented on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests URL: https://github.com/apache/spark/pull/23804#issuecomment-464279495 Merged build finished. Test PASSed. This is an

[GitHub] AmplabJenkins removed a comment on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests URL: https://github.com/apache/spark/pull/23804#issuecomment-464279495 Merged build finished. Test PASSed. This

[GitHub] SparkQA removed a comment on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests

2019-02-15 Thread GitBox
SparkQA removed a comment on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests URL: https://github.com/apache/spark/pull/23804#issuecomment-464227566 **[Test build #102404 has

[GitHub] SparkQA commented on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests

2019-02-15 Thread GitBox
SparkQA commented on issue #23804: [WIP][SPARK-26896] JDK 11 module adjustments for running tests URL: https://github.com/apache/spark/pull/23804#issuecomment-464279361 **[Test build #102404 has

[GitHub] SparkQA commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
SparkQA commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464278279 **[Test build #102409 has

[GitHub] AmplabJenkins removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464278156 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464278155 Merged build finished. Test PASSed.

[GitHub] AmplabJenkins removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464278155 Merged build finished. Test PASSed.

[GitHub] AmplabJenkins commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#issuecomment-464278156 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] SongYadong commented on issue #23794: [SPARK-26884][CORE] Let task acquire memory accurately when using spilled memory

2019-02-15 Thread GitBox
SongYadong commented on issue #23794: [SPARK-26884][CORE] Let task acquire memory accurately when using spilled memory URL: https://github.com/apache/spark/pull/23794#issuecomment-464275258 Thanks for review. It's right the memory manager will try to give the right amount. But when going

[GitHub] liupc commented on a change in pull request #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame

2019-02-15 Thread GitBox
liupc commented on a change in pull request #23602: [SPARK-26674][CORE]Consolidate CompositeByteBuf when reading large frame URL: https://github.com/apache/spark/pull/23602#discussion_r257441590 ## File path:

[GitHub] asfgit closed pull request #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
asfgit closed pull request #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339 This is an automated message from the Apache Git Service. To respond to the

[GitHub] holdenk commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
holdenk commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464273651 Merged to master This is an automated message from the Apache

[GitHub] AmplabJenkins removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464273374 Merged build finished. Test PASSed. This is an

[GitHub] AmplabJenkins removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464273377 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464273377 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] SparkQA removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
SparkQA removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464270324 **[Test build #102407 has

[GitHub] AmplabJenkins commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464273374 Merged build finished. Test PASSed. This is an

[GitHub] SparkQA commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
SparkQA commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464273285 **[Test build #102407 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102407/testReport)**

[GitHub] holdenk commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
holdenk commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464273171 Looks like Jenkins listened, everything passed so will merge to master.

[GitHub] edwinalu commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics.

2019-02-15 Thread GitBox
edwinalu commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#discussion_r257440553 ## File path: core/src/main/scala/org/apache/spark/SparkContext.scala ## @@

[GitHub] edwinalu commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics.

2019-02-15 Thread GitBox
edwinalu commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#discussion_r257440322 ## File path: core/src/main/scala/org/apache/spark/executor/Executor.scala ##

[GitHub] edwinalu commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics.

2019-02-15 Thread GitBox
edwinalu commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#discussion_r257440251 ## File path: core/src/main/scala/org/apache/spark/SparkContext.scala ## @@

[GitHub] holdenk commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data.

2019-02-15 Thread GitBox
holdenk commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data. URL: https://github.com/apache/spark/pull/23795#discussion_r257439925 ## File path: python/pyspark/sql/types.py

[GitHub] AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464271644 Test PASSed. Refer to this link for build results (access rights to

[GitHub] holdenk commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data.

2019-02-15 Thread GitBox
holdenk commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data. URL: https://github.com/apache/spark/pull/23795#discussion_r257439790 ## File path: python/pyspark/sql/types.py

[GitHub] AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464271641 Merged build finished. Test PASSed.

[GitHub] SparkQA commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
SparkQA commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464271730 **[Test build #102408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102408/testReport)** for PR

[GitHub] AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464271641 Merged build finished. Test PASSed.

[GitHub] AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464271531 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464271529 Build finished. Test PASSed. This is an automated message from

[GitHub] AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464271644 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464271529 Build finished. Test PASSed. This is an automated

[GitHub] AmplabJenkins commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464271531 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] SparkQA removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
SparkQA removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464192922 **[Test build #102403 has

[GitHub] wypoon commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics.

2019-02-15 Thread GitBox
wypoon commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#discussion_r257439819 ## File path: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ## @@

[GitHub] SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464271447 **[Test build #102403 has

[GitHub] AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-453656628 Can one of the admins verify this patch? This is an

[GitHub] holdenk commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
holdenk commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464271175 jenkins ok to test This is an automated message from the Apache Git

[GitHub] holdenk commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2019-02-15 Thread GitBox
holdenk commented on issue #17968: [SPARK-9792] Make DenseMatrix equality semantical URL: https://github.com/apache/spark/pull/17968#issuecomment-464271163 Jenkins OK to test Are you still actively wortking on this and if so would you update it to master?

[GitHub] holdenk commented on issue #20028: [SPARK-19053][ML]Supporting multiple evaluation metrics in DataFrame-based API

2019-02-15 Thread GitBox
holdenk commented on issue #20028: [SPARK-19053][ML]Supporting multiple evaluation metrics in DataFrame-based API URL: https://github.com/apache/spark/pull/20028#issuecomment-464271070 Is this still being actively worked on?

[GitHub] AmplabJenkins removed a comment on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle URL: https://github.com/apache/spark/pull/23792#issuecomment-464270877 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] holdenk commented on issue #23793: [SPARK-24736][k8s] Let spark-submit handle dependency resolution.

2019-02-15 Thread GitBox
holdenk commented on issue #23793: [SPARK-24736][k8s] Let spark-submit handle dependency resolution. URL: https://github.com/apache/spark/pull/23793#issuecomment-464270862 So I'm a little confused here since if we look at the YARN cluster manager we also see similar logic around setting

[GitHub] AmplabJenkins removed a comment on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle URL: https://github.com/apache/spark/pull/23792#issuecomment-464270873 Merged build finished. Test PASSed. This

[GitHub] AmplabJenkins commented on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle URL: https://github.com/apache/spark/pull/23792#issuecomment-464270877 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins commented on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle URL: https://github.com/apache/spark/pull/23792#issuecomment-464270873 Merged build finished. Test PASSed. This is an

[GitHub] SparkQA removed a comment on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle

2019-02-15 Thread GitBox
SparkQA removed a comment on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle URL: https://github.com/apache/spark/pull/23792#issuecomment-464192851 **[Test build #102402 has

[GitHub] SparkQA commented on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle

2019-02-15 Thread GitBox
SparkQA commented on issue #23792: [SPARK-26882] Check the Kubernetes integration tests scalatyle URL: https://github.com/apache/spark/pull/23792#issuecomment-464270679 **[Test build #102402 has

[GitHub] AmplabJenkins removed a comment on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite URL: https://github.com/apache/spark/pull/23807#issuecomment-464270406 Test FAILed. Refer to this link for build results (access rights to CI server

[GitHub] SparkQA removed a comment on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite

2019-02-15 Thread GitBox
SparkQA removed a comment on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite URL: https://github.com/apache/spark/pull/23807#issuecomment-464256809 **[Test build #102406 has

[GitHub] AmplabJenkins commented on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite URL: https://github.com/apache/spark/pull/23807#issuecomment-464270406 Test FAILed. Refer to this link for build results (access rights to CI server

[GitHub] AmplabJenkins commented on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite URL: https://github.com/apache/spark/pull/23807#issuecomment-464270404 Merged build finished. Test FAILed.

[GitHub] AmplabJenkins removed a comment on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite URL: https://github.com/apache/spark/pull/23807#issuecomment-464270404 Merged build finished. Test FAILed.

[GitHub] SparkQA commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
SparkQA commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464270324 **[Test build #102407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102407/testReport)**

[GitHub] AmplabJenkins removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464270146 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] SparkQA commented on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite

2019-02-15 Thread GitBox
SparkQA commented on issue #23807: [SPARK-26897][SQL][TEST] Update Spark 2.3.x testing from HiveExternalCatalogVersionsSuite URL: https://github.com/apache/spark/pull/23807#issuecomment-464270362 **[Test build #102406 has

[GitHub] AmplabJenkins removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464270142 Merged build finished. Test PASSed. This is an

[GitHub] AmplabJenkins commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464270142 Merged build finished. Test PASSed. This is an

[GitHub] AmplabJenkins commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464270146 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464269503 Merged build finished. Test PASSed.

[GitHub] holdenk commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
holdenk commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464269764 Jenkins retest this please This is an automated message from

[GitHub] holdenk commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2019-02-15 Thread GitBox
holdenk commented on issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway URL: https://github.com/apache/spark/pull/18339#issuecomment-464269807 @parente if you could merge in master that would trigger a Jenkins run.

[GitHub] AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464269507 Test PASSed. Refer to this link for build results (access rights to

[GitHub] holdenk commented on a change in pull request #23741: [SPARK-22798][PYTHON][ML]Add multiple column support to PySpark StringIndexer

2019-02-15 Thread GitBox
holdenk commented on a change in pull request #23741: [SPARK-22798][PYTHON][ML]Add multiple column support to PySpark StringIndexer URL: https://github.com/apache/spark/pull/23741#discussion_r257438402 ## File path: python/pyspark/ml/wrapper.py ## @@ -87,9 +87,19 @@ def

[GitHub] AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464269503 Merged build finished. Test PASSed.

[GitHub] holdenk commented on a change in pull request #23741: [SPARK-22798][PYTHON][ML]Add multiple column support to PySpark StringIndexer

2019-02-15 Thread GitBox
holdenk commented on a change in pull request #23741: [SPARK-22798][PYTHON][ML]Add multiple column support to PySpark StringIndexer URL: https://github.com/apache/spark/pull/23741#discussion_r257438503 ## File path: python/pyspark/ml/wrapper.py ## @@ -87,9 +87,19 @@ def

[GitHub] AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464269507 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] SparkQA removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-02-15 Thread GitBox
SparkQA removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-464189231 **[Test build #102401 has

  1   2   3   4   5   6   >