[GitHub] [spark] BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template

2019-08-06 Thread GitBox
BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template URL: https://github.com/apache/spark/pull/25310#discussion_r311171967 ## File path: .github/PULL_REQUEST_TEMPLATE ## @@ -1,10 +1,40 @@ -## What changes were

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232 @dongjoon-hyun By default the user will not be able to pass any spark.driver.extraJavaOptions settings since his properties are put in a

[GitHub] [spark] SparkQA commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks

2019-08-06 Thread GitBox
SparkQA commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks URL: https://github.com/apache/spark/pull/25369#issuecomment-518761636 **[Test build #108727 has

[GitHub] [spark] skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518762919 a This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] skonto removed a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto removed a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518762919 a This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone

2019-08-06 Thread GitBox
SparkQA removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone URL: https://github.com/apache/spark/pull/25047#issuecomment-518720708 **[Test build #108725 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs URL: https://github.com/apache/spark/pull/25368#issuecomment-518775442 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs URL: https://github.com/apache/spark/pull/25368#issuecomment-518775437 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone

2019-08-06 Thread GitBox
SparkQA commented on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone URL: https://github.com/apache/spark/pull/25047#issuecomment-518775434 **[Test build #108725 has

[GitHub] [spark] AmplabJenkins commented on issue #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs URL: https://github.com/apache/spark/pull/25368#issuecomment-518775437 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs URL: https://github.com/apache/spark/pull/25368#issuecomment-518775442 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2 URL: https://github.com/apache/spark/pull/25115#issuecomment-518779618 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2 URL: https://github.com/apache/spark/pull/25115#issuecomment-518779603 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-518728006 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #25180: [SPARK-28423][SQL] Merge Scan and Batch/Stream

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25180: [SPARK-28423][SQL] Merge Scan and Batch/Stream URL: https://github.com/apache/spark/pull/25180#issuecomment-518727515 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

2019-08-06 Thread GitBox
SparkQA removed a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-518717058 **[Test build #108724 has

[GitHub] [spark] SparkQA removed a comment on issue #25180: [SPARK-28423][SQL] Merge Scan and Batch/Stream

2019-08-06 Thread GitBox
SparkQA removed a comment on issue #25180: [SPARK-28423][SQL] Merge Scan and Batch/Stream URL: https://github.com/apache/spark/pull/25180#issuecomment-518639294 **[Test build #108717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108717/testReport)** for

[GitHub] [spark] AmplabJenkins removed a comment on issue #25180: [SPARK-28423][SQL] Merge Scan and Batch/Stream

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25180: [SPARK-28423][SQL] Merge Scan and Batch/Stream URL: https://github.com/apache/spark/pull/25180#issuecomment-518727506 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] srowen commented on a change in pull request #25324: [SPARK-21481][ML][FOLLOWUP] HashingTF Cleanup and Tiny Optimizations

2019-08-06 Thread GitBox
srowen commented on a change in pull request #25324: [SPARK-21481][ML][FOLLOWUP] HashingTF Cleanup and Tiny Optimizations URL: https://github.com/apache/spark/pull/25324#discussion_r311138530 ## File path: mllib/src/main/scala/org/apache/spark/ml/feature/HashingTF.scala ##

[GitHub] [spark] AmplabJenkins commented on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-518728017 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

2019-08-06 Thread GitBox
SparkQA commented on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-518727822 **[Test build #108724 has

[GitHub] [spark] rdblue commented on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2

2019-08-06 Thread GitBox
rdblue commented on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2 URL: https://github.com/apache/spark/pull/25115#issuecomment-518736975 @cloud-fan, @xianyinxin, I don't think this should use a `Filter` node as a child just because some rules run on `Filter` and not

[GitHub] [spark] AmplabJenkins removed a comment on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-518736362 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3

2019-08-06 Thread GitBox
SparkQA commented on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-518737175 **[Test build #108726 has

[GitHub] [spark] squito commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-08-06 Thread GitBox
squito commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#discussion_r311155440 ## File path:

[GitHub] [spark] BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template

2019-08-06 Thread GitBox
BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template URL: https://github.com/apache/spark/pull/25310#discussion_r311173234 ## File path: .github/PULL_REQUEST_TEMPLATE ## @@ -1,10 +1,40 @@ -## What changes were

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232 @dongjoon-hyun By default the user will not be able to pass any spark.driver.extraJavaOptions settings since his properties are put in a

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here pls: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here pls: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be

[GitHub] [spark] AmplabJenkins removed a comment on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks URL: https://github.com/apache/spark/pull/25369#issuecomment-518763728 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] ueshin commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall'

2019-08-06 Thread GitBox
ueshin commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall' URL: https://github.com/apache/spark/pull/24761#issuecomment-518766658 I'm sorry for the delay. Let me retrigger the tests since it's been a while since the last build.

[GitHub] [spark] ueshin commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall'

2019-08-06 Thread GitBox
ueshin commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall' URL: https://github.com/apache/spark/pull/24761#issuecomment-518766740 Jenkins, retest this please. This is an automated message from

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here pls: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be

[GitHub] [spark] AmplabJenkins removed a comment on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall'

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall' URL: https://github.com/apache/spark/pull/24761#issuecomment-518769448 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins removed a comment on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall'

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall' URL: https://github.com/apache/spark/pull/24761#issuecomment-518769457 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2

2019-08-06 Thread GitBox
SparkQA commented on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2 URL: https://github.com/apache/spark/pull/25115#issuecomment-518778869 **[Test build #108723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108723/testReport)** for PR

[GitHub] [spark] SparkQA removed a comment on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2

2019-08-06 Thread GitBox
SparkQA removed a comment on issue #25115: [SPARK-28351][SQL] Support DELETE in DataSource V2 URL: https://github.com/apache/spark/pull/25115#issuecomment-518695464 **[Test build #108723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108723/testReport)**

[GitHub] [spark] squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.

2019-08-06 Thread GitBox
squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#discussion_r311133908 ## File path:

[GitHub] [spark] squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.

2019-08-06 Thread GitBox
squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#discussion_r311131698 ## File path: core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java

[GitHub] [spark] squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.

2019-08-06 Thread GitBox
squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#discussion_r311128847 ## File path:

[GitHub] [spark] squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.

2019-08-06 Thread GitBox
squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#discussion_r311136689 ## File path:

[GitHub] [spark] squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.

2019-08-06 Thread GitBox
squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#discussion_r311137103 ## File path:

[GitHub] [spark] squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.

2019-08-06 Thread GitBox
squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#discussion_r311136545 ## File path:

[GitHub] [spark] squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.

2019-08-06 Thread GitBox
squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#discussion_r311141394 ## File path: core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java

[GitHub] [spark] squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API.

2019-08-06 Thread GitBox
squito commented on a change in pull request #25304: [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API. URL: https://github.com/apache/spark/pull/25304#discussion_r311131107 ## File path: core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java

[GitHub] [spark] AmplabJenkins removed a comment on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-518652164 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] rdblue commented on a change in pull request #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs

2019-08-06 Thread GitBox
rdblue commented on a change in pull request #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs URL: https://github.com/apache/spark/pull/25368#discussion_r311150819 ## File path:

[GitHub] [spark] rdblue commented on a change in pull request #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs

2019-08-06 Thread GitBox
rdblue commented on a change in pull request #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs URL: https://github.com/apache/spark/pull/25368#discussion_r311151581 ## File path:

[GitHub] [spark] wangyum commented on a change in pull request #25306: [SPARK-28573][SQL] Convert InsertIntoTable(HiveTableRelation) to DataSource inserting for partitioned table

2019-08-06 Thread GitBox
wangyum commented on a change in pull request #25306: [SPARK-28573][SQL] Convert InsertIntoTable(HiveTableRelation) to DataSource inserting for partitioned table URL: https://github.com/apache/spark/pull/25306#discussion_r311167395 ## File path:

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518748224 @dongjoon-hyun one thing I noticed is that if I set: ``` case "$1" in driver) shift 1 VERBOSE_FLAG=$(get_verbose_flag)

[GitHub] [spark] BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template

2019-08-06 Thread GitBox
BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template URL: https://github.com/apache/spark/pull/25310#discussion_r311169750 ## File path: .github/PULL_REQUEST_TEMPLATE ## @@ -1,10 +1,40 @@ -## What changes were

[GitHub] [spark] BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template

2019-08-06 Thread GitBox
BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template URL: https://github.com/apache/spark/pull/25310#discussion_r311173044 ## File path: .github/PULL_REQUEST_TEMPLATE ## @@ -1,10 +1,40 @@ -## What changes were

[GitHub] [spark] gengliangwang opened a new pull request #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks

2019-08-06 Thread GitBox
gengliangwang opened a new pull request #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks URL: https://github.com/apache/spark/pull/25369 ## What changes were proposed in this pull request? Currently, on requesting summary metrics, cached data

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here pls: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be

[GitHub] [spark] skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be appended

[GitHub] [spark] AmplabJenkins commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks URL: https://github.com/apache/spark/pull/25369#issuecomment-518763728 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks URL: https://github.com/apache/spark/pull/25369#issuecomment-518763716 Merged build finished. Test PASSed.

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here pls: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here pls: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be

[GitHub] [spark] SparkQA commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall'

2019-08-06 Thread GitBox
SparkQA commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall' URL: https://github.com/apache/spark/pull/24761#issuecomment-518767430 **[Test build #108728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108728/testReport)** for

[GitHub] [spark] AmplabJenkins commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall'

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall' URL: https://github.com/apache/spark/pull/24761#issuecomment-518769457 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall'

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #24761: [SPARK-27905] [SQL] Add higher order function 'forall' URL: https://github.com/apache/spark/pull/24761#issuecomment-518769448 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone URL: https://github.com/apache/spark/pull/25047#issuecomment-518776172 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone URL: https://github.com/apache/spark/pull/25047#issuecomment-518776172 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone URL: https://github.com/apache/spark/pull/25047#issuecomment-518776182 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone URL: https://github.com/apache/spark/pull/25047#issuecomment-518776182 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone URL: https://github.com/apache/spark/pull/25047#issuecomment-518723253 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone URL: https://github.com/apache/spark/pull/25047#issuecomment-518723267 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #25180: [SPARK-28423][SQL] Merge Scan and Batch/Stream

2019-08-06 Thread GitBox
SparkQA commented on issue #25180: [SPARK-28423][SQL] Merge Scan and Batch/Stream URL: https://github.com/apache/spark/pull/25180#issuecomment-518726781 **[Test build #108717 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108717/testReport)** for PR

[GitHub] [spark] AmplabJenkins removed a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-518728006 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-518728017 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] rdblue commented on a change in pull request #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs

2019-08-06 Thread GitBox
rdblue commented on a change in pull request #25368: [SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs URL: https://github.com/apache/spark/pull/25368#discussion_r311152228 ## File path:

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518748224 @dongjoon-hyun one thing I noticed is that if I set: ``` case "$1" in driver) shift 1 VERBOSE_FLAG=$(get_verbose_flag)

[GitHub] [spark] BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template

2019-08-06 Thread GitBox
BryanCutler commented on a change in pull request #25310: [SPARK-28578][INFRA] Improve Github pull request template URL: https://github.com/apache/spark/pull/25310#discussion_r311171074 ## File path: .github/PULL_REQUEST_TEMPLATE ## @@ -1,10 +1,40 @@ -## What changes were

[GitHub] [spark] skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto commented on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232 @dongjoon-hyun By default the user will not be able to pass any spark.driver.extraJavaOptions settings are his properties are put in a

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232 @dongjoon-hyun By default the user will not be able to pass any spark.driver.extraJavaOptions settings since his properties are put in a

[GitHub] [spark] skonto removed a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto removed a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232 @dongjoon-hyun By default the user will not be able to pass any spark.driver.extraJavaOptions settings since his properties are put in a

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518759232 @dongjoon-hyun By default the user will not be able to pass any spark.driver.extraJavaOptions settings since his properties are put in a

[GitHub] [spark] gengliangwang commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks

2019-08-06 Thread GitBox
gengliangwang commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks URL: https://github.com/apache/spark/pull/25369#issuecomment-518761523 cc @vanzin This is an

[GitHub] [spark] AmplabJenkins removed a comment on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks URL: https://github.com/apache/spark/pull/25369#issuecomment-518763716 Merged build finished. Test PASSed.

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here pls: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be

[GitHub] [spark] skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag

2019-08-06 Thread GitBox
skonto edited a comment on issue #25229: [SPARK-27900][K8s] Add jvm oom flag URL: https://github.com/apache/spark/pull/25229#issuecomment-518763517 @dongjoon-hyun a clarification here pls: > By default, DEFAULT_DRIVER_JVM_OPTIONS=-XX:OnOutOfMemoryError="kill -9 %p" and it will be

[GitHub] [spark] yeshengm commented on a change in pull request #24983: [SPARK-27714][SQL][CBO] Support Genetic Algorithm based join reorder

2019-08-06 Thread GitBox
yeshengm commented on a change in pull request #24983: [SPARK-27714][SQL][CBO] Support Genetic Algorithm based join reorder URL: https://github.com/apache/spark/pull/24983#discussion_r311206035 ## File path:

[GitHub] [spark] rdblue commented on a change in pull request #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.

2019-08-06 Thread GitBox
rdblue commented on a change in pull request #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#discussion_r311221404 ## File path:

[GitHub] [spark] sarutak opened a new pull request #25370: [SPARK-28639][CORE] Configuration doc for Barrier Mode Execution

2019-08-06 Thread GitBox
sarutak opened a new pull request #25370: [SPARK-28639][CORE] Configuration doc for Barrier Mode Execution URL: https://github.com/apache/spark/pull/25370 ## What changes were proposed in this pull request? SPARK-24817 and SPARK-24819 introduce new 3 non-internal properties for

[GitHub] [spark] AmplabJenkins commented on issue #25370: [SPARK-28639][CORE] Configuration doc for Barrier Execution Mode

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25370: [SPARK-28639][CORE] Configuration doc for Barrier Execution Mode URL: https://github.com/apache/spark/pull/25370#issuecomment-518804344 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25370: [SPARK-28639][CORE] Configuration doc for Barrier Execution Mode

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25370: [SPARK-28639][CORE] Configuration doc for Barrier Execution Mode URL: https://github.com/apache/spark/pull/25370#issuecomment-518804340 Merged build finished. Test PASSed. This is an

[GitHub] [spark] srowen commented on a change in pull request #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect

2019-08-06 Thread GitBox
srowen commented on a change in pull request #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect URL: https://github.com/apache/spark/pull/25344#discussion_r311252167 ## File path:

[GitHub] [spark] srowen commented on a change in pull request #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect

2019-08-06 Thread GitBox
srowen commented on a change in pull request #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect URL: https://github.com/apache/spark/pull/25344#discussion_r311252276 ## File path:

[GitHub] [spark] yeshengm commented on a change in pull request #24983: [SPARK-27714][SQL][CBO] Support Genetic Algorithm based join reorder

2019-08-06 Thread GitBox
yeshengm commented on a change in pull request #24983: [SPARK-27714][SQL][CBO] Support Genetic Algorithm based join reorder URL: https://github.com/apache/spark/pull/24983#discussion_r311201620 ## File path:

[GitHub] [spark] yeshengm commented on a change in pull request #24983: [SPARK-27714][SQL][CBO] Support Genetic Algorithm based join reorder

2019-08-06 Thread GitBox
yeshengm commented on a change in pull request #24983: [SPARK-27714][SQL][CBO] Support Genetic Algorithm based join reorder URL: https://github.com/apache/spark/pull/24983#discussion_r311201620 ## File path:

[GitHub] [spark] yeshengm commented on a change in pull request #24983: [SPARK-27714][SQL][CBO] Support Genetic Algorithm based join reorder

2019-08-06 Thread GitBox
yeshengm commented on a change in pull request #24983: [SPARK-27714][SQL][CBO] Support Genetic Algorithm based join reorder URL: https://github.com/apache/spark/pull/24983#discussion_r311205884 ## File path:

[GitHub] [spark] AmplabJenkins removed a comment on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-518784211 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-518784196 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-518784196 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3

2019-08-06 Thread GitBox
AmplabJenkins commented on issue #25289: [WIP][SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-518784211 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks

2019-08-06 Thread GitBox
SparkQA commented on issue #25369: [SPARK-28638][WebUI] Task summary metrics are wrong when there are running tasks URL: https://github.com/apache/spark/pull/25369#issuecomment-518792720 **[Test build #108727 has

[GitHub] [spark] squito commented on a change in pull request #25236: [SPARK-28487][k8s] More responsive dynamic allocation with K8S.

2019-08-06 Thread GitBox
squito commented on a change in pull request #25236: [SPARK-28487][k8s] More responsive dynamic allocation with K8S. URL: https://github.com/apache/spark/pull/25236#discussion_r311217419 ## File path:

[GitHub] [spark] sarutak commented on a change in pull request #25370: [SPARK-28639][CORE] Configuration doc for Barrier Mode Execution

2019-08-06 Thread GitBox
sarutak commented on a change in pull request #25370: [SPARK-28639][CORE] Configuration doc for Barrier Mode Execution URL: https://github.com/apache/spark/pull/25370#discussion_r311228569 ## File path: docs/configuration.md ## @@ -2018,6 +2018,50 @@ Apart from these, the

[GitHub] [spark] AmplabJenkins removed a comment on issue #25370: [SPARK-28639][CORE] Configuration doc for Barrier Execution Mode

2019-08-06 Thread GitBox
AmplabJenkins removed a comment on issue #25370: [SPARK-28639][CORE] Configuration doc for Barrier Execution Mode URL: https://github.com/apache/spark/pull/25370#issuecomment-518804344 Test PASSed. Refer to this link for build results (access rights to CI server needed):

<    1   2   3   4   5   6   7   8   9   >