[GitHub] [spark] viirya commented on a change in pull request #24675: [SPARK-27803][SQL][PYTHON] Fix column pruning for Python UDF

2019-05-22 Thread GitBox
viirya commented on a change in pull request #24675: [SPARK-27803][SQL][PYTHON] Fix column pruning for Python UDF URL: https://github.com/apache/spark/pull/24675#discussion_r286786053 ## File path:

[GitHub] [spark] viirya commented on a change in pull request #24675: [SPARK-27803][SQL][PYTHON] Fix column pruning for Python UDF

2019-05-22 Thread GitBox
viirya commented on a change in pull request #24675: [SPARK-27803][SQL][PYTHON] Fix column pruning for Python UDF URL: https://github.com/apache/spark/pull/24675#discussion_r286785463 ## File path:

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24680: [SPARK-26045][BUILD] Leave avro, avro-ipc dependendencies as compile scope even for hadoop-provided usages

2019-05-22 Thread GitBox
dongjoon-hyun edited a comment on issue #24680: [SPARK-26045][BUILD] Leave avro, avro-ipc dependendencies as compile scope even for hadoop-provided usages URL: https://github.com/apache/spark/pull/24680#issuecomment-495068392 I'll leave this PR here since @vanzin 's review is requested. We

[GitHub] [spark] dongjoon-hyun commented on issue #24640: [SPARK-27770] [SQL] [TEST] Port AGGREGATES.sql [Part 1]

2019-05-22 Thread GitBox
dongjoon-hyun commented on issue #24640: [SPARK-27770] [SQL] [TEST] Port AGGREGATES.sql [Part 1] URL: https://github.com/apache/spark/pull/24640#issuecomment-495074398 Could you fix the UT failure? ``` [info] - aggregates_part1.sql *** FAILED *** (3 seconds, 720 milliseconds) ```

[GitHub] [spark] pengbo removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-05-22 Thread GitBox
pengbo removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495049729 retest this please

[GitHub] [spark] pengbo commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-05-22 Thread GitBox
pengbo commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495073870 retest this please

[GitHub] [spark] HyukjinKwon commented on issue #24675: [SPARK-27803][SQL][PYTHON] Fix column pruning for Python UDF

2019-05-22 Thread GitBox
HyukjinKwon commented on issue #24675: [SPARK-27803][SQL][PYTHON] Fix column pruning for Python UDF URL: https://github.com/apache/spark/pull/24675#issuecomment-495073865 makes sense to me. This is an automated message from

[GitHub] [spark] cloud-fan commented on issue #24344: [SPARK-27440][SQL] Optimize uncorrelated predicate subquery

2019-05-22 Thread GitBox
cloud-fan commented on issue #24344: [SPARK-27440][SQL] Optimize uncorrelated predicate subquery URL: https://github.com/apache/spark/pull/24344#issuecomment-495070324 I think @dilipbiswal has a good point here. For non-correlated EXISTS/IN, it's a bad idea to collect all the data of a

[GitHub] [spark] AmplabJenkins removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495068453 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495068451 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] dongjoon-hyun commented on issue #24680: [SPARK-26045][BUILD] Leave avro, avro-ipc dependendencies as compile scope even for hadoop-provided usages

2019-05-22 Thread GitBox
dongjoon-hyun commented on issue #24680: [SPARK-26045][BUILD] Leave avro, avro-ipc dependendencies as compile scope even for hadoop-provided usages URL: https://github.com/apache/spark/pull/24680#issuecomment-495068392 I'll leave this PR here since @vanzin 's review is requested. We need

[GitHub] [spark] AmplabJenkins commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495068451 Merged build finished. Test PASSed. This is an automated message from

[GitHub] [spark] AmplabJenkins commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495068453 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
SparkQA removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495039716 **[Test build #105709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105709/testReport)** for PR

[GitHub] [spark] SparkQA commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
SparkQA commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495068179 **[Test build #105709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105709/testReport)** for PR 24559 at

[GitHub] [spark] AmplabJenkins commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495067333 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495067331 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495067331 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495067333 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
SparkQA commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495067035 **[Test build #105708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105708/testReport)** for

[GitHub] [spark] SparkQA removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
SparkQA removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495038346 **[Test build #105708 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495066701 Test PASSed. Refer to this link for build results (access

[GitHub] [spark] AmplabJenkins removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495066698 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495066698 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495066701 Test PASSed. Refer to this link for build results (access rights to

[GitHub] [spark] SparkQA removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
SparkQA removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495045176 **[Test build #105710 has

[GitHub] [spark] SparkQA commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
SparkQA commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495066402 **[Test build #105710 has

[GitHub] [spark] viirya commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
viirya commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286774379 ## File path:

[GitHub] [spark] viirya commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
viirya commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286774235 ## File path:

[GitHub] [spark] zhengruifeng edited a comment on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve

2019-05-22 Thread GitBox
zhengruifeng edited a comment on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495060029 @srowen Oh, not a pass. My expression was not correct. Sliding need a separate job to collect head

[GitHub] [spark] zhengruifeng edited a comment on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve

2019-05-22 Thread GitBox
zhengruifeng edited a comment on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495060029 @srowen Oh, not a pass. My expression was not correct. Sliding need a separate job to collect head

[GitHub] [spark] gengliangwang edited a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
gengliangwang edited a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#issuecomment-495059043 This should not be a big concern. The file writing job is almost transactional since

[GitHub] [spark] zhengruifeng commented on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve

2019-05-22 Thread GitBox
zhengruifeng commented on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495060029 @srowen Oh, not a pass. My expression was not correct. Sliding need a separate job to collect head rows on

[GitHub] [spark] gengliangwang commented on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
gengliangwang commented on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#issuecomment-495059043 This should not be a big concern. The file writing job is almost transactional since Spark

[GitHub] [spark] gengliangwang commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
gengliangwang commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286765012 ## File path:

[GitHub] [spark] gengliangwang commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
gengliangwang commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286765012 ## File path:

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
dongjoon-hyun edited a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#issuecomment-495054805 I understand the concern about the difference from our default `.schema` option. I

[GitHub] [spark] JkSelf commented on issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during broadcast join

2019-05-22 Thread GitBox
JkSelf commented on issue #21899: [SPARK-24912][SQL] Don't obscure source of OOM during broadcast join URL: https://github.com/apache/spark/pull/21899#issuecomment-495055518 @beliefer Thanks for your working. Here before we new the newPage in `val newPage = new

[GitHub] [spark] francis0407 commented on a change in pull request #24344: [SPARK-27440][SQL] Optimize uncorrelated predicate subquery

2019-05-22 Thread GitBox
francis0407 commented on a change in pull request #24344: [SPARK-27440][SQL] Optimize uncorrelated predicate subquery URL: https://github.com/apache/spark/pull/24344#discussion_r286764851 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/subquery.scala

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
dongjoon-hyun edited a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#issuecomment-495054805 I understand the concern about the difference from our default `.schema` option. I

[GitHub] [spark] dongjoon-hyun commented on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
dongjoon-hyun commented on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#issuecomment-495054805 I understand the concern about the difference from our default `.schema` option. I believe

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
dongjoon-hyun commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286763767 ## File path:

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
dongjoon-hyun commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286763619 ## File path:

[GitHub] [spark] AmplabJenkins commented on issue #23791: [SPARK-20597][SQL][SS][WIP] KafkaSourceProvider falls back on path as synonym for topic

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #23791: [SPARK-20597][SQL][SS][WIP] KafkaSourceProvider falls back on path as synonym for topic URL: https://github.com/apache/spark/pull/23791#issuecomment-495053513 Can one of the admins verify this patch?

[GitHub] [spark] HyukjinKwon commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
HyukjinKwon commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286763265 ## File path:

[GitHub] [spark] AmplabJenkins removed a comment on issue #23791: [SPARK-20597][SQL][SS][WIP] KafkaSourceProvider falls back on path as synonym for topic

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #23791: [SPARK-20597][SQL][SS][WIP] KafkaSourceProvider falls back on path as synonym for topic URL: https://github.com/apache/spark/pull/23791#issuecomment-463663060 Can one of the admins verify this patch?

[GitHub] [spark] HyukjinKwon commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
HyukjinKwon commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286762917 ## File path:

[GitHub] [spark] wangyum commented on issue #24672: [SPARK-27801] Improve performance of InMemoryFileIndex.listLeafFiles for HDFS directories with many files

2019-05-22 Thread GitBox
wangyum commented on issue #24672: [SPARK-27801] Improve performance of InMemoryFileIndex.listLeafFiles for HDFS directories with many files URL: https://github.com/apache/spark/pull/24672#issuecomment-495052170 Thank you @rrusso2007 @JoshRosen I did simple benchmark in our production

[GitHub] [spark] wangyum closed pull request #24679: [SPARK-27807][SQL] Parallel resolve leaf statuses InMemoryFileIndex

2019-05-22 Thread GitBox
wangyum closed pull request #24679: [SPARK-27807][SQL] Parallel resolve leaf statuses InMemoryFileIndex URL: https://github.com/apache/spark/pull/24679 This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] pengbo removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-05-22 Thread GitBox
pengbo removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495044115 Retest this please

[GitHub] [spark] pengbo commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-05-22 Thread GitBox
pengbo commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495049729 retest this please

[GitHub] [spark] pengbo removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-05-22 Thread GitBox
pengbo removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495042381 retest this please

[GitHub] [spark] AmplabJenkins removed a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495046736 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495046742 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495046736 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495046742 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver

2019-05-22 Thread GitBox
SparkQA removed a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495024200 **[Test build #105707 has

[GitHub] [spark] SparkQA commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver

2019-05-22 Thread GitBox
SparkQA commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495046418 **[Test build #105707 has

[GitHub] [spark] SparkQA commented on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
SparkQA commented on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495045176 **[Test build #105710 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495044831 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495044835 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] [spark] AmplabJenkins commented on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495044835 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495044831 Merged build finished. Test PASSed.

[GitHub] [spark] beliefer commented on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-22 Thread GitBox
beliefer commented on issue #24671: [MINOR][DOCS]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-495044178 Retest this please.

[GitHub] [spark] pengbo commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-05-22 Thread GitBox
pengbo commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495044115 Retest this please

[GitHub] [spark] beliefer commented on a change in pull request #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.

2019-05-22 Thread GitBox
beliefer commented on a change in pull request #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#discussion_r286753702 ## File path:

[GitHub] [spark] habren commented on issue #24663: [SPARK-27792][SQL] SkewJoin--handle only skewed keys with broadcastjoin

2019-05-22 Thread GitBox
habren commented on issue #24663: [SPARK-27792][SQL] SkewJoin--handle only skewed keys with broadcastjoin URL: https://github.com/apache/spark/pull/24663#issuecomment-495043133 @viirya Could you please review this pull request ?

[GitHub] [spark] pengbo removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-05-22 Thread GitBox
pengbo removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-494819839 retest this please

[GitHub] [spark] pengbo commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-05-22 Thread GitBox
pengbo commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495042381 retest this please

[GitHub] [spark] beliefer commented on a change in pull request #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.

2019-05-22 Thread GitBox
beliefer commented on a change in pull request #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#discussion_r286753702 ## File path:

[GitHub] [spark] sjrand edited a comment on issue #24645: [SPARK-27773][Shuffle] add metrics for number of exceptions caught in ExternalShuffleBlockHandler

2019-05-22 Thread GitBox
sjrand edited a comment on issue #24645: [SPARK-27773][Shuffle] add metrics for number of exceptions caught in ExternalShuffleBlockHandler URL: https://github.com/apache/spark/pull/24645#issuecomment-495040379 On the client (executor) side we were seeing lots of timeouts, e.g.: ```

[GitHub] [spark] sjrand edited a comment on issue #24645: [SPARK-27773][Shuffle] add metrics for number of exceptions caught in ExternalShuffleBlockHandler

2019-05-22 Thread GitBox
sjrand edited a comment on issue #24645: [SPARK-27773][Shuffle] add metrics for number of exceptions caught in ExternalShuffleBlockHandler URL: https://github.com/apache/spark/pull/24645#issuecomment-495040379 On the client (executor) side we were seeing lots of timeouts, e.g.: ```

[GitHub] [spark] sjrand edited a comment on issue #24645: [SPARK-27773][Shuffle] add metrics for number of exceptions caught in ExternalShuffleBlockHandler

2019-05-22 Thread GitBox
sjrand edited a comment on issue #24645: [SPARK-27773][Shuffle] add metrics for number of exceptions caught in ExternalShuffleBlockHandler URL: https://github.com/apache/spark/pull/24645#issuecomment-495040379 On the client (executor) side we were seeing lots of timeouts, e.g.: ```

[GitHub] [spark] sjrand commented on issue #24645: [SPARK-27773][Shuffle] add metrics for number of exceptions caught in ExternalShuffleBlockHandler

2019-05-22 Thread GitBox
sjrand commented on issue #24645: [SPARK-27773][Shuffle] add metrics for number of exceptions caught in ExternalShuffleBlockHandler URL: https://github.com/apache/spark/pull/24645#issuecomment-495040379 On the client (executor) side we were seeing lots of timeouts, e.g.: ```

[GitHub] [spark] SparkQA commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
SparkQA commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495039716 **[Test build #105709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105709/testReport)** for PR 24559 at

[GitHub] [spark] AmplabJenkins removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495039354 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495039363 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495039363 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24559: [SPARK-27658][SQL] Add FunctionCatalog API URL: https://github.com/apache/spark/pull/24559#issuecomment-495039354 Merged build finished. Test PASSed. This is an automated message from

[GitHub] [spark] SparkQA commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
SparkQA commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495038346 **[Test build #105708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105708/testReport)** for

[GitHub] [spark] AmplabJenkins removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495038020 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495038025 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495038020 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
AmplabJenkins commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495038025 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] rdblue commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2

2019-05-22 Thread GitBox
rdblue commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2 URL: https://github.com/apache/spark/pull/24233#issuecomment-495037807 @cloud-fan, I don't recall that conclusion from a sync. Can you quote from the notes that you're talking about? I'm fine

[GitHub] [spark] rdblue commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
rdblue commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495037408 @mccheah, I made the changes you requested. Should be good to go when tests pass.

[GitHub] [spark] jiangxb1987 commented on issue #24605: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks

2019-05-22 Thread GitBox
jiangxb1987 commented on issue #24605: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks URL: https://github.com/apache/spark/pull/24605#issuecomment-495034840 Thanks! Merged to master, please manually backport to 2.4!

[GitHub] [spark] jiangxb1987 closed pull request #24605: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks

2019-05-22 Thread GitBox
jiangxb1987 closed pull request #24605: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks URL: https://github.com/apache/spark/pull/24605 This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] jiangxb1987 commented on a change in pull request #24615: [SPARK-27488][CORE] Driver interface to support GPU resources

2019-05-22 Thread GitBox
jiangxb1987 commented on a change in pull request #24615: [SPARK-27488][CORE] Driver interface to support GPU resources URL: https://github.com/apache/spark/pull/24615#discussion_r286741051 ## File path: docs/configuration.md ## @@ -187,6 +187,25 @@ of the most common

[GitHub] [spark] mccheah edited a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
mccheah edited a comment on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495027809 Looks good, about what I would expect apart from some small changes.

[GitHub] [spark] mccheah commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
mccheah commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495027809 Looks good, about what we would expect apart from some small changes.

[GitHub] [spark] mccheah commented on a change in pull request #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
mccheah commented on a change in pull request #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#discussion_r286740813 ## File path:

[GitHub] [spark] mccheah commented on a change in pull request #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.

2019-05-22 Thread GitBox
mccheah commented on a change in pull request #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#discussion_r286739814 ## File path:

[GitHub] [spark] cloud-fan commented on a change in pull request #24675: [SPARK-27803][SQL] fix column pruning for python UDF

2019-05-22 Thread GitBox
cloud-fan commented on a change in pull request #24675: [SPARK-27803][SQL] fix column pruning for python UDF URL: https://github.com/apache/spark/pull/24675#discussion_r286738983 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala

[GitHub] [spark] cloud-fan commented on a change in pull request #24675: [SPARK-27803][SQL] fix column pruning for python UDF

2019-05-22 Thread GitBox
cloud-fan commented on a change in pull request #24675: [SPARK-27803][SQL] fix column pruning for python UDF URL: https://github.com/apache/spark/pull/24675#discussion_r286738848 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala

[GitHub] [spark] cloud-fan commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
cloud-fan commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286738681 ## File path:

[GitHub] [spark] cloud-fan commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
cloud-fan commented on a change in pull request #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#discussion_r286738727 ## File path:

[GitHub] [spark] SparkQA commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver

2019-05-22 Thread GitBox
SparkQA commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495024200 **[Test build #105707 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#issuecomment-495023973 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide

2019-05-22 Thread GitBox
AmplabJenkins removed a comment on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#issuecomment-495023966 Merged build finished. Test PASSed.

  1   2   3   4   5   6   7   8   >