[GitHub] [spark] attilapiros commented on a change in pull request #24072: [SPARK-27112] : Spark Scheduler encounters two independent Deadlocks …

2019-03-13 Thread GitBox
attilapiros commented on a change in pull request #24072: [SPARK-27112] : Spark Scheduler encounters two independent Deadlocks … URL: https://github.com/apache/spark/pull/24072#discussion_r265067154 ## File path:

[GitHub] [spark] SparkQA commented on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2

2019-03-13 Thread GitBox
SparkQA commented on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2 URL: https://github.com/apache/spark/pull/24078#issuecomment-472375990 **[Test build #103417 has

[GitHub] [spark] AmplabJenkins commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472378641 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA removed a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
SparkQA removed a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472305457 **[Test build #103418 has

[GitHub] [spark] SparkQA commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
SparkQA commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472378017 **[Test build #103418 has

[GitHub] [spark] AmplabJenkins commented on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion URL: https://github.com/apache/spark/pull/24068#issuecomment-472384021 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion URL: https://github.com/apache/spark/pull/24068#issuecomment-472384014 Merged build finished. Test PASSed.

[GitHub] [spark] ajithme commented on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown

2019-03-13 Thread GitBox
ajithme commented on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown URL: https://github.com/apache/spark/pull/24056#issuecomment-472389093 @srowen I think changing pool may be difficult as we may have to return a fake Future. Instead i have updated to handle

[GitHub] [spark] gengliangwang commented on a change in pull request #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions

2019-03-13 Thread GitBox
gengliangwang commented on a change in pull request #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions URL: https://github.com/apache/spark/pull/24025#discussion_r265088245 ## File path:

[GitHub] [spark] gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information

2019-03-13 Thread GitBox
gengliangwang commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#discussion_r265019884 ## File path: sql/core/src/main/scala/org/apache/spark/status/api/v1/ApiSqlRootResource.scala

[GitHub] [spark] SparkQA commented on issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it - adds inuse tracking.

2019-03-13 Thread GitBox
SparkQA commented on issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it - adds inuse tracking. URL: https://github.com/apache/spark/pull/19096#issuecomment-472329665 **[Test build #103428 has

[GitHub] [spark] viirya commented on issue #24053: [SPARK-27126][SQL] Consolidate Scala and Java type deserializerFor

2019-03-13 Thread GitBox
viirya commented on issue #24053: [SPARK-27126][SQL] Consolidate Scala and Java type deserializerFor URL: https://github.com/apache/spark/pull/24053#issuecomment-472329269 @cloud-fan I think the way to create serde expressions should be the same no matter Scala or Java. The real

[GitHub] [spark] AmplabJenkins removed a comment on issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it - adds inuse tracking.

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it - adds inuse tracking. URL: https://github.com/apache/spark/pull/19096#issuecomment-472329036 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it - adds inuse tracking.

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it - adds inuse tracking. URL: https://github.com/apache/spark/pull/19096#issuecomment-472329041 Test PASSed. Refer to this link for build results

[GitHub] [spark] SparkQA commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
SparkQA commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472335527 **[Test build #103429 has

[GitHub] [spark] AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472335572 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472335580 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it - adds inuse tracking.

2019-03-13 Thread GitBox
SparkQA commented on issue #19096: [SPARK-21869][SS] A cached Kafka producer should not be closed if any task is using it - adds inuse tracking. URL: https://github.com/apache/spark/pull/19096#issuecomment-472335687 **[Test build #103427 has

[GitHub] [spark] gengliangwang commented on issue #24076: [SPARK-27142] Provide REST API for SQL level information

2019-03-13 Thread GitBox
gengliangwang commented on issue #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#issuecomment-472340810 @ajithme Thanks for the work. To make it consistent with other API, I think we need to have at least two APIs ```

[GitHub] [spark] AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472340815 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472340806 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions

2019-03-13 Thread GitBox
SparkQA commented on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions URL: https://github.com/apache/spark/pull/24025#issuecomment-472365728 **[Test build #103423 has

[GitHub] [spark] SparkQA commented on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor

2019-03-13 Thread GitBox
SparkQA commented on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-472375848 **[Test build #103420 has

[GitHub] [spark] AmplabJenkins commented on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-472376159 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins commented on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-472376163 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2

2019-03-13 Thread GitBox
SparkQA removed a comment on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2 URL: https://github.com/apache/spark/pull/24078#issuecomment-472305454 **[Test build #103417 has

[GitHub] [spark] SparkQA removed a comment on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor

2019-03-13 Thread GitBox
SparkQA removed a comment on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-472305481 **[Test build #103420 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-472376159 Merged build finished. Test FAILed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472378641 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472378647 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution URL: https://github.com/apache/spark/pull/23981#issuecomment-472382675 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution URL: https://github.com/apache/spark/pull/23981#issuecomment-472382675 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution URL: https://github.com/apache/spark/pull/23981#issuecomment-472382677 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution URL: https://github.com/apache/spark/pull/23981#issuecomment-472382677 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24024: [MINOR][CORE]spark.diskStore.subDirectories <= 0 should throw Exception

2019-03-13 Thread GitBox
SparkQA commented on issue #24024: [MINOR][CORE]spark.diskStore.subDirectories <= 0 should throw Exception URL: https://github.com/apache/spark/pull/24024#issuecomment-472396662 **[Test build #4618 has

[GitHub] [spark] srowen commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

2019-03-13 Thread GitBox
srowen commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265095730 ## File path:

[GitHub] [spark] SparkQA commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
SparkQA commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472331904 **[Test build #103429 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472331350 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472331337 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472335580 Test FAILed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] SparkQA commented on issue #24051: [SPARK-26879][SQL] Zero based column indexing for struct function

2019-03-13 Thread GitBox
SparkQA commented on issue #24051: [SPARK-26879][SQL] Zero based column indexing for struct function URL: https://github.com/apache/spark/pull/24051#issuecomment-472354857 **[Test build #103432 has

[GitHub] [spark] chakravarthiT commented on issue #24051: [SPARK-26879][SQL] Zero based column indexing for struct function

2019-03-13 Thread GitBox
chakravarthiT commented on issue #24051: [SPARK-26879][SQL] Zero based column indexing for struct function URL: https://github.com/apache/spark/pull/24051#issuecomment-472361545 @maropu @viirya Thanks,that is good idea. Only stack and json_tuple uses zero based column indexing.So,I have

[GitHub] [spark] AmplabJenkins commented on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2 URL: https://github.com/apache/spark/pull/24078#issuecomment-472376636 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2 URL: https://github.com/apache/spark/pull/24078#issuecomment-472376638 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

2019-03-13 Thread GitBox
SparkQA commented on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#issuecomment-472381261 **[Test build #103416 has

[GitHub] [spark] AmplabJenkins commented on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array URL: https://github.com/apache/spark/pull/24073#issuecomment-472381303 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array URL: https://github.com/apache/spark/pull/24073#issuecomment-472381310 Test PASSed. Refer to this link for build results (access rights to

[GitHub] [spark] AmplabJenkins commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472380751 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA removed a comment on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array

2019-03-13 Thread GitBox
SparkQA removed a comment on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array URL: https://github.com/apache/spark/pull/24073#issuecomment-472305479 **[Test build #103419 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472380762 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] HyukjinKwon commented on issue #23952: [SPARK-26929][SQL]fix table owner use user instead of principal when create table through spark-sql or beeline

2019-03-13 Thread GitBox
HyukjinKwon commented on issue #23952: [SPARK-26929][SQL]fix table owner use user instead of principal when create table through spark-sql or beeline URL: https://github.com/apache/spark/pull/23952#issuecomment-472387067 cc @dongjoon-hyun FYI.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown URL: https://github.com/apache/spark/pull/24056#issuecomment-471539599 Can one of the admins verify this patch?

[GitHub] [spark] SparkQA commented on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown

2019-03-13 Thread GitBox
SparkQA commented on issue #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown URL: https://github.com/apache/spark/pull/24056#issuecomment-472391094 **[Test build #4617 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4617/testReport)**

[GitHub] [spark] srowen commented on a change in pull request #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown

2019-03-13 Thread GitBox
srowen commented on a change in pull request #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown URL: https://github.com/apache/spark/pull/24056#discussion_r265089571 ## File path: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala ## @@

[GitHub] [spark] AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472331337 Merged build finished. Test PASSed.

[GitHub] [spark] HyukjinKwon commented on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2

2019-03-13 Thread GitBox
HyukjinKwon commented on issue #24078: [SPARK-27034][SQL][FOLLOWUP] Update Nested Schema Pruning benchmark result with EC2 URL: https://github.com/apache/spark/pull/24078#issuecomment-472350471 Sorry if I missed something, @dongjoon-hyun. Does this PR simply updates the results against

[GitHub] [spark] AmplabJenkins removed a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472380751 Merged build finished. Test PASSed.

[GitHub] [spark] viirya edited a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
viirya edited a comment on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472380635 Thanks @dongjoon-hyun @HyukjinKwon @dilipbiswal

[GitHub] [spark] AmplabJenkins commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472380762 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array

2019-03-13 Thread GitBox
SparkQA commented on issue #24073: [SPARK-27134][SQL] array_distinct function does not work correctly with columns containing array of array URL: https://github.com/apache/spark/pull/24073#issuecomment-472380739 **[Test build #103419 has

[GitHub] [spark] viirya commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning

2019-03-13 Thread GitBox
viirya commented on issue #24077: [SPARK-27034][SQL][Followup] Rename ParquetSchemaPruning to SchemaPruning URL: https://github.com/apache/spark/pull/24077#issuecomment-472380635 Thanks @dongjoon-hyun @HyukjinKwon This is

[GitHub] [spark] SparkQA removed a comment on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion

2019-03-13 Thread GitBox
SparkQA removed a comment on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion URL: https://github.com/apache/spark/pull/24068#issuecomment-472309381 **[Test build #103425 has

[GitHub] [spark] SparkQA commented on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion

2019-03-13 Thread GitBox
SparkQA commented on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion URL: https://github.com/apache/spark/pull/24068#issuecomment-472383496 **[Test build #103425 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions URL: https://github.com/apache/spark/pull/24025#issuecomment-472390399 Build finished. Test PASSed.

[GitHub] [spark] cloud-fan closed pull request #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution

2019-03-13 Thread GitBox
cloud-fan closed pull request #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution URL: https://github.com/apache/spark/pull/23981 This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions URL: https://github.com/apache/spark/pull/24025#issuecomment-472390399 Build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins removed a comment on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions URL: https://github.com/apache/spark/pull/24025#issuecomment-472390405 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24025: [SPARK-27106][SQL] merge CaseInsensitiveStringMap and DataSourceOptions URL: https://github.com/apache/spark/pull/24025#issuecomment-472390405 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] cloud-fan commented on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution

2019-03-13 Thread GitBox
cloud-fan commented on issue #23981: [SPARK-27064][SS] create StreamingWrite at the beginning of streaming execution URL: https://github.com/apache/spark/pull/23981#issuecomment-472390258 thanks, merging to master! This is

[GitHub] [spark] DylanGuedes commented on issue #24071: [SPARK-27138][TESTS][KAFKA] Remove AdminUtils calls (fixes deprecation)

2019-03-13 Thread GitBox
DylanGuedes commented on issue #24071: [SPARK-27138][TESTS][KAFKA] Remove AdminUtils calls (fixes deprecation) URL: https://github.com/apache/spark/pull/24071#issuecomment-472395998 @HeartSaVioR That makes sense, I'll start to investigate other deprecated calls (such as `ZkUtils`) then.

[GitHub] [spark] cloud-fan commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled

2019-03-13 Thread GitBox
cloud-fan commented on issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INTERVAL clauses when ANSI mode enabled URL: https://github.com/apache/spark/pull/20433#issuecomment-472404309 LGTM except one comment:

[GitHub] [spark] ajithme commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information

2019-03-13 Thread GitBox
ajithme commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#discussion_r265104261 ## File path: sql/core/src/main/scala/org/apache/spark/status/api/v1/SqlListResource.scala

[GitHub] [spark] ajithme commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information

2019-03-13 Thread GitBox
ajithme commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#discussion_r265104302 ## File path: sql/core/src/main/scala/org/apache/spark/status/api/v1/ApiSqlRootResource.scala

[GitHub] [spark] ajithme commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information

2019-03-13 Thread GitBox
ajithme commented on a change in pull request #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#discussion_r265104367 ## File path: sql/core/src/main/scala/org/apache/spark/status/api/v1/SqlListResource.scala

[GitHub] [spark] cloud-fan commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

2019-03-13 Thread GitBox
cloud-fan commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#issuecomment-472407434 ok to test This is an automated message from

[GitHub] [spark] SparkQA commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

2019-03-13 Thread GitBox
SparkQA commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#issuecomment-472410018 **[Test build #103436 has

[GitHub] [spark] gengliangwang commented on issue #24076: [SPARK-27142] Provide REST API for SQL level information

2019-03-13 Thread GitBox
gengliangwang commented on issue #24076: [SPARK-27142] Provide REST API for SQL level information URL: https://github.com/apache/spark/pull/24076#issuecomment-472432302 >http://localhost:4040/api/v1/applications/{applicationId}/sql

[GitHub] [spark] pgandhi999 commented on issue #24072: [SPARK-27112] : Create a resource ordering between threads to resolve the deadlocks encountered …

2019-03-13 Thread GitBox
pgandhi999 commented on issue #24072: [SPARK-27112] : Create a resource ordering between threads to resolve the deadlocks encountered … URL: https://github.com/apache/spark/pull/24072#issuecomment-472453452 > but this does leave a small race for SPARK-19757, doesn't it? After this

[GitHub] [spark] shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

2019-03-13 Thread GitBox
shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265170087 ## File path:

[GitHub] [spark] dongjoon-hyun commented on issue #24049: [SPARK-27123][SQL] Improve CollapseProject to handle projects cross limit/repartition/sample

2019-03-13 Thread GitBox
dongjoon-hyun commented on issue #24049: [SPARK-27123][SQL] Improve CollapseProject to handle projects cross limit/repartition/sample URL: https://github.com/apache/spark/pull/24049#issuecomment-472475102 Hi, @maropu , @gatorsmile , @cloud-fan . I'll make a followup very soon. Is there

[GitHub] [spark] ajithme commented on a change in pull request #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown

2019-03-13 Thread GitBox
ajithme commented on a change in pull request #24056: [SPARK-26152] Synchronize Worker Cleanup with Worker Shutdown URL: https://github.com/apache/spark/pull/24056#discussion_r265117091 ## File path: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala ## @@

[GitHub] [spark] AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472418893 Test FAILed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-13 Thread GitBox
attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265152639 ## File path: core/src/main/scala/org/apache/spark/deploy/ExternalShuffleService.scala ## @@

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-13 Thread GitBox
attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265157078 ## File path: core/src/test/scala/org/apache/spark/deploy/worker/WorkerSuite.scala ## @@ -245,4

[GitHub] [spark] shahidki31 commented on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

2019-03-13 Thread GitBox
shahidki31 commented on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#issuecomment-472458641 Updated the PR for AllExecutionPageSuite.scala also. Jenkins, retest this please

[GitHub] [spark] dhruve commented on a change in pull request #24072: [SPARK-27112] : Create a resource ordering between threads to resolve the deadlocks encountered …

2019-03-13 Thread GitBox
dhruve commented on a change in pull request #24072: [SPARK-27112] : Create a resource ordering between threads to resolve the deadlocks encountered … URL: https://github.com/apache/spark/pull/24072#discussion_r265178089 ## File path:

[GitHub] [spark] AmplabJenkins commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#issuecomment-472409407 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#issuecomment-472409411 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

2019-03-13 Thread GitBox
AmplabJenkins commented on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#issuecomment-472409411 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

2019-03-13 Thread GitBox
AmplabJenkins removed a comment on issue #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#issuecomment-472409407 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter

2019-03-13 Thread GitBox
SparkQA commented on issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter URL: https://github.com/apache/spark/pull/20793#issuecomment-472418466 **[Test build #103430 has

[GitHub] [spark] maropu opened a new pull request #24081: [SPARK-27151][SQL] ClearCacheCommand should be case-object to avoid copys

2019-03-13 Thread GitBox
maropu opened a new pull request #24081: [SPARK-27151][SQL] ClearCacheCommand should be case-object to avoid copys URL: https://github.com/apache/spark/pull/24081 ## What changes were proposed in this pull request? To avoid unnecessary copys, this pr changed `ClearCacheCommand`into

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-13 Thread GitBox
attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265144401 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -358,6

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-13 Thread GitBox
attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265152639 ## File path: core/src/main/scala/org/apache/spark/deploy/ExternalShuffleService.scala ## @@

[GitHub] [spark] attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-13 Thread GitBox
attilapiros commented on a change in pull request #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#discussion_r265152639 ## File path: core/src/main/scala/org/apache/spark/deploy/ExternalShuffleService.scala ## @@

[GitHub] [spark] shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

2019-03-13 Thread GitBox
shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265170087 ## File path:

[GitHub] [spark] shahidki31 commented on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

2019-03-13 Thread GitBox
shahidki31 commented on issue #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#issuecomment-472462273 Jenkins, retest this please. This is

[GitHub] [spark] shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

2019-03-13 Thread GitBox
shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265170087 ## File path:

[GitHub] [spark] shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test

2019-03-13 Thread GitBox
shahidki31 commented on a change in pull request #24079: [SPARK-27145][Minor]Close store in the SQLAppStatusListenerSuite after test URL: https://github.com/apache/spark/pull/24079#discussion_r265170087 ## File path:

[GitHub] [spark] cloud-fan commented on a change in pull request #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long

2019-03-13 Thread GitBox
cloud-fan commented on a change in pull request #24019: [SPARK-27099][SQL] Add 'xxhash64' for hashing arbitrary columns to Long URL: https://github.com/apache/spark/pull/24019#discussion_r265108608 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala

<    1   2   3   4   5   6   7   8   9   10   >