[jira] [Assigned] (SPARK-25017) Add test suite for ContextBarrierState

[ https://issues.apache.org/jira/browse/SPARK-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25017: Assignee: (was: Apache Spark) > Add test suite for ContextBarrierState >

[jira] [Commented] (SPARK-25017) Add test suite for ContextBarrierState

[ https://issues.apache.org/jira/browse/SPARK-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586950#comment-16586950 ] Apache Spark commented on SPARK-25017: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-25017) Add test suite for ContextBarrierState

[ https://issues.apache.org/jira/browse/SPARK-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25017: Assignee: Apache Spark > Add test suite for ContextBarrierState >

[jira] [Commented] (SPARK-25167) Minor fixes for R sql tests (tests that fail in development environment)

[ https://issues.apache.org/jira/browse/SPARK-25167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586923#comment-16586923 ] Apache Spark commented on SPARK-25167: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-25167) Minor fixes for R sql tests (tests that fail in development environment)

[ https://issues.apache.org/jira/browse/SPARK-25167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25167: Assignee: Apache Spark > Minor fixes for R sql tests (tests that fail in development

[jira] [Assigned] (SPARK-25167) Minor fixes for R sql tests (tests that fail in development environment)

[ https://issues.apache.org/jira/browse/SPARK-25167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25167: Assignee: (was: Apache Spark) > Minor fixes for R sql tests (tests that fail in

[jira] [Created] (SPARK-25167) Minor fixes for R sql tests (tests that fail in development environment)

Dilip Biswal created SPARK-25167: Summary: Minor fixes for R sql tests (tests that fail in development environment) Key: SPARK-25167 URL: https://issues.apache.org/jira/browse/SPARK-25167 Project:

[jira] [Assigned] (SPARK-23679) uiWebUrl show inproper URL when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23679: Assignee: Apache Spark > uiWebUrl show inproper URL when running on YARN >

[jira] [Assigned] (SPARK-23679) uiWebUrl show inproper URL when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23679: Assignee: (was: Apache Spark) > uiWebUrl show inproper URL when running on YARN >

[jira] [Commented] (SPARK-23679) uiWebUrl show inproper URL when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586907#comment-16586907 ] Apache Spark commented on SPARK-23679: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-25165) Cannot parse Hive Struct

[ https://issues.apache.org/jira/browse/SPARK-25165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586901#comment-16586901 ] Frank Yin edited comment on SPARK-25165 at 8/21/18 3:48 AM:    

[jira] [Comment Edited] (SPARK-25165) Cannot parse Hive Struct

[ https://issues.apache.org/jira/browse/SPARK-25165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586901#comment-16586901 ] Frank Yin edited comment on SPARK-25165 at 8/21/18 3:48 AM:    

[jira] [Comment Edited] (SPARK-25165) Cannot parse Hive Struct

[ https://issues.apache.org/jira/browse/SPARK-25165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586901#comment-16586901 ] Frank Yin edited comment on SPARK-25165 at 8/21/18 3:47 AM:    

[jira] [Commented] (SPARK-25165) Cannot parse Hive Struct

[ https://issues.apache.org/jira/browse/SPARK-25165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586901#comment-16586901 ] Frank Yin commented on SPARK-25165: --- #!/usr/bin/env python # -*- coding: UTF-8 -*- # encoding=utf8

[jira] [Commented] (SPARK-25157) Streaming of image files from directory

[ https://issues.apache.org/jira/browse/SPARK-25157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586847#comment-16586847 ] Hyukjin Kwon commented on SPARK-25157: -- Is this blocked by SPARK-22666? > Streaming of image

[jira] [Commented] (SPARK-25165) Cannot parse Hive Struct

[ https://issues.apache.org/jira/browse/SPARK-25165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586842#comment-16586842 ] Hyukjin Kwon commented on SPARK-25165: -- Mind if I ask a reproducer please? > Cannot parse Hive

[jira] [Assigned] (SPARK-25166) Reduce the number of write operations for shuffle write.

[ https://issues.apache.org/jira/browse/SPARK-25166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25166: Assignee: Apache Spark > Reduce the number of write operations for shuffle write. >

[jira] [Assigned] (SPARK-25166) Reduce the number of write operations for shuffle write.

[ https://issues.apache.org/jira/browse/SPARK-25166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25166: Assignee: (was: Apache Spark) > Reduce the number of write operations for shuffle

[jira] [Commented] (SPARK-25166) Reduce the number of write operations for shuffle write.

[ https://issues.apache.org/jira/browse/SPARK-25166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586831#comment-16586831 ] Apache Spark commented on SPARK-25166: -- User '10110346' has created a pull request for this issue:

[jira] [Updated] (SPARK-25166) Reduce the number of write operations for shuffle write.

[ https://issues.apache.org/jira/browse/SPARK-25166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-25166: Description: Currently, only one record is written to a buffer each time, which increases the number of

[jira] [Created] (SPARK-25166) Reduce the number of write operations for shuffle write.

liuxian created SPARK-25166: --- Summary: Reduce the number of write operations for shuffle write. Key: SPARK-25166 URL: https://issues.apache.org/jira/browse/SPARK-25166 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-25132) Case-insensitive field resolution when reading from Parquet/ORC

[ https://issues.apache.org/jira/browse/SPARK-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25132. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22148

[jira] [Assigned] (SPARK-25132) Case-insensitive field resolution when reading from Parquet/ORC

[ https://issues.apache.org/jira/browse/SPARK-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25132: Assignee: Chenxiao Mao > Case-insensitive field resolution when reading from Parquet/ORC

[jira] [Updated] (SPARK-25134) Csv column pruning with checking of headers throws incorrect error

[ https://issues.apache.org/jira/browse/SPARK-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25134: - Fix Version/s: 2.4.0 > Csv column pruning with checking of headers throws incorrect error >

[jira] [Resolved] (SPARK-25134) Csv column pruning with checking of headers throws incorrect error

[ https://issues.apache.org/jira/browse/SPARK-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25134. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/22123 > Csv column pruning

[jira] [Updated] (SPARK-25134) Csv column pruning with checking of headers throws incorrect error

[ https://issues.apache.org/jira/browse/SPARK-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25134: - Priority: Major (was: Minor) > Csv column pruning with checking of headers throws incorrect

[jira] [Assigned] (SPARK-25134) Csv column pruning with checking of headers throws incorrect error

[ https://issues.apache.org/jira/browse/SPARK-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25134: Assignee: Koert Kuipers > Csv column pruning with checking of headers throws incorrect

[jira] [Updated] (SPARK-25134) Csv column pruning with checking of headers throws incorrect error

[ https://issues.apache.org/jira/browse/SPARK-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25134: - Affects Version/s: (was: 2.3.1) 2.4.0 > Csv column pruning with

[jira] [Resolved] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25144. -- Resolution: Fixed > distinct on Dataset leads to exception due to Managed memory leak

[jira] [Assigned] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25144: Assignee: Dongjoon Hyun > distinct on Dataset leads to exception due to Managed memory

[jira] [Updated] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25144: - Fix Version/s: 2.3.2 2.2.3 > distinct on Dataset leads to exception due to

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586647#comment-16586647 ] Xin Yao commented on SPARK-23128: - Thanks [~XuanYuan] for sharing those awesome numbers.   If I

[jira] [Commented] (SPARK-23874) Upgrade apache/arrow to 0.10.0

[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586612#comment-16586612 ] Bryan Cutler commented on SPARK-23874: -- For the Python fixes, yes the user would have to upgrade

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586605#comment-16586605 ] Xin Yao commented on SPARK-23128: - Thank [~carsonwang] for the great work. This looks really

[jira] [Commented] (SPARK-23874) Upgrade apache/arrow to 0.10.0

[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586593#comment-16586593 ] Xiao Li commented on SPARK-23874: - [~bryanc]To get these fixes, we need to upgrade pyarrow to 0.10,

[jira] [Commented] (SPARK-24442) Add configuration parameter to adjust the numbers of records and the charters per row before truncation when a user runs.show()

[ https://issues.apache.org/jira/browse/SPARK-24442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586544#comment-16586544 ] Apache Spark commented on SPARK-24442: -- User 'AndrewKL' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24442) Add configuration parameter to adjust the numbers of records and the charters per row before truncation when a user runs.show()

[ https://issues.apache.org/jira/browse/SPARK-24442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24442: Assignee: (was: Apache Spark) > Add configuration parameter to adjust the numbers of

[jira] [Assigned] (SPARK-24442) Add configuration parameter to adjust the numbers of records and the charters per row before truncation when a user runs.show()

[ https://issues.apache.org/jira/browse/SPARK-24442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24442: Assignee: Apache Spark > Add configuration parameter to adjust the numbers of records

[jira] [Commented] (SPARK-24442) Add configuration parameter to adjust the numbers of records and the charters per row before truncation when a user runs.show()

[ https://issues.apache.org/jira/browse/SPARK-24442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586545#comment-16586545 ] Andrew K Long commented on SPARK-24442: --- I've created a pull request!

[jira] [Created] (SPARK-25165) Cannot parse Hive Struct

Frank Yin created SPARK-25165: - Summary: Cannot parse Hive Struct Key: SPARK-25165 URL: https://issues.apache.org/jira/browse/SPARK-25165 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-24418) Upgrade to Scala 2.11.12

[ https://issues.apache.org/jira/browse/SPARK-24418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586483#comment-16586483 ] Apache Spark commented on SPARK-24418: -- User 'dbtsai' has created a pull request for this issue:

[jira] [Resolved] (SPARK-24639) Add three configs in the doc

[ https://issues.apache.org/jira/browse/SPARK-24639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24639. --- Resolution: Won't Fix > Add three configs in the doc > > >

[jira] [Resolved] (SPARK-24834) Utils#nanSafeCompare{Double,Float} functions do not differ from normal java double/float comparison

[ https://issues.apache.org/jira/browse/SPARK-24834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24834. --- Resolution: Won't Fix See PR – I think we can't do this directly as it changes semantics

[jira] [Created] (SPARK-25164) Parquet reader builds entire list of columns once for each column

Bruce Robbins created SPARK-25164: - Summary: Parquet reader builds entire list of columns once for each column Key: SPARK-25164 URL: https://issues.apache.org/jira/browse/SPARK-25164 Project: Spark

[jira] [Commented] (SPARK-21375) Add date and timestamp support to ArrowConverters for toPandas() collection

[ https://issues.apache.org/jira/browse/SPARK-21375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586389#comment-16586389 ] Eric Wohlstadter commented on SPARK-21375: -- [~bryanc] Thanks for the feedback. I think the

[jira] [Commented] (SPARK-24432) Add support for dynamic resource allocation

[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586378#comment-16586378 ] James Carter commented on SPARK-24432: -- We're looking to shift our production Spark workloads from 

[jira] [Commented] (SPARK-21375) Add date and timestamp support to ArrowConverters for toPandas() collection

[ https://issues.apache.org/jira/browse/SPARK-21375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586369#comment-16586369 ] Bryan Cutler commented on SPARK-21375: -- Hi [~ewohlstadter], the timestamp values should be in UTC.

[jira] [Updated] (SPARK-25162) Kubernetes 'in-cluster' client mode and value of spark.driver.host

[ https://issues.apache.org/jira/browse/SPARK-25162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Carter updated SPARK-25162: - Summary: Kubernetes 'in-cluster' client mode and value of spark.driver.host (was: Kubernetes

[jira] [Updated] (SPARK-25162) Kubernetes 'in-cluster' client mode and value spark.driver.host

[ https://issues.apache.org/jira/browse/SPARK-25162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Carter updated SPARK-25162: - Description: When creating Kubernetes scheduler 'in-cluster' using client mode, the value for

[jira] [Created] (SPARK-25163) Flaky test: o.a.s.util.collection.ExternalAppendOnlyMapSuite.spilling with compression

Shixiong Zhu created SPARK-25163: Summary: Flaky test: o.a.s.util.collection.ExternalAppendOnlyMapSuite.spilling with compression Key: SPARK-25163 URL: https://issues.apache.org/jira/browse/SPARK-25163

[jira] [Created] (SPARK-25162) Kubernetes 'in-cluster' client mode and value spark.driver.host

James Carter created SPARK-25162: Summary: Kubernetes 'in-cluster' client mode and value spark.driver.host Key: SPARK-25162 URL: https://issues.apache.org/jira/browse/SPARK-25162 Project: Spark

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586314#comment-16586314 ] Yinan Li commented on SPARK-24434: -- [~skonto] I will make sure the assignee gets properly set for

[jira] [Assigned] (SPARK-25161) Fix several bugs in failure handling of barrier execution mode

[ https://issues.apache.org/jira/browse/SPARK-25161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25161: Assignee: (was: Apache Spark) > Fix several bugs in failure handling of barrier

[jira] [Assigned] (SPARK-25161) Fix several bugs in failure handling of barrier execution mode

[ https://issues.apache.org/jira/browse/SPARK-25161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25161: Assignee: Apache Spark > Fix several bugs in failure handling of barrier execution mode

[jira] [Commented] (SPARK-25161) Fix several bugs in failure handling of barrier execution mode

[ https://issues.apache.org/jira/browse/SPARK-25161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586268#comment-16586268 ] Apache Spark commented on SPARK-25161: -- User 'jiangxb1987' has created a pull request for this

[jira] [Created] (SPARK-25161) Fix several bugs in failure handling of barrier execution mode

Jiang Xingbo created SPARK-25161: Summary: Fix several bugs in failure handling of barrier execution mode Key: SPARK-25161 URL: https://issues.apache.org/jira/browse/SPARK-25161 Project: Spark

[jira] [Commented] (SPARK-25126) avoid creating OrcFile.Reader for all orc files

[ https://issues.apache.org/jira/browse/SPARK-25126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586237#comment-16586237 ] Apache Spark commented on SPARK-25126: -- User 'raofu' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23573) Create linter rule to prevent misuse of SparkContext.hadoopConfiguration in SQL modules

[ https://issues.apache.org/jira/browse/SPARK-23573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23573. - Resolution: Duplicate Assignee: Gengliang Wang Fix Version/s: 2.4.0 > Create linter

[jira] [Assigned] (SPARK-25126) avoid creating OrcFile.Reader for all orc files

[ https://issues.apache.org/jira/browse/SPARK-25126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25126: Assignee: (was: Apache Spark) > avoid creating OrcFile.Reader for all orc files >

[jira] [Assigned] (SPARK-25126) avoid creating OrcFile.Reader for all orc files

[ https://issues.apache.org/jira/browse/SPARK-25126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25126: Assignee: Apache Spark > avoid creating OrcFile.Reader for all orc files >

[jira] [Commented] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586168#comment-16586168 ] Apache Spark commented on SPARK-25144: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586138#comment-16586138 ] Apache Spark commented on SPARK-25144: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-25140) Add optional logging to UnsafeProjection.create when it falls back to interpreted mode

[ https://issues.apache.org/jira/browse/SPARK-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586091#comment-16586091 ] Apache Spark commented on SPARK-25140: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25140) Add optional logging to UnsafeProjection.create when it falls back to interpreted mode

[ https://issues.apache.org/jira/browse/SPARK-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25140: Assignee: (was: Apache Spark) > Add optional logging to UnsafeProjection.create when

[jira] [Assigned] (SPARK-25140) Add optional logging to UnsafeProjection.create when it falls back to interpreted mode

[ https://issues.apache.org/jira/browse/SPARK-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25140: Assignee: Apache Spark > Add optional logging to UnsafeProjection.create when it falls

[jira] [Commented] (SPARK-23711) Add fallback to interpreted execution logic

[ https://issues.apache.org/jira/browse/SPARK-23711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586090#comment-16586090 ] Apache Spark commented on SPARK-23711: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-25147) GroupedData.apply pandas_udf crashing

[ https://issues.apache.org/jira/browse/SPARK-25147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585911#comment-16585911 ] Mike Sukmanowsky commented on SPARK-25147: -- Confirmed, works with: Python 2.7.15

[jira] [Comment Edited] (SPARK-19498) Discussion: Making MLlib APIs extensible for 3rd party libraries

[ https://issues.apache.org/jira/browse/SPARK-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527351#comment-16527351 ] Lucas Partridge edited comment on SPARK-19498 at 8/20/18 1:03 PM: -- Ok

[jira] [Comment Edited] (SPARK-19498) Discussion: Making MLlib APIs extensible for 3rd party libraries

[ https://issues.apache.org/jira/browse/SPARK-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527351#comment-16527351 ] Lucas Partridge edited comment on SPARK-19498 at 8/20/18 1:02 PM: -- Ok

[jira] [Resolved] (SPARK-25160) Remove sql configuration spark.sql.avro.outputTimestampType

[ https://issues.apache.org/jira/browse/SPARK-25160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25160. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22151

[jira] [Assigned] (SPARK-25160) Remove sql configuration spark.sql.avro.outputTimestampType

[ https://issues.apache.org/jira/browse/SPARK-25160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25160: Assignee: Gengliang Wang > Remove sql configuration spark.sql.avro.outputTimestampType >

[jira] [Commented] (SPARK-23034) Display tablename for `HiveTableScan` node in UI

[ https://issues.apache.org/jira/browse/SPARK-23034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585786#comment-16585786 ] Apache Spark commented on SPARK-23034: -- User 'maropu' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585689#comment-16585689 ] Stavros Kontopoulos edited comment on SPARK-24434 at 8/20/18 10:17 AM:

[jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585689#comment-16585689 ] Stavros Kontopoulos edited comment on SPARK-24434 at 8/20/18 9:34 AM:

[jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585689#comment-16585689 ] Stavros Kontopoulos edited comment on SPARK-24434 at 8/20/18 9:32 AM:

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585689#comment-16585689 ] Stavros Kontopoulos commented on SPARK-24434: - [~onursatici] I am working on this, but since

[jira] [Assigned] (SPARK-25159) json schema inference should only trigger one job

[ https://issues.apache.org/jira/browse/SPARK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25159: Assignee: Apache Spark (was: Wenchen Fan) > json schema inference should only trigger

[jira] [Assigned] (SPARK-25159) json schema inference should only trigger one job

[ https://issues.apache.org/jira/browse/SPARK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25159: Assignee: Wenchen Fan (was: Apache Spark) > json schema inference should only trigger

[jira] [Commented] (SPARK-25159) json schema inference should only trigger one job

[ https://issues.apache.org/jira/browse/SPARK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585631#comment-16585631 ] Apache Spark commented on SPARK-25159: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25160) Remove sql configuration spark.sql.avro.outputTimestampType

[ https://issues.apache.org/jira/browse/SPARK-25160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25160: Assignee: Apache Spark > Remove sql configuration spark.sql.avro.outputTimestampType >

[jira] [Assigned] (SPARK-25160) Remove sql configuration spark.sql.avro.outputTimestampType

[ https://issues.apache.org/jira/browse/SPARK-25160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25160: Assignee: (was: Apache Spark) > Remove sql configuration

[jira] [Commented] (SPARK-25160) Remove sql configuration spark.sql.avro.outputTimestampType

[ https://issues.apache.org/jira/browse/SPARK-25160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585609#comment-16585609 ] Apache Spark commented on SPARK-25160: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-25160) Remove sql configuration spark.sql.avro.outputTimestampType

Gengliang Wang created SPARK-25160: -- Summary: Remove sql configuration spark.sql.avro.outputTimestampType Key: SPARK-25160 URL: https://issues.apache.org/jira/browse/SPARK-25160 Project: Spark

[jira] [Comment Edited] (SPARK-23714) Add metrics for cached KafkaConsumer

[ https://issues.apache.org/jira/browse/SPARK-23714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585587#comment-16585587 ] Jungtaek Lim edited comment on SPARK-23714 at 8/20/18 8:06 AM: ---

[jira] [Commented] (SPARK-23714) Add metrics for cached KafkaConsumer

[ https://issues.apache.org/jira/browse/SPARK-23714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585587#comment-16585587 ] Jungtaek Lim commented on SPARK-23714: -- [~yuzhih...@gmail.com] Maybe we can just apply Apache

[jira] [Commented] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585554#comment-16585554 ] Ayoub Benali commented on SPARK-25144: -- [~jerryshao] according the comments above this bug doesn't

[jira] [Created] (SPARK-25159) json schema inference should only trigger one job

Wenchen Fan created SPARK-25159: --- Summary: json schema inference should only trigger one job Key: SPARK-25159 URL: https://issues.apache.org/jira/browse/SPARK-25159 Project: Spark Issue Type:

[jira] [Updated] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-25144: Target Version/s: 2.2.3, 2.3.2 > distinct on Dataset leads to exception due to Managed memory

[jira] [Commented] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585482#comment-16585482 ] Saisai Shao commented on SPARK-25144: - Does this exist in master branch? > distinct on Dataset

[jira] [Commented] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585481#comment-16585481 ] Dongjoon Hyun commented on SPARK-25144: --- Ping [~jerryshao] since you are a release engineer. >

[jira] [Updated] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25144: -- Affects Version/s: 2.0.2 > distinct on Dataset leads to exception due to Managed memory leak

[jira] [Commented] (SPARK-23679) uiWebUrl show inproper URL when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585473#comment-16585473 ] Saisai Shao commented on SPARK-23679: - Let me fix this issue. This is mainly a RM HA introduced

[jira] [Updated] (SPARK-23679) uiWebUrl show inproper URL when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23679: Component/s: YARN > uiWebUrl show inproper URL when running on YARN >

[jira] [Assigned] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25144: Assignee: Apache Spark > distinct on Dataset leads to exception due to Managed memory

[jira] [Assigned] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25144: Assignee: (was: Apache Spark) > distinct on Dataset leads to exception due to

[jira] [Commented] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585472#comment-16585472 ] Apache Spark commented on SPARK-25144: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25144: -- Environment: (was: spark 2.3.1) > distinct on Dataset leads to exception due to Managed

[jira] [Updated] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25144: -- Affects Version/s: 2.3.2 2.1.3 2.2.2 > distinct

[jira] [Updated] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25144: -- Component/s: (was: Spark Core) (was: Optimizer) > distinct on

[jira] [Reopened] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-25144: --- I'll reopen this since I can reproduce this in 2.1.3, 2.2.2, 2.3.2-RC5. I found the difference

  1   2   >