[jira] [Updated] (SPARK-30675) Spark Streaming Job stopped reading events from Queue upon Deregister Exception

[ https://issues.apache.org/jira/browse/SPARK-30675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mullaivendhan Ariaputhri updated SPARK-30675: - Attachment: Instance-Config-P2.JPG

[jira] [Updated] (SPARK-30675) Spark Streaming Job stopped reading events from Queue upon Deregister Exception

[ https://issues.apache.org/jira/browse/SPARK-30675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mullaivendhan Ariaputhri updated SPARK-30675: - Description:   *+Stream+* We have observed discrepancy in  kinesis

[jira] [Resolved] (SPARK-30665) Eliminate pypandoc dependency

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30665. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27376

[jira] [Assigned] (SPARK-30665) Eliminate pypandoc dependency

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30665: Assignee: Nicholas Chammas > Eliminate pypandoc dependency >

[jira] [Updated] (SPARK-30675) Spark Streaming Job stopped reading events from Queue upon Deregister Exception

[ https://issues.apache.org/jira/browse/SPARK-30675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mullaivendhan Ariaputhri updated SPARK-30675: - Description:   *+Stream+* We have observed discrepancy in  kinesis

[jira] [Updated] (SPARK-30675) Spark Streaming Job stopped reading events from Queue upon Deregister Exception

[ https://issues.apache.org/jira/browse/SPARK-30675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mullaivendhan Ariaputhri updated SPARK-30675: - Description:   *+Stream+* We have observed discrepancy in  kinesis

[jira] [Updated] (SPARK-30675) Spark Streaming Job stopped reading events from Queue upon Deregister Exception

[ https://issues.apache.org/jira/browse/SPARK-30675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mullaivendhan Ariaputhri updated SPARK-30675: - Description:   *+Stream+* We have observed discrepancy in  kinesis

[jira] [Updated] (SPARK-30675) Spark Streaming Job stopped reading events from Queue upon Deregister Exception

[ https://issues.apache.org/jira/browse/SPARK-30675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mullaivendhan Ariaputhri updated SPARK-30675: - Description:   *+Stream+* We have observed discrepancy in  kinesis

[jira] [Updated] (SPARK-30675) Spark Streaming Job stopped reading events from Queue upon Deregister Exception

[ https://issues.apache.org/jira/browse/SPARK-30675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-30675: - Component/s: (was: Structured Streaming) (was: Spark Submit)

[jira] [Updated] (SPARK-30675) Spark Streaming Job stopped reading events from Queue upon Deregister Exception

[ https://issues.apache.org/jira/browse/SPARK-30675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mullaivendhan Ariaputhri updated SPARK-30675: - Description: *+Spark/EMR+* >From the driver logs, it has been found

[jira] [Created] (SPARK-30675) Spark Streaming Job stopped reading events from Queue upon Deregister Exception

Mullaivendhan Ariaputhri created SPARK-30675: Summary: Spark Streaming Job stopped reading events from Queue upon Deregister Exception Key: SPARK-30675 URL:

[jira] [Updated] (SPARK-30674) Use python3 in dev/lint-python

[ https://issues.apache.org/jira/browse/SPARK-30674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30674: -- Description: `lint-python` fails at python2. We had better use python3 explicitly. (was:

[jira] [Created] (SPARK-30674) Use python3 in dev/lint-python

Dongjoon Hyun created SPARK-30674: - Summary: Use python3 in dev/lint-python Key: SPARK-30674 URL: https://issues.apache.org/jira/browse/SPARK-30674 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-30673) Test cases in HiveShowCreateTableSuite should create Hive table instead of Datasource table

L. C. Hsieh created SPARK-30673: --- Summary: Test cases in HiveShowCreateTableSuite should create Hive table instead of Datasource table Key: SPARK-30673 URL: https://issues.apache.org/jira/browse/SPARK-30673

[jira] [Updated] (SPARK-30665) Eliminate pypandoc dependency

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30665: - Component/s: Build > Eliminate pypandoc dependency > - > >

[jira] [Updated] (SPARK-30665) Eliminate pypandoc dependency

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30665: - Component/s: (was: Build) Documentation > Eliminate pypandoc dependency >

[jira] [Comment Edited] (SPARK-21823) ALTER TABLE table statements such as RENAME and CHANGE columns should raise error if there are any dependent constraints.

[ https://issues.apache.org/jira/browse/SPARK-21823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025611#comment-17025611 ] sakshi chourasia edited comment on SPARK-21823 at 1/30/20 5:20 AM: --- Hi

[jira] [Assigned] (SPARK-30435) update Spark SQL guide of Supported Hive Features

[ https://issues.apache.org/jira/browse/SPARK-30435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30435: - Assignee: angerszhu > update Spark SQL guide of Supported Hive Features >

[jira] [Resolved] (SPARK-30435) update Spark SQL guide of Supported Hive Features

[ https://issues.apache.org/jira/browse/SPARK-30435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30435. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27106

[jira] [Resolved] (SPARK-30672) numpy is a dependency for building PySpark API docs

[ https://issues.apache.org/jira/browse/SPARK-30672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30672. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27390

[jira] [Assigned] (SPARK-30672) numpy is a dependency for building PySpark API docs

[ https://issues.apache.org/jira/browse/SPARK-30672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30672: Assignee: Nicholas Chammas > numpy is a dependency for building PySpark API docs >

[jira] [Comment Edited] (SPARK-21823) ALTER TABLE table statements such as RENAME and CHANGE columns should raise error if there are any dependent constraints.

[ https://issues.apache.org/jira/browse/SPARK-21823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025611#comment-17025611 ] sakshi chourasia edited comment on SPARK-21823 at 1/30/20 3:56 AM: --- Hi

[jira] [Updated] (SPARK-30665) Eliminate pypandoc dependency

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30665: - Summary: Eliminate pypandoc dependency (was: Remove Pandoc dependency in PySpark

[jira] [Created] (SPARK-30672) numpy is a dependency for building PySpark API docs

Nicholas Chammas created SPARK-30672: Summary: numpy is a dependency for building PySpark API docs Key: SPARK-30672 URL: https://issues.apache.org/jira/browse/SPARK-30672 Project: Spark

[jira] [Commented] (SPARK-30665) Remove Pandoc dependency in PySpark setup.py

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026399#comment-17026399 ] Nicholas Chammas commented on SPARK-30665: --  > Remove Pandoc dependency in PySpark setup.py >

[jira] [Resolved] (SPARK-30618) Why does SparkSQL allow `WHERE` to be table alias?

[ https://issues.apache.org/jira/browse/SPARK-30618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30618. -- Resolution: Invalid Please ask questions into mailing list. See 

[jira] [Commented] (SPARK-30619) org.slf4j.Logger and org.apache.commons.collections classes not built as part of hadoop-provided profile

[ https://issues.apache.org/jira/browse/SPARK-30619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026391#comment-17026391 ] Hyukjin Kwon commented on SPARK-30619: -- [~abhisrao] can you show reproducer and error messages? >

[jira] [Resolved] (SPARK-30643) Add support for embedding Hive 3

[ https://issues.apache.org/jira/browse/SPARK-30643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30643. -- Resolution: Later > Add support for embedding Hive 3 > > >

[jira] [Commented] (SPARK-30643) Add support for embedding Hive 3

[ https://issues.apache.org/jira/browse/SPARK-30643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026389#comment-17026389 ] Hyukjin Kwon commented on SPARK-30643: -- Yeah, it needs some huge efforts to upgrade this, and a lot

[jira] [Commented] (SPARK-30646) transform_keys function throws exception as "Cannot use null as map key", but there isn't any null key in the map

[ https://issues.apache.org/jira/browse/SPARK-30646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026385#comment-17026385 ] Hyukjin Kwon commented on SPARK-30646: -- Use concat to concatenate strings {code:java} scala>

[jira] [Resolved] (SPARK-30646) transform_keys function throws exception as "Cannot use null as map key", but there isn't any null key in the map

[ https://issues.apache.org/jira/browse/SPARK-30646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30646. -- Resolution: Invalid > transform_keys function throws exception as "Cannot use null as map

[jira] [Resolved] (SPARK-30647) When creating a custom datasource File NotFoundExpection happens

[ https://issues.apache.org/jira/browse/SPARK-30647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30647. -- Resolution: Incomplete > When creating a custom datasource File NotFoundExpection happens >

[jira] [Commented] (SPARK-30647) When creating a custom datasource File NotFoundExpection happens

[ https://issues.apache.org/jira/browse/SPARK-30647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026374#comment-17026374 ] Hyukjin Kwon commented on SPARK-30647: -- I think this is fixed in the latest versions of Spark.

[jira] [Commented] (SPARK-30649) Azure Spark read : ContentMD5 header is missing in the response

[ https://issues.apache.org/jira/browse/SPARK-30649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026372#comment-17026372 ] Hyukjin Kwon commented on SPARK-30649: -- Firstly please just don't copy and paste the error

[jira] [Resolved] (SPARK-30649) Azure Spark read : ContentMD5 header is missing in the response

[ https://issues.apache.org/jira/browse/SPARK-30649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30649. -- Resolution: Incomplete > Azure Spark read : ContentMD5 header is missing in the response >

[jira] [Updated] (SPARK-30649) Azure Spark read : ContentMD5 header is missing in the response

[ https://issues.apache.org/jira/browse/SPARK-30649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30649: - Priority: Major (was: Blocker) > Azure Spark read : ContentMD5 header is missing in the

[jira] [Commented] (SPARK-30650) The parquet file written by spark often incurs corrupted footer and hence not readable

[ https://issues.apache.org/jira/browse/SPARK-30650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026369#comment-17026369 ] Hyukjin Kwon commented on SPARK-30650: -- Spark versions before 2.3 are EOL. Can you verify if there

[jira] [Resolved] (SPARK-30650) The parquet file written by spark often incurs corrupted footer and hence not readable

[ https://issues.apache.org/jira/browse/SPARK-30650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30650. -- Resolution: Incomplete > The parquet file written by spark often incurs corrupted footer and

[jira] [Commented] (SPARK-29578) JDK 1.8.0_232 timezone updates cause "Kwajalein" test failures again

[ https://issues.apache.org/jira/browse/SPARK-29578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026365#comment-17026365 ] Dongjoon Hyun commented on SPARK-29578: --- This lands at `branch-2.4` via

[jira] [Updated] (SPARK-29578) JDK 1.8.0_232 timezone updates cause "Kwajalein" test failures again

[ https://issues.apache.org/jira/browse/SPARK-29578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29578: -- Fix Version/s: 2.4.5 > JDK 1.8.0_232 timezone updates cause "Kwajalein" test failures again >

[jira] [Commented] (SPARK-30670) Pipes for PySpark

[ https://issues.apache.org/jira/browse/SPARK-30670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026354#comment-17026354 ] Hyukjin Kwon commented on SPARK-30670: -- There is already {{transform}}.  > Pipes for PySpark >

[jira] [Resolved] (SPARK-30670) Pipes for PySpark

[ https://issues.apache.org/jira/browse/SPARK-30670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30670. -- Resolution: Duplicate > Pipes for PySpark > - > > Key:

[jira] [Resolved] (SPARK-30529) Improve error messages when Executor dies before registering with driver

[ https://issues.apache.org/jira/browse/SPARK-30529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30529. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27385

[jira] [Assigned] (SPARK-30529) Improve error messages when Executor dies before registering with driver

[ https://issues.apache.org/jira/browse/SPARK-30529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30529: - Assignee: Thomas Graves > Improve error messages when Executor dies before registering

[jira] [Commented] (SPARK-29419) Seq.toDS / spark.createDataset(Seq) is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-29419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026313#comment-17026313 ] Dongjoon Hyun commented on SPARK-29419: --- I switched this to `Bug` because this is marked as

[jira] [Updated] (SPARK-29419) Seq.toDS / spark.createDataset(Seq) is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-29419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29419: -- Issue Type: Bug (was: Improvement) > Seq.toDS / spark.createDataset(Seq) is not thread-safe

[jira] [Updated] (SPARK-29492) SparkThriftServer can't support jar class as table serde class when executestatement in sync mode

[ https://issues.apache.org/jira/browse/SPARK-29492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29492: -- Affects Version/s: (was: 2.4.0) > SparkThriftServer can't support jar class as table

[jira] [Updated] (SPARK-30538) A not very elegant way to control ouput small file

[ https://issues.apache.org/jira/browse/SPARK-30538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30538: -- Affects Version/s: (was: 2.4.0) > A not very elegant way to control ouput small file >

[jira] [Updated] (SPARK-29995) Structured Streaming file-sink log grow indefinitely

[ https://issues.apache.org/jira/browse/SPARK-29995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29995: -- Issue Type: Bug (was: Improvement) > Structured Streaming file-sink log grow indefinitely >

[jira] [Updated] (SPARK-29799) Split a kafka partition into multiple KafkaRDD partitions in the kafka external plugin for Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-29799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29799: -- Affects Version/s: (was: 2.4.3) (was: 2.1.0)

[jira] [Updated] (SPARK-28817) Support standard Javadoc packaging to allow automatic javadoc location settings

[ https://issues.apache.org/jira/browse/SPARK-28817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28817: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Support standard Javadoc

[jira] [Updated] (SPARK-28452) CSV datasource writer do not support maxCharsPerColumn option

[ https://issues.apache.org/jira/browse/SPARK-28452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28452: -- Affects Version/s: (was: 2.4.3) 3.0.0 > CSV datasource writer do

[jira] [Updated] (SPARK-29952) Pandas UDFs do not support vectors as input

[ https://issues.apache.org/jira/browse/SPARK-29952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29952: -- Affects Version/s: (was: 2.4.3) 3.0.0 > Pandas UDFs do not support

[jira] [Updated] (SPARK-30669) Introduce AdmissionControl API to Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-30669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30669: -- Affects Version/s: (was: 2.4.4) 3.0.0 > Introduce AdmissionControl

[jira] [Commented] (SPARK-30665) Remove Pandoc dependency in PySpark setup.py

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026309#comment-17026309 ] Dongjoon Hyun commented on SPARK-30665: --- Hi, [~nchammas].  For `Improvement`, `Affected Version`

[jira] [Updated] (SPARK-30665) Remove Pandoc dependency in PySpark setup.py

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30665: -- Affects Version/s: (was: 2.4.4) (was: 2.4.3)

[jira] [Commented] (SPARK-28594) Allow event logs for running streaming apps to be rolled over.

[ https://issues.apache.org/jira/browse/SPARK-28594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026307#comment-17026307 ] Jungtaek Lim commented on SPARK-28594: -- While I commented some tasks for improvement, technically

[jira] [Resolved] (SPARK-28594) Allow event logs for running streaming apps to be rolled over.

[ https://issues.apache.org/jira/browse/SPARK-28594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-28594. -- Fix Version/s: 3.0.0 Resolution: Fixed > Allow event logs for running streaming apps

[jira] [Commented] (SPARK-29367) pandas udf not working with latest pyarrow release (0.15.0)

[ https://issues.apache.org/jira/browse/SPARK-29367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026292#comment-17026292 ] Dongjoon Hyun commented on SPARK-29367: --- Adjusted document is added to `branch-2.4` via

[jira] [Updated] (SPARK-29367) pandas udf not working with latest pyarrow release (0.15.0)

[ https://issues.apache.org/jira/browse/SPARK-29367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29367: -- Fix Version/s: 2.4.5 > pandas udf not working with latest pyarrow release (0.15.0) >

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026287#comment-17026287 ] Foster Langbein commented on SPARK-12312: - Yes we followed the same approach as referred to by

[jira] [Updated] (SPARK-30310) SparkUncaughtExceptionHandler halts running process unexpectedly

[ https://issues.apache.org/jira/browse/SPARK-30310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30310: -- Fix Version/s: 2.4.5 > SparkUncaughtExceptionHandler halts running process unexpectedly >

[jira] [Commented] (SPARK-30310) SparkUncaughtExceptionHandler halts running process unexpectedly

[ https://issues.apache.org/jira/browse/SPARK-30310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026286#comment-17026286 ] Dongjoon Hyun commented on SPARK-30310: --- This is backported to branch-2.4 via

[jira] [Created] (SPARK-30671) SparkSession emptyDataFrame should not create an RDD

Herman van Hövell created SPARK-30671: - Summary: SparkSession emptyDataFrame should not create an RDD Key: SPARK-30671 URL: https://issues.apache.org/jira/browse/SPARK-30671 Project: Spark

[jira] [Resolved] (SPARK-29543) Support Structured Streaming UI

[ https://issues.apache.org/jira/browse/SPARK-29543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-29543. -- Fix Version/s: 3.0.0 Assignee: Genmao Yu Resolution: Done > Support

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026245#comment-17026245 ] John Lonergan commented on SPARK-12312: --- @nabacg funnily enough we came up with almost the same

[jira] [Commented] (SPARK-27733) Upgrade to Avro 1.9.x

[ https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026240#comment-17026240 ] Ismaël Mejía commented on SPARK-27733: -- Oh I was not aware that Spark will rely still on 1.2.1

[jira] [Updated] (SPARK-30512) Use a dedicated boss event group loop in the netty pipeline for external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-30512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-30512: -- Fix Version/s: 2.4.5 > Use a dedicated boss event group loop in the netty pipeline for

[jira] [Assigned] (SPARK-30512) Use a dedicated boss event group loop in the netty pipeline for external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-30512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-30512: - Assignee: Chandni Singh > Use a dedicated boss event group loop in the netty pipeline

[jira] [Resolved] (SPARK-30512) Use a dedicated boss event group loop in the netty pipeline for external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-30512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-30512. --- Fix Version/s: 3.0.0 Resolution: Fixed this could be pulled back into branch-2.X as

[jira] [Commented] (SPARK-16452) basic INFORMATION_SCHEMA support

[ https://issues.apache.org/jira/browse/SPARK-16452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026184#comment-17026184 ] Aaron Steers commented on SPARK-16452: -- Hello, everyone. I would like to revitalize this thread. Is

[jira] [Commented] (SPARK-28556) Error should also be sent to QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-28556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026147#comment-17026147 ] Dongjoon Hyun commented on SPARK-28556: --- That sounds great! > Error should also be sent to

[jira] [Created] (SPARK-30670) Pipes for PySpark

Vincent created SPARK-30670: --- Summary: Pipes for PySpark Key: SPARK-30670 URL: https://issues.apache.org/jira/browse/SPARK-30670 Project: Spark Issue Type: New Feature Components: SQL

[jira] [Commented] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026134#comment-17026134 ] Tyson Condie commented on SPARK-30602: --  The design looks to bring in some good optimizations from

[jira] [Commented] (SPARK-28556) Error should also be sent to QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-28556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026098#comment-17026098 ] Shixiong Zhu commented on SPARK-28556: -- This also reminds me that we should also review all public

[jira] [Commented] (SPARK-30662) ALS/MLP extend HasBlockSize

[ https://issues.apache.org/jira/browse/SPARK-30662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026095#comment-17026095 ] Huaxin Gao commented on SPARK-30662: I will work on this. > ALS/MLP extend HasBlockSize >

[jira] [Commented] (SPARK-27733) Upgrade to Avro 1.9.x

[ https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026079#comment-17026079 ] Dongjoon Hyun commented on SPARK-27733: --- [~iemejia]. I'm wondering why you think like that? For

[jira] [Commented] (SPARK-28556) Error should also be sent to QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-28556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026068#comment-17026068 ] Dongjoon Hyun commented on SPARK-28556: --- Got it. Thanks for the confirmation, [~zsxwing]. > Error

[jira] [Commented] (SPARK-27733) Upgrade to Avro 1.9.x

[ https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026043#comment-17026043 ] Ismaël Mejía commented on SPARK-27733: -- Upgrade in the Spark side should be relatively

[jira] [Commented] (SPARK-26346) Upgrade parquet to 1.11.1

[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026041#comment-17026041 ] Ismaël Mejía commented on SPARK-26346: -- Since Parquet depends on Avro 1.9.1 shouldn't SPARK-27733

[jira] [Resolved] (SPARK-30582) Spark UI is not showing Aggregated Metrics by Executor in stage page

[ https://issues.apache.org/jira/browse/SPARK-30582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30582. -- Resolution: Fixed Issue resolved by pull request 27292

[jira] [Assigned] (SPARK-30582) Spark UI is not showing Aggregated Metrics by Executor in stage page

[ https://issues.apache.org/jira/browse/SPARK-30582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30582: Assignee: Saurabh Chawla > Spark UI is not showing Aggregated Metrics by Executor in

[jira] [Commented] (SPARK-30582) Spark UI is not showing Aggregated Metrics by Executor in stage page

[ https://issues.apache.org/jira/browse/SPARK-30582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025944#comment-17025944 ] Sean R. Owen commented on SPARK-30582: -- [~saurabhc100] it's OK now as it's resolved, but generally

[jira] [Updated] (SPARK-30582) Spark UI is not showing Aggregated Metrics by Executor in stage page

[ https://issues.apache.org/jira/browse/SPARK-30582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30582: - Priority: Minor (was: Major) > Spark UI is not showing Aggregated Metrics by Executor in stage

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025860#comment-17025860 ] nabacg commented on SPARK-12312: Sure, will do [~gsomogyi] > JDBC connection to Kerberos secured

[jira] [Comment Edited] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025254#comment-17025254 ] nabacg edited comment on SPARK-12312 at 1/29/20 1:13 PM: - My suggestion was for

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025822#comment-17025822 ] Gabor Somogyi commented on SPARK-12312: --- [~nabacg] thanks for sharin, pretty sure there are

[jira] [Commented] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025729#comment-17025729 ] Maxim Gekk commented on SPARK-30668: We can try to revert this 

[jira] [Commented] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025698#comment-17025698 ] Herman van Hövell commented on SPARK-30668: --- I don't think we should revert the proleptic

[jira] [Comment Edited] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025681#comment-17025681 ] Maxim Gekk edited comment on SPARK-30668 at 1/29/20 8:43 AM: - [~marmbrus]

[jira] [Commented] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025681#comment-17025681 ] Maxim Gekk commented on SPARK-30668: If [~marmbrus] develops something new, he could use correct

[jira] [Commented] (SPARK-30668) to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

[ https://issues.apache.org/jira/browse/SPARK-30668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025673#comment-17025673 ] Xiao Li commented on SPARK-30668: - [~hvanhovell] Making it configurable looks necessary. Today, Michael