[jira] [Created] (SPARK-27574) spark on kubernetes driver pod phase changed from running to pending and starts another container in pod

2019-04-25 Thread Will Zhang (JIRA)
Will Zhang created SPARK-27574: -- Summary: spark on kubernetes driver pod phase changed from running to pending and starts another container in pod Key: SPARK-27574 URL:

[jira] [Comment Edited] (SPARK-27562) Complete the verification mechanism for shuffle transmitted data

2019-04-25 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826602#comment-16826602 ] feiwang edited comment on SPARK-27562 at 4/26/19 2:57 AM: -- [~irashid] What is

[jira] [Commented] (SPARK-27562) Complete the verification mechanism for shuffle transmitted data

2019-04-25 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826602#comment-16826602 ] feiwang commented on SPARK-27562: - [~irashid] > Complete the verification mechanism for shuffle

[jira] [Commented] (SPARK-27529) Spark Streaming consumer dies with kafka.common.OffsetOutOfRangeException

2019-04-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826594#comment-16826594 ] Hyukjin Kwon commented on SPARK-27529: -- It's difficult to diagnose if it was a correct behaviour or

[jira] [Updated] (SPARK-27573) Skip partial aggregation when data is already partitioned (or collapse adjacent partial and final aggregates)

2019-04-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27573: --- Description: When an aggregation requires a shuffle, Spark SQL performs separate partial and final

[jira] [Resolved] (SPARK-23831) Add org.apache.derby to IsolatedClientLoader

2019-04-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23831. -- Resolution: Fixed Its fixed in HIVE-21639. Probably, it's fixed in Spark too. Let's reopen

[jira] [Updated] (SPARK-27573) Skip partial aggregation when data is already partitioned (or collapse adjacent partial and final aggregates)

2019-04-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27573: --- Summary: Skip partial aggregation when data is already partitioned (or collapse adjacent partial

[jira] [Updated] (SPARK-27573) Collapse adjacent physical aggregate operators when possible

2019-04-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27573: --- Description: When an aggregation requires a shuffle, Spark SQL performs separate partial and final

[jira] [Updated] (SPARK-27573) Collapse adjacent physical aggregate operators when possible

2019-04-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27573: --- Description: When an aggregation requires a shuffle, Spark SQL performs separate partial and final

[jira] [Updated] (SPARK-27573) Collapse adjacent physical aggregate operators when possible

2019-04-25 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27573: --- Summary: Collapse adjacent physical aggregate operators when possible (was: Collapse adjacent

[jira] [Created] (SPARK-27573) Collapse adjacent aggregate physical operators when possible

2019-04-25 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-27573: -- Summary: Collapse adjacent aggregate physical operators when possible Key: SPARK-27573 URL: https://issues.apache.org/jira/browse/SPARK-27573 Project: Spark

[jira] [Created] (SPARK-27572) IN predicate should be nullable

2019-04-25 Thread Mingcong Han (JIRA)
Mingcong Han created SPARK-27572: Summary: IN predicate should be nullable Key: SPARK-27572 URL: https://issues.apache.org/jira/browse/SPARK-27572 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-27439) Use analyzed plan when explaining Dataset

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27439: Assignee: (was: Apache Spark) > Use analyzed plan when explaining Dataset >

[jira] [Assigned] (SPARK-27439) Use analyzed plan when explaining Dataset

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27439: Assignee: Apache Spark > Use analyzed plan when explaining Dataset >

[jira] [Reopened] (SPARK-27439) Use analyzed plan when explaining Dataset

2019-04-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-27439: --- Assignee: (was: Liang-Chi Hsieh) > Use analyzed plan when explaining Dataset >

[jira] [Commented] (SPARK-27439) Use analyzed plan when explaining Dataset

2019-04-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826577#comment-16826577 ] Dongjoon Hyun commented on SPARK-27439: --- One possible followup will be pass logical plan and

[jira] [Commented] (SPARK-27439) Use analyzed plan when explaining Dataset

2019-04-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826575#comment-16826575 ] Dongjoon Hyun commented on SPARK-27439: --- My bad. I'll revert the patch. [~huonw], [~viirya],

[jira] [Commented] (SPARK-27439) Use analyzed plan when explaining Dataset

2019-04-25 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826551#comment-16826551 ] Liang-Chi Hsieh commented on SPARK-27439: - I will look into it. Thanks [~huonw] > Use analyzed

[jira] [Commented] (SPARK-27439) Use analyzed plan when explaining Dataset

2019-04-25 Thread Huon Wilson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826542#comment-16826542 ] Huon Wilson commented on SPARK-27439: - I think this (partially) broke {{df.explain(extended =

[jira] [Assigned] (SPARK-27571) Spark 3.0 build warnings: reflectiveCalls edition

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27571: Assignee: Apache Spark (was: Sean Owen) > Spark 3.0 build warnings: reflectiveCalls

[jira] [Assigned] (SPARK-27571) Spark 3.0 build warnings: reflectiveCalls edition

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27571: Assignee: Sean Owen (was: Apache Spark) > Spark 3.0 build warnings: reflectiveCalls

[jira] [Created] (SPARK-27571) Spark 3.0 build warnings: reflectiveCalls edition

2019-04-25 Thread Sean Owen (JIRA)
Sean Owen created SPARK-27571: - Summary: Spark 3.0 build warnings: reflectiveCalls edition Key: SPARK-27571 URL: https://issues.apache.org/jira/browse/SPARK-27571 Project: Spark Issue Type:

[jira] [Commented] (SPARK-26268) Decouple shuffle data from Spark deployment

2019-04-25 Thread Ben Sidhom (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826525#comment-16826525 ] Ben Sidhom commented on SPARK-26268: This is related to but different than SPARK-25299. The goal

[jira] [Assigned] (SPARK-26268) Decouple shuffle data from Spark deployment

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26268: Assignee: Apache Spark > Decouple shuffle data from Spark deployment >

[jira] [Assigned] (SPARK-26268) Decouple shuffle data from Spark deployment

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26268: Assignee: (was: Apache Spark) > Decouple shuffle data from Spark deployment >

[jira] [Commented] (SPARK-27434) memory leak in spark driver

2019-04-25 Thread Ryne Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826425#comment-16826425 ] Ryne Yang commented on SPARK-27434: --- submitted a PR for this fix. 

[jira] [Updated] (SPARK-27570) java.io.EOFException Reached the end of stream - Reading Parquet from Swift

2019-04-25 Thread Harry Hough (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harry Hough updated SPARK-27570: Description: I did see issue SPARK-25966 but it seems there are some differences as his problem

[jira] [Updated] (SPARK-27570) java.io.EOFException Reached the end of stream - Reading Parquet from Swift

2019-04-25 Thread Harry Hough (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harry Hough updated SPARK-27570: Description: I did see issue SPARK-25966 but it seems there are some differences as his problem

[jira] [Updated] (SPARK-27570) java.io.EOFException Reached the end of stream - Reading Parquet from Swift

2019-04-25 Thread Harry Hough (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harry Hough updated SPARK-27570: Description: I did see issue SPARK-25966 but it seems there are some differences as his problem

[jira] [Updated] (SPARK-27570) java.io.EOFException Reached the end of stream - Reading Parquet from Swift

2019-04-25 Thread Harry Hough (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harry Hough updated SPARK-27570: Description: I did see issue SPARK-25966 but it seems there are some differences as his problem

[jira] [Updated] (SPARK-27570) java.io.EOFException Reached the end of stream - Reading Parquet from Swift

2019-04-25 Thread Harry Hough (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harry Hough updated SPARK-27570: Description: I did see issue SPARK-25966 but it seems there are some differences as his problem

[jira] [Assigned] (SPARK-27434) memory leak in spark driver

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27434: Assignee: Apache Spark > memory leak in spark driver > --- > >

[jira] [Assigned] (SPARK-27434) memory leak in spark driver

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27434: Assignee: (was: Apache Spark) > memory leak in spark driver >

[jira] [Updated] (SPARK-27570) java.io.EOFException Reached the end of stream - Reading Parquet from Swift

2019-04-25 Thread Harry Hough (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harry Hough updated SPARK-27570: Description: I did see issue SPARK-25966 but it seems there are some differences as his problem

[jira] [Created] (SPARK-27570) java.io.EOFException Reached the end of stream - Reading Parquet from Swift

2019-04-25 Thread Harry Hough (JIRA)
Harry Hough created SPARK-27570: --- Summary: java.io.EOFException Reached the end of stream - Reading Parquet from Swift Key: SPARK-27570 URL: https://issues.apache.org/jira/browse/SPARK-27570 Project:

[jira] [Commented] (SPARK-19335) Spark should support doing an efficient DataFrame Upsert via JDBC

2019-04-25 Thread Danny Guinther (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826378#comment-16826378 ] Danny Guinther commented on SPARK-19335: Any update on this? Also, please forgive this dumb

[jira] [Created] (SPARK-27569) Pandas UDF prefetches Arrow batches in the queue while executing the current batch

2019-04-25 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-27569: - Summary: Pandas UDF prefetches Arrow batches in the queue while executing the current batch Key: SPARK-27569 URL: https://issues.apache.org/jira/browse/SPARK-27569

[jira] [Assigned] (SPARK-27518) Show statistics in Optimized Logical Plan in the "Details" On SparkSQL ui page when CBO is enabled

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27518: Assignee: Apache Spark > Show statistics in Optimized Logical Plan in the "Details" On

[jira] [Assigned] (SPARK-27556) Exclude com.zaxxer:HikariCP-java7 from hadoop-yarn-server-web-proxy

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27556: Assignee: (was: Apache Spark) > Exclude com.zaxxer:HikariCP-java7 from

[jira] [Assigned] (SPARK-27024) Executor interface for cluster managers to support GPU resources

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27024: Assignee: Thomas Graves (was: Apache Spark) > Executor interface for cluster managers

[jira] [Assigned] (SPARK-27521) move data source v2 API to catalyst module

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27521: Assignee: Apache Spark (was: Wenchen Fan) > move data source v2 API to catalyst module

[jira] [Assigned] (SPARK-27506) Function `from_avro` doesn't allow deserialization of data using other compatible schemas

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27506: Assignee: (was: Apache Spark) > Function `from_avro` doesn't allow deserialization

[jira] [Assigned] (SPARK-27506) Function `from_avro` doesn't allow deserialization of data using other compatible schemas

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27506: Assignee: Apache Spark > Function `from_avro` doesn't allow deserialization of data

[jira] [Assigned] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12312: Assignee: Apache Spark > JDBC connection to Kerberos secured databases fails on remote

[jira] [Assigned] (SPARK-27468) "Storage Level" in "RDD Storage Page" is not correct

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27468: Assignee: Apache Spark > "Storage Level" in "RDD Storage Page" is not correct >

[jira] [Assigned] (SPARK-27552) The configuration `hive.exec.stagingdir` is invalid on Windows OS

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27552: Assignee: Apache Spark > The configuration `hive.exec.stagingdir` is invalid on Windows

[jira] [Assigned] (SPARK-25719) Search functionality in datatables in stages page should search over formatted data rather than the raw data

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25719: Assignee: (was: Apache Spark) > Search functionality in datatables in stages page

[jira] [Assigned] (SPARK-27525) Exclude commons-httpclient when interacting with different versions of the HiveMetastoreClient

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27525: Assignee: Apache Spark > Exclude commons-httpclient when interacting with different

[jira] [Assigned] (SPARK-27402) Fix hadoop-3.2 test issue(except the hive-thriftserver module)

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27402: Assignee: (was: Apache Spark) > Fix hadoop-3.2 test issue(except the

[jira] [Assigned] (SPARK-27545) Uncache table needs to delete the temporary view created when the cache table is executed.

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27545: Assignee: Apache Spark > Uncache table needs to delete the temporary view created when

[jira] [Assigned] (SPARK-27330) ForeachWriter is not being closed once a batch is aborted

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27330: Assignee: (was: Apache Spark) > ForeachWriter is not being closed once a batch is

[jira] [Assigned] (SPARK-27482) Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27482: Assignee: Apache Spark > Show BroadcastHashJoinExec numOutputRows statistics info on

[jira] [Assigned] (SPARK-27552) The configuration `hive.exec.stagingdir` is invalid on Windows OS

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27552: Assignee: (was: Apache Spark) > The configuration `hive.exec.stagingdir` is invalid

[jira] [Assigned] (SPARK-27521) move data source v2 API to catalyst module

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27521: Assignee: Wenchen Fan (was: Apache Spark) > move data source v2 API to catalyst module

[jira] [Assigned] (SPARK-25719) Search functionality in datatables in stages page should search over formatted data rather than the raw data

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25719: Assignee: Apache Spark > Search functionality in datatables in stages page should search

[jira] [Assigned] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12312: Assignee: (was: Apache Spark) > JDBC connection to Kerberos secured databases fails

[jira] [Assigned] (SPARK-27518) Show statistics in Optimized Logical Plan in the "Details" On SparkSQL ui page when CBO is enabled

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27518: Assignee: (was: Apache Spark) > Show statistics in Optimized Logical Plan in the

[jira] [Assigned] (SPARK-27525) Exclude commons-httpclient when interacting with different versions of the HiveMetastoreClient

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27525: Assignee: (was: Apache Spark) > Exclude commons-httpclient when interacting with

[jira] [Assigned] (SPARK-27474) avoid retrying a task failed with CommitDeniedException many times

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27474: Assignee: Wenchen Fan (was: Apache Spark) > avoid retrying a task failed with

[jira] [Assigned] (SPARK-27510) Master fall into dead loop while launching executor failed in Worker

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27510: Assignee: (was: Apache Spark) > Master fall into dead loop while launching executor

[jira] [Assigned] (SPARK-27468) "Storage Level" in "RDD Storage Page" is not correct

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27468: Assignee: (was: Apache Spark) > "Storage Level" in "RDD Storage Page" is not correct

[jira] [Assigned] (SPARK-27556) Exclude com.zaxxer:HikariCP-java7 from hadoop-yarn-server-web-proxy

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27556: Assignee: Apache Spark > Exclude com.zaxxer:HikariCP-java7 from

[jira] [Assigned] (SPARK-27482) Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27482: Assignee: (was: Apache Spark) > Show BroadcastHashJoinExec numOutputRows statistics

[jira] [Assigned] (SPARK-27510) Master fall into dead loop while launching executor failed in Worker

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27510: Assignee: Apache Spark > Master fall into dead loop while launching executor failed in

[jira] [Assigned] (SPARK-27545) Uncache table needs to delete the temporary view created when the cache table is executed.

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27545: Assignee: (was: Apache Spark) > Uncache table needs to delete the temporary view

[jira] [Assigned] (SPARK-27474) avoid retrying a task failed with CommitDeniedException many times

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27474: Assignee: Apache Spark (was: Wenchen Fan) > avoid retrying a task failed with

[jira] [Assigned] (SPARK-27299) Design: Property graph construction, save/load, and query APIs

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27299: Assignee: Martin Junghanns (was: Apache Spark) > Design: Property graph construction,

[jira] [Assigned] (SPARK-27347) [MESOS] Fix supervised driver retry logic when agent crashes/restarts

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27347: Assignee: (was: Apache Spark) > [MESOS] Fix supervised driver retry logic when agent

[jira] [Assigned] (SPARK-27418) Migrate Parquet to File Data Source V2

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27418: Assignee: Apache Spark > Migrate Parquet to File Data Source V2 >

[jira] [Assigned] (SPARK-27330) ForeachWriter is not being closed once a batch is aborted

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27330: Assignee: Apache Spark > ForeachWriter is not being closed once a batch is aborted >

[jira] [Assigned] (SPARK-27457) modify bean encoder to support avro objects

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27457: Assignee: (was: Apache Spark) > modify bean encoder to support avro objects >

[jira] [Assigned] (SPARK-27354) Move incompatible code from the hive-thriftserver module to sql/hive-thriftserver/v1.2.1

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27354: Assignee: Apache Spark > Move incompatible code from the hive-thriftserver module to >

[jira] [Assigned] (SPARK-27388) expression encoder for avro objects

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27388: Assignee: (was: Apache Spark) > expression encoder for avro objects >

[jira] [Assigned] (SPARK-27348) HeartbeatReceiver doesn't remove lost executors from CoarseGrainedSchedulerBackend

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27348: Assignee: Apache Spark > HeartbeatReceiver doesn't remove lost executors from >

[jira] [Assigned] (SPARK-27441) Add read/write tests to Hive serde tables(include Parquet vectorized reader)

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27441: Assignee: Apache Spark > Add read/write tests to Hive serde tables(include Parquet

[jira] [Assigned] (SPARK-27388) expression encoder for avro objects

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27388: Assignee: Apache Spark > expression encoder for avro objects >

[jira] [Assigned] (SPARK-27366) Spark scheduler internal changes to support GPU scheduling

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27366: Assignee: Apache Spark (was: Xingbo Jiang) > Spark scheduler internal changes to

[jira] [Assigned] (SPARK-27343) Use ConfigEntry for hardcoded configs for spark-sql-kafka

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27343: Assignee: (was: Apache Spark) > Use ConfigEntry for hardcoded configs for

[jira] [Assigned] (SPARK-27355) make query execution more sensitive to epoch message late or lost

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27355: Assignee: (was: Apache Spark) > make query execution more sensitive to epoch message

[jira] [Assigned] (SPARK-27024) Executor interface for cluster managers to support GPU resources

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27024: Assignee: Apache Spark (was: Thomas Graves) > Executor interface for cluster managers

[jira] [Assigned] (SPARK-26356) Remove SaveMode from data source v2 API

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26356: Assignee: (was: Apache Spark) > Remove SaveMode from data source v2 API >

[jira] [Assigned] (SPARK-27413) Keep the same epoch pace between driver and executor.

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27413: Assignee: Apache Spark > Keep the same epoch pace between driver and executor. >

[jira] [Assigned] (SPARK-27457) modify bean encoder to support avro objects

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27457: Assignee: Apache Spark > modify bean encoder to support avro objects >

[jira] [Assigned] (SPARK-27319) Filter out dir based on PathFilter before listing them

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27319: Assignee: (was: Apache Spark) > Filter out dir based on PathFilter before listing

[jira] [Assigned] (SPARK-27425) Add count_if functions

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27425: Assignee: Apache Spark > Add count_if functions > -- > >

[jira] [Assigned] (SPARK-27299) Design: Property graph construction, save/load, and query APIs

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27299: Assignee: Apache Spark (was: Martin Junghanns) > Design: Property graph construction,

[jira] [Assigned] (SPARK-27366) Spark scheduler internal changes to support GPU scheduling

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27366: Assignee: Xingbo Jiang (was: Apache Spark) > Spark scheduler internal changes to

[jira] [Assigned] (SPARK-27402) Fix hadoop-3.2 test issue(except the hive-thriftserver module)

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27402: Assignee: Apache Spark > Fix hadoop-3.2 test issue(except the hive-thriftserver module)

[jira] [Assigned] (SPARK-27343) Use ConfigEntry for hardcoded configs for spark-sql-kafka

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27343: Assignee: Apache Spark > Use ConfigEntry for hardcoded configs for spark-sql-kafka >

[jira] [Assigned] (SPARK-27354) Move incompatible code from the hive-thriftserver module to sql/hive-thriftserver/v1.2.1

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27354: Assignee: (was: Apache Spark) > Move incompatible code from the hive-thriftserver

[jira] [Assigned] (SPARK-27355) make query execution more sensitive to epoch message late or lost

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27355: Assignee: Apache Spark > make query execution more sensitive to epoch message late or

[jira] [Assigned] (SPARK-27348) HeartbeatReceiver doesn't remove lost executors from CoarseGrainedSchedulerBackend

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27348: Assignee: (was: Apache Spark) > HeartbeatReceiver doesn't remove lost executors from

[jira] [Assigned] (SPARK-27413) Keep the same epoch pace between driver and executor.

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27413: Assignee: (was: Apache Spark) > Keep the same epoch pace between driver and

[jira] [Assigned] (SPARK-27425) Add count_if functions

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27425: Assignee: (was: Apache Spark) > Add count_if functions > -- > >

[jira] [Assigned] (SPARK-27294) Multi-cluster Kafka delegation token support

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27294: Assignee: Apache Spark > Multi-cluster Kafka delegation token support >

[jira] [Assigned] (SPARK-27347) [MESOS] Fix supervised driver retry logic when agent crashes/restarts

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27347: Assignee: Apache Spark > [MESOS] Fix supervised driver retry logic when agent

[jira] [Assigned] (SPARK-27294) Multi-cluster Kafka delegation token support

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27294: Assignee: (was: Apache Spark) > Multi-cluster Kafka delegation token support >

[jira] [Assigned] (SPARK-27441) Add read/write tests to Hive serde tables(include Parquet vectorized reader)

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27441: Assignee: (was: Apache Spark) > Add read/write tests to Hive serde tables(include

[jira] [Assigned] (SPARK-27418) Migrate Parquet to File Data Source V2

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27418: Assignee: (was: Apache Spark) > Migrate Parquet to File Data Source V2 >

[jira] [Assigned] (SPARK-27214) Upgrading locality level when lots of pending tasks have been waiting more than locality.wait

2019-04-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27214: Assignee: (was: Apache Spark) > Upgrading locality level when lots of pending tasks

  1   2   3   >