[jira] [Updated] (SPARK-27172) CRLF Injection/HTTP response splitting on spark embedded jetty servlet.

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27172: - Description: Can we upgrade embedded jetty servlet on spark 1.6.2? Will there be any

[jira] [Updated] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27167: - Description: Will there be a big impact on the system if current /static/jquery-1.11.1.min.js

[jira] [Updated] (SPARK-27156) why is the "http://:18080/static" browse able?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27156: - Issue Type: Bug (was: Question) > why is the "http://:18080/static" browse able? >

[jira] [Updated] (SPARK-27172) CRLF Injection/HTTP response splitting on spark embedded jetty servlet.

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27172: - Issue Type: Dependency upgrade (was: Question) > CRLF Injection/HTTP response splitting on

[jira] [Updated] (SPARK-27172) CRLF Injection/HTTP response splitting on spark embedded jetty servlet.

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27172: - Description: Can we upgrade embedded jetty servlet on spark 1.6.2? As per our vulnerability

[jira] [Updated] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27167: - Issue Type: Dependency upgrade (was: Question) > What would be the possible impact if I

[jira] [Updated] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27167: - Description: Will there be a big impact on my system if my current

[jira] [Updated] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27167: - Attachment: (was: Vulnerability Javascript library.xlsx) > What would be the possible

[jira] [Updated] (SPARK-27172) CRLF Injection/HTTP response splitting on spark embedded jetty servlet.

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27172: - Description: Can we upgrade embedded jetty servlet on spark 1.6.2? Is this possible or will

[jira] [Updated] (SPARK-27172) CRLF Injection/HTTP response splitting on spark embedded jetty servlet.

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27172: - Attachment: (was: CRLF injection - Sheet1.pdf) > CRLF Injection/HTTP response splitting on

[jira] [Updated] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27167: - Attachment: Vulnerability Javascript library.xlsx > What would be the possible impact if I

[jira] [Updated] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27167: - Description: Will there be a big impact on my system if my current

[jira] [Updated] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27167: - Description: Will there be a big impact on my system if my current

[jira] [Commented] (SPARK-22506) Spark thrift server can not impersonate user in kerberos

2019-03-14 Thread Wataru Yukawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793308#comment-16793308 ] Wataru Yukawa commented on SPARK-22506: --- Hi, Spark thrift server can impersonate a user in our 

[jira] [Created] (SPARK-27172) CRLF Injection/HTTP response splitting on spark embedded jetty servlet.

2019-03-14 Thread Jerry Garcia (JIRA)
Jerry Garcia created SPARK-27172: Summary: CRLF Injection/HTTP response splitting on spark embedded jetty servlet. Key: SPARK-27172 URL: https://issues.apache.org/jira/browse/SPARK-27172 Project:

[jira] [Updated] (SPARK-27172) CRLF Injection/HTTP response splitting on spark embedded jetty servlet.

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27172: - Attachment: CRLF injection - Sheet1.pdf > CRLF Injection/HTTP response splitting on spark

[jira] [Resolved] (SPARK-27132) Improve file source V2 framework

2019-03-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27132. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24066

[jira] [Assigned] (SPARK-27132) Improve file source V2 framework

2019-03-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27132: --- Assignee: Gengliang Wang > Improve file source V2 framework >

[jira] [Assigned] (SPARK-27136) Remove data source option check_files_exist

2019-03-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27136: --- Assignee: Gengliang Wang > Remove data source option check_files_exist >

[jira] [Updated] (SPARK-27172) CRLF Injection/HTTP response splitting on spark embedded jetty servlet.

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27172: - Description: Can we upgrade embedded jetty servlet on spark 1.6.2? Is this possible or will

[jira] [Resolved] (SPARK-27166) Improve `printSchema` to print up to the given level

2019-03-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27166. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0 This is

[jira] [Updated] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27167: - Description: Will there be a big impact on my system if my current

[jira] [Resolved] (SPARK-27136) Remove data source option check_files_exist

2019-03-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27136. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24069

[jira] [Updated] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Garcia updated SPARK-27167: - Description: Will there be a big impact on my system if my current

[jira] [Resolved] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27107. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0

[jira] [Resolved] (SPARK-27165) Upgrade Apache ORC to 1.5.5

2019-03-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27165. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0

[jira] [Created] (SPARK-27171) Support Full-Partiton limit in the first scan

2019-03-14 Thread deshanxiao (JIRA)
deshanxiao created SPARK-27171: -- Summary: Support Full-Partiton limit in the first scan Key: SPARK-27171 URL: https://issues.apache.org/jira/browse/SPARK-27171 Project: Spark Issue Type:

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

2019-03-14 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793268#comment-16793268 ] Gengliang Wang commented on SPARK-27142: +1 on the proposal. > Provide REST API for SQL level

[jira] [Created] (SPARK-27170) Better error message for syntax error with extraneous comma in the SQL parser

2019-03-14 Thread Wataru Yukawa (JIRA)
Wataru Yukawa created SPARK-27170: - Summary: Better error message for syntax error with extraneous comma in the SQL parser Key: SPARK-27170 URL: https://issues.apache.org/jira/browse/SPARK-27170

[jira] [Commented] (SPARK-26778) Implement file source V2 partitioning pruning

2019-03-14 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793263#comment-16793263 ] Gengliang Wang commented on SPARK-26778: Sorry, I meant file source partition pruning. I have

[jira] [Updated] (SPARK-26778) Implement file source V2 partitioning pruning

2019-03-14 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-26778: --- Summary: Implement file source V2 partitioning pruning (was: Implement file source V2

[jira] [Resolved] (SPARK-26343) Speed up running the kubernetes integration tests locally

2019-03-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26343. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23380

[jira] [Updated] (SPARK-27169) number of active tasks is negative on executors page

2019-03-14 Thread acupple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] acupple updated SPARK-27169: Attachment: QQ20190315-102235.png > number of active tasks is negative on executors page >

[jira] [Updated] (SPARK-27169) number of active tasks is negative on executors page

2019-03-14 Thread acupple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] acupple updated SPARK-27169: Description: I use spark to process some data in HDFS and HBASE, I use one thread consume message from a

[jira] [Updated] (SPARK-27169) number of active tasks is negative on executors page

2019-03-14 Thread acupple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] acupple updated SPARK-27169: Attachment: QQ20190315-102215.png > number of active tasks is negative on executors page >

[jira] [Updated] (SPARK-27169) number of active tasks is negative on executors page

2019-03-14 Thread acupple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] acupple updated SPARK-27169: Description: I use spark to process some data in hdfs and hbase, and the concurrency is 16.  but when

[jira] [Created] (SPARK-27169) number of active tasks is negative on executors page

2019-03-14 Thread acupple (JIRA)
acupple created SPARK-27169: --- Summary: number of active tasks is negative on executors page Key: SPARK-27169 URL: https://issues.apache.org/jira/browse/SPARK-27169 Project: Spark Issue Type: Bug

[jira] [Reopened] (SPARK-27141) Use ConfigEntry for hardcoded configs Yarn

2019-03-14 Thread wangjiaochun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangjiaochun reopened SPARK-27141: -- > Use ConfigEntry for hardcoded configs Yarn > -- > >

[jira] [Commented] (SPARK-27152) Column equality does not work for aliased columns.

2019-03-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793251#comment-16793251 ] Hyukjin Kwon commented on SPARK-27152: -- So, in which case is it important? > Column equality does

[jira] [Assigned] (SPARK-27164) RDD.countApprox on empty RDDs schedules jobs which never complete

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27164: Assignee: Apache Spark > RDD.countApprox on empty RDDs schedules jobs which never

[jira] [Assigned] (SPARK-27164) RDD.countApprox on empty RDDs schedules jobs which never complete

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27164: Assignee: (was: Apache Spark) > RDD.countApprox on empty RDDs schedules jobs which

[jira] [Assigned] (SPARK-27168) Add docker integration test for MsSql Server

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27168: Assignee: Apache Spark > Add docker integration test for MsSql Server >

[jira] [Assigned] (SPARK-27168) Add docker integration test for MsSql Server

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27168: Assignee: (was: Apache Spark) > Add docker integration test for MsSql Server >

[jira] [Commented] (SPARK-27168) Add docker integration test for MsSql Server

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793234#comment-16793234 ] Apache Spark commented on SPARK-27168: -- User 'lipzhu' has created a pull request for this issue:

[jira] [Commented] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-03-14 Thread KaiXu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793233#comment-16793233 ] KaiXu commented on SPARK-27100: --- hi [~hyukjin.kwon], the workload I'm running is ALS from Hibench, the

[jira] [Updated] (SPARK-27168) Add docker integration test for MsSql Server

2019-03-14 Thread Zhu, Lipeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu, Lipeng updated SPARK-27168: Issue Type: Test (was: Bug) > Add docker integration test for MsSql Server >

[jira] [Created] (SPARK-27168) Add docker integration test for MsSql Server

2019-03-14 Thread Zhu, Lipeng (JIRA)
Zhu, Lipeng created SPARK-27168: --- Summary: Add docker integration test for MsSql Server Key: SPARK-27168 URL: https://issues.apache.org/jira/browse/SPARK-27168 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-27164) RDD.countApprox on empty RDDs schedules jobs which never complete

2019-03-14 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793222#comment-16793222 ] Ajith S commented on SPARK-27164: - i will be working on this > RDD.countApprox on empty RDDs schedules

[jira] [Assigned] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27070: - Assignee: Yuli Fiterman > DefaultPartitionCoalescer can lock up driver for hours >

[jira] [Resolved] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27070. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23986

[jira] [Updated] (SPARK-26176) Verify column name when creating table via `STORED AS`

2019-03-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26176: -- Issue Type: Improvement (was: Bug) > Verify column name when creating table via `STORED AS`

[jira] [Updated] (SPARK-26176) Verify column name when creating table via `STORED AS`

2019-03-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26176: -- Priority: Minor (was: Major) > Verify column name when creating table via `STORED AS` >

[jira] [Assigned] (SPARK-26990) Difference in handling of mixed-case partition column names after SPARK-26188

2019-03-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-26990: --- Assignee: Gengliang Wang > Difference in handling of mixed-case partition column names after

[jira] [Created] (SPARK-27167) What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ?

2019-03-14 Thread Jerry Garcia (JIRA)
Jerry Garcia created SPARK-27167: Summary: What would be the possible impact if I upgrade /static/jquery-1.11.1.min.js ? Key: SPARK-27167 URL: https://issues.apache.org/jira/browse/SPARK-27167

[jira] [Assigned] (SPARK-27166) Improve `printSchema` to print up to the given level

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27166: Assignee: (was: Apache Spark) > Improve `printSchema` to print up to the given level

[jira] [Resolved] (SPARK-27158) dev/mima and dev/scalastyle support dynamic profiles

2019-03-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27158. -- Resolution: Fixed Fix Version/s: 3.0.0 Fixed in

[jira] [Assigned] (SPARK-27158) dev/mima and dev/scalastyle support dynamic profiles

2019-03-14 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27158: Assignee: Yuming Wang > dev/mima and dev/scalastyle support dynamic profiles >

[jira] [Created] (SPARK-27166) Improve `printSchema` to print up to the given level

2019-03-14 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-27166: - Summary: Improve `printSchema` to print up to the given level Key: SPARK-27166 URL: https://issues.apache.org/jira/browse/SPARK-27166 Project: Spark Issue

[jira] [Assigned] (SPARK-27166) Improve `printSchema` to print up to the given level

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27166: Assignee: Apache Spark > Improve `printSchema` to print up to the given level >

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793118#comment-16793118 ] Apache Spark commented on SPARK-27107: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-27165) Upgrade Apache ORC to 1.5.5

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27165: Assignee: Apache Spark > Upgrade Apache ORC to 1.5.5 > --- > >

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793125#comment-16793125 ] Apache Spark commented on SPARK-27107: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-27165) Upgrade Apache ORC to 1.5.5

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27165: Assignee: (was: Apache Spark) > Upgrade Apache ORC to 1.5.5 >

[jira] [Updated] (SPARK-27165) Upgrade Apache ORC to 1.5.5

2019-03-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27165: -- Description: This issue aims to update Apache ORC dependency to fix SPARK-27107 . {code:java}

[jira] [Assigned] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27107: Assignee: Apache Spark > Spark SQL Job failing because of Kryo buffer overflow with ORC

[jira] [Assigned] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27107: Assignee: (was: Apache Spark) > Spark SQL Job failing because of Kryo buffer

[jira] [Updated] (SPARK-27165) Upgrade Apache ORC to 1.5.5

2019-03-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27165: -- Description: This issue aims to update Apache ORC dependency to fix  SPARK-27107 . (was: This

[jira] [Created] (SPARK-27165) Upgrade Apache ORC to 1.5.5

2019-03-14 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-27165: - Summary: Upgrade Apache ORC to 1.5.5 Key: SPARK-27165 URL: https://issues.apache.org/jira/browse/SPARK-27165 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-27107) Spark SQL Job failing because of Kryo buffer overflow with ORC

2019-03-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793110#comment-16793110 ] Dongjoon Hyun commented on SPARK-27107: --- The vote passed. I'm preparing the PRs. > Spark SQL Job

[jira] [Comment Edited] (SPARK-27098) Flaky missing file parts when writing to Ceph without error

2019-03-14 Thread Martin Loncaric (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793082#comment-16793082 ] Martin Loncaric edited comment on SPARK-27098 at 3/14/19 9:28 PM: --

[jira] [Commented] (SPARK-27098) Flaky missing file parts when writing to Ceph without error

2019-03-14 Thread Martin Loncaric (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793082#comment-16793082 ] Martin Loncaric commented on SPARK-27098: - [~ste...@apache.org] Does this make more sense to

[jira] [Commented] (SPARK-27098) Flaky missing file parts when writing to Ceph without error

2019-03-14 Thread Martin Loncaric (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793080#comment-16793080 ] Martin Loncaric commented on SPARK-27098: - I've gotten the debug logs for (1.), but can't make

[jira] [Updated] (SPARK-27098) Flaky missing file parts when writing to Ceph without error

2019-03-14 Thread Martin Loncaric (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Loncaric updated SPARK-27098: Attachment: sanitized_stdout_1.txt > Flaky missing file parts when writing to Ceph

[jira] [Updated] (SPARK-27164) RDD.countApprox on empty RDDs schedules jobs which never complete

2019-03-14 Thread Ryan Moore (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Moore updated SPARK-27164: --- Attachment: Screen Shot 2019-03-14 at 1.49.19 PM.png > RDD.countApprox on empty RDDs schedules jobs

[jira] [Created] (SPARK-27164) RDD.countApprox on empty RDDs schedules jobs which never complete

2019-03-14 Thread Ryan Moore (JIRA)
Ryan Moore created SPARK-27164: -- Summary: RDD.countApprox on empty RDDs schedules jobs which never complete Key: SPARK-27164 URL: https://issues.apache.org/jira/browse/SPARK-27164 Project: Spark

[jira] [Resolved] (SPARK-27145) Close store after test, in the SQLAppStatusListenerSuite

2019-03-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27145. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24079

[jira] [Assigned] (SPARK-27145) Close store after test, in the SQLAppStatusListenerSuite

2019-03-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27145: -- Assignee: shahid > Close store after test, in the SQLAppStatusListenerSuite >

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

2019-03-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792975#comment-16792975 ] Marcelo Vanzin commented on SPARK-27142: I'm not sure I understand your point, Sean. We expose

[jira] [Comment Edited] (SPARK-5997) Increase partition count without performing a shuffle

2019-03-14 Thread nirav patel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792964#comment-16792964 ] nirav patel edited comment on SPARK-5997 at 3/14/19 6:56 PM: - Adding another

[jira] [Updated] (SPARK-27006) SPIP: .NET bindings for Apache Spark

2019-03-14 Thread Terry Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-27006: -- Priority: Major (was: Minor) > SPIP: .NET bindings for Apache Spark >

[jira] [Commented] (SPARK-5997) Increase partition count without performing a shuffle

2019-03-14 Thread nirav patel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792964#comment-16792964 ] nirav patel commented on SPARK-5997: Adding another possible use case for this ask - I am hitting 

[jira] [Comment Edited] (SPARK-27006) SPIP: .NET bindings for Apache Spark

2019-03-14 Thread Tyson Condie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792939#comment-16792939 ] Tyson Condie edited comment on SPARK-27006 at 3/14/19 6:23 PM: --- I would

[jira] [Commented] (SPARK-27006) SPIP: .NET bindings for Apache Spark

2019-03-14 Thread Tyson Condie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792939#comment-16792939 ] Tyson Condie commented on SPARK-27006: -- I would like to briefly illuminate what I think this SPIP

[jira] [Assigned] (SPARK-27163) Cleanup and consolidate Pandas UDF functionality

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27163: Assignee: (was: Apache Spark) > Cleanup and consolidate Pandas UDF functionality >

[jira] [Assigned] (SPARK-27163) Cleanup and consolidate Pandas UDF functionality

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27163: Assignee: Apache Spark > Cleanup and consolidate Pandas UDF functionality >

[jira] [Updated] (SPARK-27163) Cleanup and consolidate Pandas UDF functionality

2019-03-14 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27163: - Priority: Minor (was: Major) > Cleanup and consolidate Pandas UDF functionality >

[jira] [Created] (SPARK-27163) Cleanup and consolidate Pandas UDF functionality

2019-03-14 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-27163: Summary: Cleanup and consolidate Pandas UDF functionality Key: SPARK-27163 URL: https://issues.apache.org/jira/browse/SPARK-27163 Project: Spark Issue Type:

[jira] [Commented] (SPARK-26778) Implement file source V2 partitioning

2019-03-14 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792843#comment-16792843 ] Ryan Blue commented on SPARK-26778: --- [~Gengliang.Wang], can you clarify what this issue is tracking?

[jira] [Updated] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-14 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-26742: --- Fix Version/s: 2.4.2 > Bump Kubernetes Client Version to 4.1.2 >

[jira] [Updated] (SPARK-27158) dev/mima and dev/scalastyle support dynamic profiles

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27158: Issue Type: Sub-task (was: Improvement) Parent: SPARK-23710 > dev/mima and

[jira] [Created] (SPARK-27162) Add new method getOriginalMap in CaseInsensitiveStringMap

2019-03-14 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-27162: -- Summary: Add new method getOriginalMap in CaseInsensitiveStringMap Key: SPARK-27162 URL: https://issues.apache.org/jira/browse/SPARK-27162 Project: Spark

[jira] [Updated] (SPARK-23710) Upgrade the built-in Hive to 2.3.4 for hadoop-3.1

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Issue Type: Umbrella (was: Improvement) > Upgrade the built-in Hive to 2.3.4 for hadoop-3.1 >

[jira] [Assigned] (SPARK-27162) Add new method getOriginalMap in CaseInsensitiveStringMap

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27162: Assignee: Apache Spark > Add new method getOriginalMap in CaseInsensitiveStringMap >

[jira] [Assigned] (SPARK-27162) Add new method getOriginalMap in CaseInsensitiveStringMap

2019-03-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27162: Assignee: (was: Apache Spark) > Add new method getOriginalMap in

[jira] [Updated] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27130: Issue Type: Sub-task (was: Improvement) Parent: SPARK-23710 > Automatically select

[jira] [Updated] (SPARK-27054) Remove Calcite dependency

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27054: Issue Type: Sub-task (was: Improvement) Parent: SPARK-23710 > Remove Calcite dependency

[jira] [Updated] (SPARK-23710) Upgrade the built-in Hive to 2.3.4 for hadoop-3.1

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: Spark fail to run on Hadoop 3.x, because Hive's ShimLoader considers Hadoop 3.x to

[jira] [Comment Edited] (SPARK-27152) Column equality does not work for aliased columns.

2019-03-14 Thread Ryan Radtke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792793#comment-16792793 ] Ryan Radtke edited comment on SPARK-27152 at 3/14/19 3:36 PM: -- If you are

[jira] [Commented] (SPARK-27152) Column equality does not work for aliased columns.

2019-03-14 Thread Ryan Radtke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792793#comment-16792793 ] Ryan Radtke commented on SPARK-27152: - If you are abstracting elt then it important.  Also, its just

[jira] [Comment Edited] (SPARK-27152) Column equality does not work for aliased columns.

2019-03-14 Thread Ryan Radtke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792793#comment-16792793 ] Ryan Radtke edited comment on SPARK-27152 at 3/14/19 3:34 PM: -- If you are

  1   2   >