[jira] [Created] (SPARK-23510) Support read data from Hive 2.2 and Hive 2.3 metastore

Yuming Wang created SPARK-23510: --- Summary: Support read data from Hive 2.2 and Hive 2.3 metastore Key: SPARK-23510 URL: https://issues.apache.org/jira/browse/SPARK-23510 Project: Spark Issue

[jira] [Assigned] (SPARK-23510) Support read data from Hive 2.2 and Hive 2.3 metastore

[ https://issues.apache.org/jira/browse/SPARK-23510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23510: Assignee: Apache Spark > Support read data from Hive 2.2 and Hive 2.3 metastore >

[jira] [Assigned] (SPARK-23510) Support read data from Hive 2.2 and Hive 2.3 metastore

[ https://issues.apache.org/jira/browse/SPARK-23510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23510: Assignee: (was: Apache Spark) > Support read data from Hive 2.2 and Hive 2.3

[jira] [Commented] (SPARK-23510) Support read data from Hive 2.2 and Hive 2.3 metastore

[ https://issues.apache.org/jira/browse/SPARK-23510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375636#comment-16375636 ] Apache Spark commented on SPARK-23510: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-23510) Support read data from Hive 2.2 and Hive 2.3 metastore

[ https://issues.apache.org/jira/browse/SPARK-23510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375660#comment-16375660 ] Yuming Wang commented on SPARK-23510: - [~JPMoresmau] Can you try 

[jira] [Commented] (SPARK-23458) OrcSuite flaky test

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375721#comment-16375721 ] Dongjoon Hyun commented on SPARK-23458: --- I added a link to `ParquetQuerySuite` because

[jira] [Updated] (SPARK-23458) Flaky test: OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23458: -- Summary: Flaky test: OrcQuerySuite (was: OrcSuite flaky test) > Flaky test: OrcQuerySuite >

[jira] [Comment Edited] (SPARK-23458) Flaky test: OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375721#comment-16375721 ] Dongjoon Hyun edited comment on SPARK-23458 at 2/24/18 6:37 PM: I updated

[jira] [Updated] (SPARK-23458) Flaky test: OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23458: -- Component/s: Tests > Flaky test: OrcQuerySuite > -- > >

[jira] [Comment Edited] (SPARK-23458) OrcSuite flaky test

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375721#comment-16375721 ] Dongjoon Hyun edited comment on SPARK-23458 at 2/24/18 6:34 PM: I added a

[jira] [Updated] (SPARK-23458) Flaky test: OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23458: -- Issue Type: Bug (was: Task) > Flaky test: OrcQuerySuite > -- > >

[jira] [Created] (SPARK-23508) blockManagerIdCache in BlockManagerId may cause oom

zhoukang created SPARK-23508: Summary: blockManagerIdCache in BlockManagerId may cause oom Key: SPARK-23508 URL: https://issues.apache.org/jira/browse/SPARK-23508 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23508) blockManagerIdCache in BlockManagerId may cause oom

[ https://issues.apache.org/jira/browse/SPARK-23508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-23508: - Description: blockManagerIdCache in BlockManagerId will not remove old values which may cause oom

[jira] [Updated] (SPARK-23508) blockManagerIdCache in BlockManagerId may cause oom

[ https://issues.apache.org/jira/browse/SPARK-23508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-23508: - Attachment: elephant-oom.png elepahnt-oom1.png > blockManagerIdCache in BlockManagerId

[jira] [Commented] (SPARK-23508) blockManagerIdCache in BlockManagerId may cause oom

[ https://issues.apache.org/jira/browse/SPARK-23508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375492#comment-16375492 ] Apache Spark commented on SPARK-23508: -- User 'caneGuy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23508) blockManagerIdCache in BlockManagerId may cause oom

[ https://issues.apache.org/jira/browse/SPARK-23508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23508: Assignee: Apache Spark > blockManagerIdCache in BlockManagerId may cause oom >

[jira] [Assigned] (SPARK-23508) blockManagerIdCache in BlockManagerId may cause oom

[ https://issues.apache.org/jira/browse/SPARK-23508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23508: Assignee: (was: Apache Spark) > blockManagerIdCache in BlockManagerId may cause oom >

[jira] [Commented] (SPARK-23448) Dataframe returns wrong result when column don't respect datatype

[ https://issues.apache.org/jira/browse/SPARK-23448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375439#comment-16375439 ] Liang-Chi Hsieh commented on SPARK-23448: - In fact this is exactly the JSON parser's behavior,

[jira] [Updated] (SPARK-23508) blockManagerIdCache in BlockManagerId may cause oom

[ https://issues.apache.org/jira/browse/SPARK-23508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-23508: - Description: blockManagerIdCache in BlockManagerId will not remove old values which may cause oom

[jira] [Created] (SPARK-23507) Migrate file-based data sources to data source v2

Gengliang Wang created SPARK-23507: -- Summary: Migrate file-based data sources to data source v2 Key: SPARK-23507 URL: https://issues.apache.org/jira/browse/SPARK-23507 Project: Spark Issue

[jira] [Created] (SPARK-23509) Upgrade commons-net from 2.2 to 3.1

PandaMonkey created SPARK-23509: --- Summary: Upgrade commons-net from 2.2 to 3.1 Key: SPARK-23509 URL: https://issues.apache.org/jira/browse/SPARK-23509 Project: Spark Issue Type: Dependency

[jira] [Updated] (SPARK-23509) Upgrade commons-net from 2.2 to 3.1

[ https://issues.apache.org/jira/browse/SPARK-23509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PandaMonkey updated SPARK-23509: Description: Hi, after analyzing spark-master\core\pom.xml, we found that Spark-core depends on

[jira] [Updated] (SPARK-23509) Upgrade commons-net from 2.2 to 3.1

[ https://issues.apache.org/jira/browse/SPARK-23509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PandaMonkey updated SPARK-23509: Attachment: spark.txt > Upgrade commons-net from 2.2 to 3.1 > ---

[jira] [Commented] (SPARK-20411) New features for expression.scalalang.typed

[ https://issues.apache.org/jira/browse/SPARK-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375559#comment-16375559 ] Diego Fanesi commented on SPARK-20411: -- In SPARK-20890 new default aggregators for Long and Double

[jira] [Commented] (SPARK-16996) Hive ACID delta files not seen

[ https://issues.apache.org/jira/browse/SPARK-16996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375758#comment-16375758 ] Frédéric ESCANDELL commented on SPARK-16996: On Hdp 2.6, i confirm that the steps described

[jira] [Comment Edited] (SPARK-16996) Hive ACID delta files not seen

[ https://issues.apache.org/jira/browse/SPARK-16996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375758#comment-16375758 ] Frédéric ESCANDELL edited comment on SPARK-16996 at 2/24/18 8:15 PM: -

[jira] [Comment Edited] (SPARK-16996) Hive ACID delta files not seen

[ https://issues.apache.org/jira/browse/SPARK-16996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375758#comment-16375758 ] Frédéric ESCANDELL edited comment on SPARK-16996 at 2/24/18 8:16 PM: -

[jira] [Created] (SPARK-23511) Catalyst: Implement GetField

Nadav Samet created SPARK-23511: --- Summary: Catalyst: Implement GetField Key: SPARK-23511 URL: https://issues.apache.org/jira/browse/SPARK-23511 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375859#comment-16375859 ] Apache Spark commented on SPARK-22839: -- User 'ifilonenko' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22839: Assignee: (was: Apache Spark) > Refactor Kubernetes code for configuring

[jira] [Assigned] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22839: Assignee: Apache Spark > Refactor Kubernetes code for configuring driver/executor pods to

[jira] [Created] (SPARK-23512) Complex operations on Dataframe corrupts data

Nazarii Bardiuk created SPARK-23512: --- Summary: Complex operations on Dataframe corrupts data Key: SPARK-23512 URL: https://issues.apache.org/jira/browse/SPARK-23512 Project: Spark Issue

[jira] [Assigned] (SPARK-23405) The task will hang up when a small table left semi join a big table

[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23405: Assignee: (was: Apache Spark) > The task will hang up when a small table left semi

[jira] [Commented] (SPARK-23405) The task will hang up when a small table left semi join a big table

[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375973#comment-16375973 ] Apache Spark commented on SPARK-23405: -- User 'KaiXinXiaoLei' has created a pull request for this

[jira] [Assigned] (SPARK-23405) The task will hang up when a small table left semi join a big table

[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23405: Assignee: Apache Spark > The task will hang up when a small table left semi join a big

[jira] [Updated] (SPARK-22324) Upgrade Arrow to version 0.8.0 and upgrade Netty to 4.1.17

[ https://issues.apache.org/jira/browse/SPARK-22324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-22324: Summary: Upgrade Arrow to version 0.8.0 and upgrade Netty to 4.1.17 (was: Upgrade Arrow to

[jira] [Updated] (SPARK-23207) Shuffle+Repartition on an DataFrame could lead to incorrect answers

[ https://issues.apache.org/jira/browse/SPARK-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-23207: Fix Version/s: (was: 2.4.0) > Shuffle+Repartition on an DataFrame could lead to incorrect

[jira] [Created] (SPARK-23514) Replace spark.sparkContext.hadoopConfiguration by spark.sessionState.newHadoopConf()

Xiao Li created SPARK-23514: --- Summary: Replace spark.sparkContext.hadoopConfiguration by spark.sessionState.newHadoopConf() Key: SPARK-23514 URL: https://issues.apache.org/jira/browse/SPARK-23514 Project:

[jira] [Updated] (SPARK-23405) The task will hang up when a small table left semi join a big table

[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-23405: -- Description: # I run a sql: `select ls.cs_order_number from ls left semi join catalog_sales

[jira] [Commented] (SPARK-23514) Replace spark.sparkContext.hadoopConfiguration by spark.sessionState.newHadoopConf()

[ https://issues.apache.org/jira/browse/SPARK-23514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375978#comment-16375978 ] Xiao Li commented on SPARK-23514: - cc [~dongjoon] Do you want to make a try? > Replace

[jira] [Created] (SPARK-23513) java.io.IOException: Expected 12 fields, but got 5 for row :Spark submit error

Rawia created SPARK-23513: -- Summary: java.io.IOException: Expected 12 fields, but got 5 for row :Spark submit error Key: SPARK-23513 URL: https://issues.apache.org/jira/browse/SPARK-23513 Project: Spark