[jira] [Commented] (SPARK-24302) when using spark persist(),"KryoException:IndexOutOfBoundsException" happens

2018-05-21 Thread yijukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482330#comment-16482330 ] yijukang commented on SPARK-24302: -- [~hyukjin.kwon]  OK thanks > when using spark

[jira] [Created] (SPARK-24332) Fix places reading 'spark.network.timeout' as milliseconds

2018-05-21 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-24332: Summary: Fix places reading 'spark.network.timeout' as milliseconds Key: SPARK-24332 URL: https://issues.apache.org/jira/browse/SPARK-24332 Project: Spark

[jira] [Commented] (SPARK-24273) Failure while using .checkpoint method

2018-05-21 Thread Jami Malikzade (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482562#comment-16482562 ] Jami Malikzade commented on SPARK-24273: [~kiszk] , I have just noticed that I am getting this

[jira] [Assigned] (SPARK-24330) Refactor ExecuteWriteTask in FileFormatWriter with DataWriter(V2)

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24330: Assignee: (was: Apache Spark) > Refactor ExecuteWriteTask in FileFormatWriter with

[jira] [Resolved] (SPARK-23935) High-order function: map_entries(map<K, V>) → array<row<K,V>>

2018-05-21 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23935. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21236

[jira] [Assigned] (SPARK-24330) Refactor ExecuteWriteTask in FileFormatWriter with DataWriter(V2)

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24330: Assignee: Apache Spark > Refactor ExecuteWriteTask in FileFormatWriter with

[jira] [Commented] (SPARK-24330) Refactor ExecuteWriteTask in FileFormatWriter with DataWriter(V2)

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482539#comment-16482539 ] Apache Spark commented on SPARK-24330: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-24331) Add cardinality / arrays_overlap / array_repeat / map_entries

2018-05-21 Thread Marek Novotny (JIRA)
Marek Novotny created SPARK-24331: - Summary: Add cardinality / arrays_overlap / array_repeat / map_entries Key: SPARK-24331 URL: https://issues.apache.org/jira/browse/SPARK-24331 Project: Spark

[jira] [Updated] (SPARK-24330) Refactor ExecuteWriteTask in FileFormatWriter with DataWriter(V2)

2018-05-21 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-24330: --- Description: Refactor ExecuteWriteTask in FileFormatWriter to reduce common logic and

[jira] [Assigned] (SPARK-23935) High-order function: map_entries(map<K, V>) → array<row<K,V>>

2018-05-21 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-23935: - Assignee: Marek Novotny > High-order function: map_entries(map) → array>

[jira] [Commented] (SPARK-24331) Add cardinality / arrays_overlap / array_repeat / map_entries

2018-05-21 Thread Marek Novotny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482577#comment-16482577 ] Marek Novotny commented on SPARK-24331: --- I will work on this. Thanks! > Add cardinality /

[jira] [Assigned] (SPARK-24332) Fix places reading 'spark.network.timeout' as milliseconds

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24332: Assignee: Shixiong Zhu (was: Apache Spark) > Fix places reading 'spark.network.timeout'

[jira] [Commented] (SPARK-24332) Fix places reading 'spark.network.timeout' as milliseconds

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482698#comment-16482698 ] Apache Spark commented on SPARK-24332: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24332) Fix places reading 'spark.network.timeout' as milliseconds

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24332: Assignee: Apache Spark (was: Shixiong Zhu) > Fix places reading 'spark.network.timeout'

[jira] [Commented] (SPARK-6235) Address various 2G limits

2018-05-21 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482699#comment-16482699 ] Imran Rashid commented on SPARK-6235: - Linked a [design

[jira] [Commented] (SPARK-20389) Upgrade kryo to fix NegativeArraySizeException

2018-05-21 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482408#comment-16482408 ] Arseniy Tashoyan commented on SPARK-20389: -- I confirm: the issue is in place for Spark 2.2.1. I

[jira] [Created] (SPARK-24330) Refactor ExecuteWriteTask in FileFormatWriter with DataWriter(V2)

2018-05-21 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-24330: -- Summary: Refactor ExecuteWriteTask in FileFormatWriter with DataWriter(V2) Key: SPARK-24330 URL: https://issues.apache.org/jira/browse/SPARK-24330 Project: Spark

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

2018-05-21 Thread Truong Duc Kien (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482476#comment-16482476 ] Truong Duc Kien commented on SPARK-23309: - We're also having also a performance problem with

[jira] [Commented] (SPARK-19790) OutputCommitCoordinator should not allow another task to commit after an ExecutorFailure

2018-05-21 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482512#comment-16482512 ] Steve Loughran commented on SPARK-19790: Update on this, having spent lots of time working in the

[jira] [Commented] (SPARK-24273) Failure while using .checkpoint method

2018-05-21 Thread Jami Malikzade (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482533#comment-16482533 ] Jami Malikzade commented on SPARK-24273: [~kiszk]   I was able to reproduce in small piece of

[jira] [Updated] (SPARK-20389) Upgrade kryo to fix NegativeArraySizeException

2018-05-21 Thread Arseniy Tashoyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arseniy Tashoyan updated SPARK-20389: - Affects Version/s: 2.2.1 > Upgrade kryo to fix NegativeArraySizeException >

[jira] [Updated] (SPARK-24316) Spark sql queries stall for column width more 6k for parquet based table

2018-05-21 Thread Bimalendu Choudhary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bimalendu Choudhary updated SPARK-24316: Description: When we create a table from a data frame using spark sql with columns

[jira] [Commented] (SPARK-6235) Address various 2G limits

2018-05-21 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482901#comment-16482901 ] Imran Rashid commented on SPARK-6235: - [~tgraves] WAL -- write-ahead-log for receiver-based streaming.

[jira] [Resolved] (SPARK-7132) Add fit with validation set to spark.ml GBT

2018-05-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-7132. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21129

[jira] [Assigned] (SPARK-23416) Flaky test: KafkaSourceStressForDontFailOnDataLossSuite.stress test for failOnDataLoss=false

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23416: Assignee: Apache Spark > Flaky test: KafkaSourceStressForDontFailOnDataLossSuite.stress

[jira] [Commented] (SPARK-23416) Flaky test: KafkaSourceStressForDontFailOnDataLossSuite.stress test for failOnDataLoss=false

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482922#comment-16482922 ] Apache Spark commented on SPARK-23416: -- User 'jose-torres' has created a pull request for this

[jira] [Updated] (SPARK-24316) Spark sql queries stall for column width more 6k for parquet based table

2018-05-21 Thread Bimalendu Choudhary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bimalendu Choudhary updated SPARK-24316: Affects Version/s: 2.4.0 2.3.0 Description: When

[jira] [Resolved] (SPARK-24234) create the bottom-of-task RDD with row buffer

2018-05-21 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-24234. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21337

[jira] [Updated] (SPARK-24316) Spark sql queries stall for column width more 6k for parquet based table

2018-05-21 Thread Bimalendu Choudhary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bimalendu Choudhary updated SPARK-24316: Description: When we create a table from a data frame using spark sql with columns

[jira] [Commented] (SPARK-6235) Address various 2G limits

2018-05-21 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482746#comment-16482746 ] Thomas Graves commented on SPARK-6235: -- >> Still unsupported: * large task results * large blocks

[jira] [Created] (SPARK-24333) Add fit with validation set to spark.ml GBT: Python API

2018-05-21 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-24333: - Summary: Add fit with validation set to spark.ml GBT: Python API Key: SPARK-24333 URL: https://issues.apache.org/jira/browse/SPARK-24333 Project: Spark

[jira] [Updated] (SPARK-24316) Spark sql queries stall for column width more 6k for parquet based table

2018-05-21 Thread Bimalendu Choudhary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bimalendu Choudhary updated SPARK-24316: Description: When we create a table from a data frame using spark sql with columns

[jira] [Assigned] (SPARK-23754) StopIterator exception in Python UDF results in partial result

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23754: Assignee: (was: Apache Spark) > StopIterator exception in Python UDF results in

[jira] [Commented] (SPARK-23754) StopIterator exception in Python UDF results in partial result

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482884#comment-16482884 ] Apache Spark commented on SPARK-23754: -- User 'e-dorigatti' has created a pull request for this

[jira] [Assigned] (SPARK-23754) StopIterator exception in Python UDF results in partial result

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23754: Assignee: Apache Spark > StopIterator exception in Python UDF results in partial result >

[jira] [Assigned] (SPARK-23416) Flaky test: KafkaSourceStressForDontFailOnDataLossSuite.stress test for failOnDataLoss=false

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23416: Assignee: (was: Apache Spark) > Flaky test:

[jira] [Assigned] (SPARK-24234) create the bottom-of-task RDD with row buffer

2018-05-21 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-24234: - Assignee: Jose Torres > create the bottom-of-task RDD with row buffer >

[jira] [Commented] (SPARK-24234) create the bottom-of-task RDD with row buffer

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482998#comment-16482998 ] Apache Spark commented on SPARK-24234: -- User 'jose-torres' has created a pull request for this

[jira] [Commented] (SPARK-23928) High-order function: shuffle(x) → array

2018-05-21 Thread H Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483095#comment-16483095 ] H Lu commented on SPARK-23928: -- Dear watchers, Can you please help review and comment the PR? THANKS A

[jira] [Commented] (SPARK-24334) Race condition in ArrowPythonRunner causes unclean shutdown of Arrow memory allocator

2018-05-21 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483004#comment-16483004 ] Li Jin commented on SPARK-24334: I have done some investigation and will submit a PR soon. > Race

[jira] [Created] (SPARK-24334) Race condition in ArrowPythonRunner causes unclean shutdown of Arrow memory allocator

2018-05-21 Thread Li Jin (JIRA)
Li Jin created SPARK-24334: -- Summary: Race condition in ArrowPythonRunner causes unclean shutdown of Arrow memory allocator Key: SPARK-24334 URL: https://issues.apache.org/jira/browse/SPARK-24334 Project:

[jira] [Resolved] (SPARK-24325) Tests for Hadoop's LinesReader

2018-05-21 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24325. - Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 2.4.0 > Tests for Hadoop's

[jira] [Assigned] (SPARK-23928) High-order function: shuffle(x) → array

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23928: Assignee: Apache Spark > High-order function: shuffle(x) → array >

[jira] [Assigned] (SPARK-23928) High-order function: shuffle(x) → array

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23928: Assignee: (was: Apache Spark) > High-order function: shuffle(x) → array >

[jira] [Commented] (SPARK-23928) High-order function: shuffle(x) → array

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483086#comment-16483086 ] Apache Spark commented on SPARK-23928: -- User 'pkuwm' has created a pull request for this issue:

[jira] [Created] (SPARK-24335) Dataset.map schema not applied in some cases

2018-05-21 Thread Robert Reid (JIRA)
Robert Reid created SPARK-24335: --- Summary: Dataset.map schema not applied in some cases Key: SPARK-24335 URL: https://issues.apache.org/jira/browse/SPARK-24335 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-24309) AsyncEventQueue should handle an interrupt from a Listener

2018-05-21 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24309: -- Assignee: Imran Rashid > AsyncEventQueue should handle an interrupt from a Listener >

[jira] [Commented] (SPARK-24336) Support 'pass through' transformation in BasicOperators

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483209#comment-16483209 ] Apache Spark commented on SPARK-24336: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-24336) Support 'pass through' transformation in BasicOperators

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24336: Assignee: (was: Apache Spark) > Support 'pass through' transformation in

[jira] [Resolved] (SPARK-24309) AsyncEventQueue should handle an interrupt from a Listener

2018-05-21 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24309. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by

[jira] [Assigned] (SPARK-24336) Support 'pass through' transformation in BasicOperators

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24336: Assignee: Apache Spark > Support 'pass through' transformation in BasicOperators >

[jira] [Commented] (SPARK-22055) Port release scripts

2018-05-21 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483217#comment-16483217 ] Marcelo Vanzin commented on SPARK-22055: IMO it would be a better idea to take Felix's docker

[jira] [Created] (SPARK-24336) Support 'pass through' transformation in BasicOperators

2018-05-21 Thread Jungtaek Lim (JIRA)
Jungtaek Lim created SPARK-24336: Summary: Support 'pass through' transformation in BasicOperators Key: SPARK-24336 URL: https://issues.apache.org/jira/browse/SPARK-24336 Project: Spark

[jira] [Created] (SPARK-24337) Improve the error message for invalid SQL conf value

2018-05-21 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-24337: Summary: Improve the error message for invalid SQL conf value Key: SPARK-24337 URL: https://issues.apache.org/jira/browse/SPARK-24337 Project: Spark Issue

[jira] [Commented] (SPARK-24204) Verify a write schema in Json/Orc/ParquetFileFormat

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483468#comment-16483468 ] Apache Spark commented on SPARK-24204: -- User 'maropu' has created a pull request for this issue:

[jira] [Updated] (SPARK-24313) Collection functions interpreted execution doesn't work with complex types

2018-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-24313: Priority: Critical (was: Minor) > Collection functions interpreted execution doesn't work with

[jira] [Resolved] (SPARK-24292) Proxy user cannot connect to HiveMetastore in local mode

2018-05-21 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin resolved SPARK-24292. Resolution: Duplicate > Proxy user cannot connect to HiveMetastore in local mode >

[jira] [Updated] (SPARK-24313) Collection functions interpreted execution doesn't work with complex types

2018-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-24313: Labels: correctness (was: ) > Collection functions interpreted execution doesn't work with

[jira] [Updated] (SPARK-24339) spark sql can not prune column in transform/map/reduce query

2018-05-21 Thread xdcjie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xdcjie updated SPARK-24339: --- Priority: Minor (was: Major) > spark sql can not prune column in transform/map/reduce query >

[jira] [Assigned] (SPARK-24204) Verify a write schema in Json/Orc/ParquetFileFormat

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24204: Assignee: (was: Apache Spark) > Verify a write schema in Json/Orc/ParquetFileFormat >

[jira] [Updated] (SPARK-24338) Spark SQL fails to create a Hive table when running in a Apache Sentry-secured Environment

2018-05-21 Thread Chaoran Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoran Yu updated SPARK-24338: --- Summary: Spark SQL fails to create a Hive table when running in a Apache Sentry-secured Environment

[jira] [Assigned] (SPARK-24204) Verify a write schema in Json/Orc/ParquetFileFormat

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24204: Assignee: Apache Spark > Verify a write schema in Json/Orc/ParquetFileFormat >

[jira] [Created] (SPARK-24339) spark sql can not prune column in transform/map/reduce query

2018-05-21 Thread xdcjie (JIRA)
xdcjie created SPARK-24339: -- Summary: spark sql can not prune column in transform/map/reduce query Key: SPARK-24339 URL: https://issues.apache.org/jira/browse/SPARK-24339 Project: Spark Issue

[jira] [Updated] (SPARK-24338) Spark SQL fails to create a table in Hive when running in a Apache Sentry-secured Environment

2018-05-21 Thread Chaoran Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoran Yu updated SPARK-24338: --- Attachment: exception.txt > Spark SQL fails to create a table in Hive when running in a Apache >

[jira] [Created] (SPARK-24338) Spark SQL fails to create a table in Hive when running in a Apache Sentry-secured Environment

2018-05-21 Thread Chaoran Yu (JIRA)
Chaoran Yu created SPARK-24338: -- Summary: Spark SQL fails to create a table in Hive when running in a Apache Sentry-secured Environment Key: SPARK-24338 URL: https://issues.apache.org/jira/browse/SPARK-24338

[jira] [Updated] (SPARK-24338) Spark SQL fails to create a table in Hive when running in a Apache Sentry-secured Environment

2018-05-21 Thread Chaoran Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoran Yu updated SPARK-24338: --- Description: This

[jira] [Resolved] (SPARK-24328) Fix scala.MatchError in literals.sql.out

2018-05-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24328. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.3.0 > Fix

[jira] [Assigned] (SPARK-24242) RangeExec should have correct outputOrdering

2018-05-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24242: Assignee: Liang-Chi Hsieh > RangeExec should have correct outputOrdering >

[jira] [Resolved] (SPARK-24242) RangeExec should have correct outputOrdering

2018-05-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24242. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21291

[jira] [Assigned] (SPARK-24323) Java lint errors

2018-05-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24323: Assignee: Kazuaki Ishizaki > Java lint errors > > > Key:

[jira] [Resolved] (SPARK-24323) Java lint errors

2018-05-21 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24323. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/21374 > Java lint errors >

[jira] [Commented] (SPARK-24283) Make standard scaler work without legacy MLlib

2018-05-21 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482277#comment-16482277 ] zhengruifeng commented on SPARK-24283: -- [~holdenk] do you mean that we should impl the prediction in

[jira] [Created] (SPARK-24329) Remove comments filtering before parsing of CSV files

2018-05-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-24329: -- Summary: Remove comments filtering before parsing of CSV files Key: SPARK-24329 URL: https://issues.apache.org/jira/browse/SPARK-24329 Project: Spark Issue

[jira] [Assigned] (SPARK-24329) Remove comments filtering before parsing of CSV files

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24329: Assignee: (was: Apache Spark) > Remove comments filtering before parsing of CSV files

[jira] [Commented] (SPARK-24329) Remove comments filtering before parsing of CSV files

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482295#comment-16482295 ] Apache Spark commented on SPARK-24329: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24329) Remove comments filtering before parsing of CSV files

2018-05-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24329: Assignee: Apache Spark > Remove comments filtering before parsing of CSV files >

[jira] [Resolved] (SPARK-24209) 0 configuration Knox gateway support in SHS

2018-05-21 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24209. Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.4.0 > 0