[jira] [Commented] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

2018-10-19 Thread Sunayan Saikia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657743#comment-16657743 ] Sunayan Saikia commented on SPARK-13478: [~vanzin]: My use case's a bit different. Imagine, I'm

[jira] [Assigned] (SPARK-25786) If the ByteBuffer.hasArray is false , it will throw UnsupportedOperationException for Kryo

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25786: Assignee: Apache Spark > If the ByteBuffer.hasArray is false , it will throw >

[jira] [Assigned] (SPARK-25786) If the ByteBuffer.hasArray is false , it will throw UnsupportedOperationException for Kryo

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25786: Assignee: (was: Apache Spark) > If the ByteBuffer.hasArray is false , it will throw

[jira] [Commented] (SPARK-25786) If the ByteBuffer.hasArray is false , it will throw UnsupportedOperationException for Kryo

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657689#comment-16657689 ] Apache Spark commented on SPARK-25786: -- User '10110346' has created a pull request for this issue:

[jira] [Commented] (SPARK-25786) If the ByteBuffer.hasArray is false , it will throw UnsupportedOperationException for Kryo

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657688#comment-16657688 ] Apache Spark commented on SPARK-25786: -- User '10110346' has created a pull request for this issue:

[jira] [Updated] (SPARK-25786) If the ByteBuffer.hasArray is false , it will throw UnsupportedOperationException for Kryo

2018-10-19 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-25786: Environment: (was: `{color:#ffc66d}deserialize{color}` for kryo,  the type of input parameter is

[jira] [Created] (SPARK-25786) If the ByteBuffer.hasArray is false , it will throw UnsupportedOperationException for Kryo

2018-10-19 Thread liuxian (JIRA)
liuxian created SPARK-25786: --- Summary: If the ByteBuffer.hasArray is false , it will throw UnsupportedOperationException for Kryo Key: SPARK-25786 URL: https://issues.apache.org/jira/browse/SPARK-25786

[jira] [Resolved] (SPARK-25785) Add prettyNames for from_json, to_json, from_csv, and schema_of_json

2018-10-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25785. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 3.0.0 Fixed in

[jira] [Commented] (SPARK-25785) Add prettyNames for from_json, to_json, from_csv, and schema_of_json

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657672#comment-16657672 ] Apache Spark commented on SPARK-25785: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-25785) Add prettyNames for from_json, to_json, from_csv, and schema_of_json

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657670#comment-16657670 ] Apache Spark commented on SPARK-25785: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-25785) Add prettyNames for from_json, to_json, from_csv, and schema_of_json

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25785: Assignee: Apache Spark > Add prettyNames for from_json, to_json, from_csv, and

[jira] [Assigned] (SPARK-25785) Add prettyNames for from_json, to_json, from_csv, and schema_of_json

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25785: Assignee: (was: Apache Spark) > Add prettyNames for from_json, to_json, from_csv,

[jira] [Created] (SPARK-25785) Add prettyNames for from_json, to_json, from_csv, and schema_of_json

2018-10-19 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-25785: Summary: Add prettyNames for from_json, to_json, from_csv, and schema_of_json Key: SPARK-25785 URL: https://issues.apache.org/jira/browse/SPARK-25785 Project: Spark

[jira] [Assigned] (SPARK-25784) Infer filters from constraints after rewriting predicate subquery

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25784: Assignee: (was: Apache Spark) > Infer filters from constraints after rewriting

[jira] [Assigned] (SPARK-25784) Infer filters from constraints after rewriting predicate subquery

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25784: Assignee: Apache Spark > Infer filters from constraints after rewriting predicate

[jira] [Commented] (SPARK-25784) Infer filters from constraints after rewriting predicate subquery

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657646#comment-16657646 ] Apache Spark commented on SPARK-25784: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-25784) Infer filters from constraints after rewriting predicate subquery

2018-10-19 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25784: --- Summary: Infer filters from constraints after rewriting predicate subquery Key: SPARK-25784 URL: https://issues.apache.org/jira/browse/SPARK-25784 Project: Spark

[jira] [Updated] (SPARK-25782) Add PCA Aggregator to support grouping

2018-10-19 Thread Matt Saunders (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Saunders updated SPARK-25782: -- Description: I built an Aggregator that computes PCA on grouped datasets. I wanted to use

[jira] [Issue Comment Deleted] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25783: Comment: was deleted (was: cc seems it's related to SPARK-24715 which [~viirya] and [~dongjoon] 

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657619#comment-16657619 ] DB Tsai commented on SPARK-25783: - cc seems it's related to SPARK-24715 which [~viirya] and [~dongjoon] 

[jira] [Resolved] (SPARK-22795) Raise error when line search in FirstOrderMinimizer failed

2018-10-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22795. --- Resolution: Won't Fix > Raise error when line search in FirstOrderMinimizer failed >

[jira] [Updated] (SPARK-25782) Add PCA Aggregator to support grouping

2018-10-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25782: -- Target Version/s: (was: 3.0.0) > Add PCA Aggregator to support grouping >

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657543#comment-16657543 ] Apache Spark commented on SPARK-25299: -- User 'ifilonenko' has created a pull request for this

[jira] [Assigned] (SPARK-25299) Use remote storage for persisting shuffle data

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25299: Assignee: Apache Spark > Use remote storage for persisting shuffle data >

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657539#comment-16657539 ] Apache Spark commented on SPARK-25299: -- User 'ifilonenko' has created a pull request for this

[jira] [Assigned] (SPARK-25299) Use remote storage for persisting shuffle data

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25299: Assignee: (was: Apache Spark) > Use remote storage for persisting shuffle data >

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657520#comment-16657520 ] Marcelo Vanzin commented on SPARK-25783: In fact, the dependency used to exist but was removed

[jira] [Comment Edited] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657455#comment-16657455 ] koert kuipers edited comment on SPARK-25783 at 10/19/18 10:25 PM: -- i

[jira] [Assigned] (SPARK-25745) docker-image-tool.sh ignores errors from Docker

2018-10-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25745: -- Assignee: Rob Vesse > docker-image-tool.sh ignores errors from Docker >

[jira] [Resolved] (SPARK-25745) docker-image-tool.sh ignores errors from Docker

2018-10-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25745. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22748

[jira] [Issue Comment Deleted] (SPARK-24610) wholeTextFiles broken for small files

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24610: -- Comment: was deleted (was: User '10110346' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-24610) wholeTextFiles broken for small files

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24610: -- Comment: was deleted (was: User '10110346' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657455#comment-16657455 ] koert kuipers edited comment on SPARK-25783 at 10/19/18 9:25 PM: - i was

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657455#comment-16657455 ] koert kuipers commented on SPARK-25783: --- i was just to same conclusion. i couldn't find jline

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657435#comment-16657435 ] Marcelo Vanzin commented on SPARK-25783: I think the problem that the "without hadoop" profiles

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657416#comment-16657416 ] Dongjoon Hyun commented on SPARK-25783: --- IIRC, in Spark 2.4, the minimum supported Hadoop version

[jira] [Commented] (SPARK-25782) Add PCA Aggregator to support grouping

2018-10-19 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657398#comment-16657398 ] Erik Erlandson commented on SPARK-25782: An ML Estimator also arguably would be a good API to

[jira] [Commented] (SPARK-25782) Add PCA Aggregator to support grouping

2018-10-19 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657394#comment-16657394 ] Erik Erlandson commented on SPARK-25782: Thanks [~mttsndrs]! I agree it makes sense to support

[jira] [Updated] (SPARK-25782) Add PCA Aggregator to support grouping

2018-10-19 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-25782: --- Target Version/s: 3.0.0 Component/s: ML Issue Type: New Feature (was:

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657368#comment-16657368 ] Xiao Li commented on SPARK-25783: - cc [~dbtsai] Could you take a look? > Spark shell fails because of

[jira] [Updated] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-25783: -- Environment: spark 2.4.0-rc3 on hadoop 2.6.0 (cdh 5.15.1) with -Phadoop-provided (was: spark

[jira] [Updated] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-25783: -- Environment: spark 2.4.0-rc3 on hadoop 2.6.0 (cdh 5.15.1) with (was: spark 2.4.0-rc3 which

[jira] [Updated] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-25783: -- Description: error i get when launching spark-shell is: {code:bash} Spark context Web UI

[jira] [Created] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread koert kuipers (JIRA)
koert kuipers created SPARK-25783: - Summary: Spark shell fails because of jline incompatibility Key: SPARK-25783 URL: https://issues.apache.org/jira/browse/SPARK-25783 Project: Spark Issue

[jira] [Commented] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657318#comment-16657318 ] Thomas Graves commented on SPARK-25692: --- [~redsanket] can you please take a look at this > Flaky

[jira] [Created] (SPARK-25782) Add PCA Aggregator to support grouping

2018-10-19 Thread Matt Saunders (JIRA)
Matt Saunders created SPARK-25782: - Summary: Add PCA Aggregator to support grouping Key: SPARK-25782 URL: https://issues.apache.org/jira/browse/SPARK-25782 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

2018-10-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657214#comment-16657214 ] Steve Loughran commented on SPARK-19143: thanks. when I get that S3a patch in spark will be

[jira] [Resolved] (SPARK-25704) Replication of > 2GB block fails due to bad config default

2018-10-19 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-25704. -- Resolution: Fixed Fix Version/s: 2.4.0 Resolved by pr

[jira] [Updated] (SPARK-25781) relative importance of linear regression

2018-10-19 Thread ruxi zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ruxi zhang updated SPARK-25781: --- Description: There is an R package relaimpo that generates relative importance for linear

[jira] [Updated] (SPARK-25781) relative importance of linear regression

2018-10-19 Thread ruxi zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ruxi zhang updated SPARK-25781: --- Attachment: v17i01.pdf > relative importance of linear regression >

[jira] [Created] (SPARK-25781) relative importance of linear regression

2018-10-19 Thread ruxi zhang (JIRA)
ruxi zhang created SPARK-25781: -- Summary: relative importance of linear regression Key: SPARK-25781 URL: https://issues.apache.org/jira/browse/SPARK-25781 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-21402) Fix java array of structs deserialization

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657147#comment-16657147 ] Dongjoon Hyun commented on SPARK-21402: --- Thank you for confirming, [~cloud_fan]! > Fix java array

[jira] [Comment Edited] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657058#comment-16657058 ] Dongjoon Hyun edited comment on SPARK-25413 at 10/19/18 4:42 PM: - Thank

[jira] [Resolved] (SPARK-25413) [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25413. --- Resolution: Invalid Thank you for reporting and making a PR, [~sandeep.katta2007].

[jira] [Updated] (SPARK-25759) StreamingListenerBus: Listener JavaStreamingListenerWrapper threw an exception

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25759: -- Priority: Major (was: Blocker) > StreamingListenerBus: Listener JavaStreamingListenerWrapper

[jira] [Commented] (SPARK-25759) StreamingListenerBus: Listener JavaStreamingListenerWrapper threw an exception

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657041#comment-16657041 ] Dongjoon Hyun commented on SPARK-25759: --- Thank you for reporting, [~sparkLearner]. Next time,

[jira] [Updated] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25692: -- Target Version/s: 3.0.0 > Flaky test: ChunkFetchIntegrationSuite >

[jira] [Commented] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

2018-10-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657029#comment-16657029 ] Marcelo Vanzin commented on SPARK-13478: If you have a principal and a keytab you don't need a

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

2018-10-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657028#comment-16657028 ] Marcelo Vanzin commented on SPARK-19143: I don't think there's been any activity. There was a PR

[jira] [Resolved] (SPARK-25269) SQL interface support specify StorageLevel when cache table

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25269. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22263

[jira] [Assigned] (SPARK-25269) SQL interface support specify StorageLevel when cache table

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25269: - Assignee: Yuming Wang > SQL interface support specify StorageLevel when cache table >

[jira] [Updated] (SPARK-25269) SQL interface support specify StorageLevel when cache table

2018-10-19 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25269: -- Description: {code:sql} CACHE TABLE tableName OPTIONS('storageLevel' 'DISK_ONLY'); {code}

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-19 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656861#comment-16656861 ] Marco Gaido commented on SPARK-25767: - I think it is a bug (thanks for reporting this): indeed I

[jira] [Assigned] (SPARK-25768) Constant argument expecting Hive UDAFs doesn't work

2018-10-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25768: --- Assignee: Peter Toth > Constant argument expecting Hive UDAFs doesn't work >

[jira] [Resolved] (SPARK-25768) Constant argument expecting Hive UDAFs doesn't work

2018-10-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25768. - Resolution: Fixed Fix Version/s: 2.4.0 2.3.3 > Constant argument

[jira] [Comment Edited] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-19 Thread Thomas Brugiere (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656779#comment-16656779 ] Thomas Brugiere edited comment on SPARK-25767 at 10/19/18 1:00 PM: --- I

[jira] [Commented] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-19 Thread Thomas Brugiere (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656779#comment-16656779 ] Thomas Brugiere commented on SPARK-25767: - I confirm that changing from *.toSeq()* to

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

2018-10-19 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656640#comment-16656640 ] Yuming Wang commented on SPARK-21725: - [~ste...@apache.org] Thanks a lot! > spark thriftserver

[jira] [Comment Edited] (SPARK-21725) spark thriftserver insert overwrite table partition select

2018-10-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656631#comment-16656631 ] Steve Loughran edited comment on SPARK-21725 at 10/19/18 10:48 AM: ---

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

2018-10-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656631#comment-16656631 ] Steve Loughran commented on SPARK-21725: bq. can we fix it on the Hadoop side? fix what? the

[jira] [Commented] (SPARK-25779) Remove SQL query tests for function documentation by DESCRIBE FUNCTION at SQLQueryTestSuite

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656522#comment-16656522 ] Apache Spark commented on SPARK-25779: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-25779) Remove SQL query tests for function documentation by DESCRIBE FUNCTION at SQLQueryTestSuite

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25779: Assignee: Apache Spark > Remove SQL query tests for function documentation by DESCRIBE

[jira] [Commented] (SPARK-25779) Remove SQL query tests for function documentation by DESCRIBE FUNCTION at SQLQueryTestSuite

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656520#comment-16656520 ] Apache Spark commented on SPARK-25779: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-24709) Inferring schema from JSON string literal

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656484#comment-16656484 ] Apache Spark commented on SPARK-24709: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

2018-10-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656476#comment-16656476 ] Steve Loughran commented on SPARK-19143: I'm looking at this as I add DTs into S3A

[jira] [Resolved] (SPARK-25560) Allow Function Injection in SparkSessionExtensions

2018-10-19 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-25560. --- Resolution: Fixed Assignee: Russell Spitzer Fix Version/s: 3.0.0 >

[jira] [Comment Edited] (SPARK-10816) EventTime based sessionization

2018-10-19 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656435#comment-16656435 ] Jungtaek Lim edited comment on SPARK-10816 at 10/19/18 8:24 AM:

[jira] [Commented] (SPARK-10816) EventTime based sessionization

2018-10-19 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656435#comment-16656435 ] Jungtaek Lim commented on SPARK-10816: -- [~msukmanowsky] In case if you would like to still

[jira] [Commented] (SPARK-25780) Scheduling the tasks which have no higher level locality first

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656355#comment-16656355 ] Apache Spark commented on SPARK-25780: -- User '10110346' has created a pull request for this issue:

[jira] [Commented] (SPARK-25780) Scheduling the tasks which have no higher level locality first

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656353#comment-16656353 ] Apache Spark commented on SPARK-25780: -- User '10110346' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25780) Scheduling the tasks which have no higher level locality first

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25780: Assignee: Apache Spark > Scheduling the tasks which have no higher level locality first

[jira] [Assigned] (SPARK-25780) Scheduling the tasks which have no higher level locality first

2018-10-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25780: Assignee: (was: Apache Spark) > Scheduling the tasks which have no higher level

[jira] [Created] (SPARK-25780) Scheduling the tasks which have no higher level locality first

2018-10-19 Thread liuxian (JIRA)
liuxian created SPARK-25780: --- Summary: Scheduling the tasks which have no higher level locality first Key: SPARK-25780 URL: https://issues.apache.org/jira/browse/SPARK-25780 Project: Spark Issue

[jira] [Commented] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

2018-10-19 Thread Sunayan Saikia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656320#comment-16656320 ] Sunayan Saikia commented on SPARK-13478: [~vanzin]: If I want to 'spark-submit' with l keytab

[jira] [Comment Edited] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

2018-10-19 Thread Sunayan Saikia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656320#comment-16656320 ] Sunayan Saikia edited comment on SPARK-13478 at 10/19/18 6:15 AM: --

[jira] [Created] (SPARK-25779) Remove SQL query tests for function documentation by DESCRIBE FUNCTION at SQLQueryTestSuite

2018-10-19 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-25779: Summary: Remove SQL query tests for function documentation by DESCRIBE FUNCTION at SQLQueryTestSuite Key: SPARK-25779 URL: https://issues.apache.org/jira/browse/SPARK-25779

[jira] [Commented] (SPARK-21402) Fix java array of structs deserialization

2018-10-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656314#comment-16656314 ] Wenchen Fan commented on SPARK-21402: - LGTM, thanks for doing it [~dongjoon] ! > Fix java array of