[jira] [Updated] (SPARK-31112) Use multiple external catalogs to speed up metastore access

[ https://issues.apache.org/jira/browse/SPARK-31112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-31112: Summary: Use multiple external catalogs to speed up metastore access (was: Use multiple external

[jira] [Updated] (SPARK-31112) Use multiple external catalog to speed up metastore access

[ https://issues.apache.org/jira/browse/SPARK-31112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-31112: Summary: Use multiple external catalog to speed up metastore access (was: Use multiple extrenal

[jira] [Commented] (SPARK-31148) Switching inner join statement yield different result

[ https://issues.apache.org/jira/browse/SPARK-31148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059948#comment-17059948 ] L. C. Hsieh commented on SPARK-31148: - As I said above, I didn't find any Bloom Filter-Assisted

[jira] [Commented] (SPARK-31156) DataFrameStatFunctions API is not consistent with respect to Column type

[ https://issues.apache.org/jira/browse/SPARK-31156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059947#comment-17059947 ] Oleksii Kachaiev commented on SPARK-31156: -- Well, it makes harder to write high level code that

[jira] [Updated] (SPARK-31162) Provide Configuration Parameter to select/enforce the Hive Hash for Bucketing

[ https://issues.apache.org/jira/browse/SPARK-31162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31162: - Parent: SPARK-19256 Issue Type: Sub-task (was: New Feature) > Provide Configuration

[jira] [Commented] (SPARK-31148) Switching inner join statement yield different result

[ https://issues.apache.org/jira/browse/SPARK-31148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059928#comment-17059928 ] Tanapol Nearunchorn commented on SPARK-31148: - [~viirya] Please correct me if I'm wrong. In

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059927#comment-17059927 ] Felix Kizhakkel Jose commented on SPARK-17495: -- [~maropu] [~tejasp] I have created a Jira

[jira] [Commented] (SPARK-19256) Hive bucketing support

[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059925#comment-17059925 ] Felix Kizhakkel Jose commented on SPARK-19256: -- Expose a configuration parameter for Hive

[jira] [Created] (SPARK-31162) Provide Configuration Parameter to select/enforce the Hive Hash for Bucketing

Felix Kizhakkel Jose created SPARK-31162: Summary: Provide Configuration Parameter to select/enforce the Hive Hash for Bucketing Key: SPARK-31162 URL: https://issues.apache.org/jira/browse/SPARK-31162

[jira] [Commented] (SPARK-31140) Support Quick sample in RDD

[ https://issues.apache.org/jira/browse/SPARK-31140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059916#comment-17059916 ] deshanxiao commented on SPARK-31140: Sure you are right. I just suggest that if we could add a new

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059914#comment-17059914 ] Jungtaek Lim commented on SPARK-31136: -- For 2, I just initiated the discussion thread on dev@ list.

[jira] [Commented] (SPARK-31140) Support Quick sample in RDD

[ https://issues.apache.org/jira/browse/SPARK-31140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059912#comment-17059912 ] L. C. Hsieh commented on SPARK-31140: - How do we know these partitions have same number of elements?

[jira] [Created] (SPARK-31161) Refactor the on-click timeline action in streagming-page.js

Kousuke Saruta created SPARK-31161: -- Summary: Refactor the on-click timeline action in streagming-page.js Key: SPARK-31161 URL: https://issues.apache.org/jira/browse/SPARK-31161 Project: Spark

[jira] [Commented] (SPARK-31140) Support Quick sample in RDD

[ https://issues.apache.org/jira/browse/SPARK-31140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059908#comment-17059908 ] deshanxiao commented on SPARK-31140: [~viirya] Thanks for your comment! It mean that we can

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059895#comment-17059895 ] Takeshi Yamamuro commented on SPARK-17495: -- Yea, can you open a new jira for that in the

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059893#comment-17059893 ] Felix Kizhakkel Jose commented on SPARK-17495: -- @[~maropu] [~hyukjin.kwon] So we need a new

[jira] [Commented] (SPARK-31102) spark-sql fails to parse when contains comment

[ https://issues.apache.org/jira/browse/SPARK-31102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059891#comment-17059891 ] Takeshi Yamamuro commented on SPARK-31102: -- Looks nice , thanks for the work. > spark-sql

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059890#comment-17059890 ] Takeshi Yamamuro commented on SPARK-17495: -- Ah, I see. I've looked over the merged PRs and, in

[jira] [Comment Edited] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059890#comment-17059890 ] Takeshi Yamamuro edited comment on SPARK-17495 at 3/16/20, 2:00 AM:

[jira] [Commented] (SPARK-31123) Drop does not work after join with aliases

[ https://issues.apache.org/jira/browse/SPARK-31123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059889#comment-17059889 ] L. C. Hsieh commented on SPARK-31123: - I tested with current master branch. Looks it is resolved:

[jira] [Commented] (SPARK-31102) spark-sql fails to parse when contains comment

[ https://issues.apache.org/jira/browse/SPARK-31102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059888#comment-17059888 ] Javier Fuentes commented on SPARK-31102: [~yumwang] I have checked 2 things: # The

[jira] [Commented] (SPARK-31140) Support Quick sample in RDD

[ https://issues.apache.org/jira/browse/SPARK-31140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059887#comment-17059887 ] L. C. Hsieh commented on SPARK-31140: - What does it mean to "filter the raw partition"? > Support

[jira] [Updated] (SPARK-31146) Leverage the helper method for aliasing in all SQL expressions

[ https://issues.apache.org/jira/browse/SPARK-31146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31146: - Issue Type: Bug (was: Improvement) > Leverage the helper method for aliasing in all SQL

[jira] [Resolved] (SPARK-31157) IDEA开发连接远程Spark集群报错

[ https://issues.apache.org/jira/browse/SPARK-31157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31157. -- Resolution: Invalid Please write it in English, test it against higher versions such as 2.4.x

[jira] [Updated] (SPARK-31158) IDEA开发连接远程Spark集群报错

[ https://issues.apache.org/jira/browse/SPARK-31158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-31158: Priority: Major (was: Blocker) > IDEA开发连接远程Spark集群报错 > --- > >

[jira] [Comment Edited] (SPARK-31145) spark2.4.0 read hbase2.1.0 Error

[ https://issues.apache.org/jira/browse/SPARK-31145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059884#comment-17059884 ] L. C. Hsieh edited comment on SPARK-31145 at 3/16/20, 1:40 AM: --- Please

[jira] [Commented] (SPARK-31145) spark2.4.0 read hbase2.1.0 Error

[ https://issues.apache.org/jira/browse/SPARK-31145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059884#comment-17059884 ] L. C. Hsieh commented on SPARK-31145: - Please take a look SPARK-26432. It is known issue that Spark

[jira] [Resolved] (SPARK-31145) spark2.4.0 read hbase2.1.0 Error

[ https://issues.apache.org/jira/browse/SPARK-31145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-31145. - Resolution: Duplicate > spark2.4.0 read hbase2.1.0 Error > > >

[jira] [Updated] (SPARK-31145) spark2.4.0 read hbase2.1.0 Error

[ https://issues.apache.org/jira/browse/SPARK-31145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-31145: Priority: Major (was: Blocker) > spark2.4.0 read hbase2.1.0 Error >

[jira] [Commented] (SPARK-31148) Switching inner join statement yield different result

[ https://issues.apache.org/jira/browse/SPARK-31148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059882#comment-17059882 ] L. C. Hsieh commented on SPARK-31148: - Am I missing anything? Seems I cannot find any Bloom

[jira] [Comment Edited] (SPARK-31148) Switching inner join statement yield different result

[ https://issues.apache.org/jira/browse/SPARK-31148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059882#comment-17059882 ] L. C. Hsieh edited comment on SPARK-31148 at 3/16/20, 1:32 AM: --- Am I

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059880#comment-17059880 ] Hyukjin Kwon commented on SPARK-17495: -- +1 for [~maropu]'s action here. Seems there is no related

[jira] [Resolved] (SPARK-31068) IllegalArgumentException in BroadcastExchangeExec

[ https://issues.apache.org/jira/browse/SPARK-31068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31068. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27828

[jira] [Assigned] (SPARK-31068) IllegalArgumentException in BroadcastExchangeExec

[ https://issues.apache.org/jira/browse/SPARK-31068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-31068: Assignee: Lantao Jin > IllegalArgumentException in BroadcastExchangeExec >

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059847#comment-17059847 ] Felix Kizhakkel Jose commented on SPARK-17495: -- @[~maropu] Is there any configuration

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059846#comment-17059846 ] Felix Kizhakkel Jose commented on SPARK-17495: -- Oh, thats great. I will check. > Hive hash

[jira] [Resolved] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-17495. -- Fix Version/s: 2.2.0 Resolution: Fixed Resolved by 

[jira] [Updated] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-17495: - Labels: (was: bulk-closed) > Hive hash implementation > > >

[jira] [Updated] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-17495: - Affects Version/s: 2.2.0 > Hive hash implementation > > >

[jira] [Comment Edited] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059844#comment-17059844 ] Takeshi Yamamuro edited comment on SPARK-17495 at 3/16/20, 1:07 AM:

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059844#comment-17059844 ] Takeshi Yamamuro commented on SPARK-17495: -- I checked the commit logs; {code:java} $git

[jira] [Assigned] (SPARK-31153) Cleanup several failures in lint-python

[ https://issues.apache.org/jira/browse/SPARK-31153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31153: Assignee: Nicholas Chammas > Cleanup several failures in lint-python >

[jira] [Resolved] (SPARK-31153) Cleanup several failures in lint-python

[ https://issues.apache.org/jira/browse/SPARK-31153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31153. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27917

[jira] [Resolved] (SPARK-31160) Resolved attribute(s) missing from ...

[ https://issues.apache.org/jira/browse/SPARK-31160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-31160. - Resolution: Not A Problem > Resolved attribute(s) missing from ... >

[jira] [Commented] (SPARK-31160) Resolved attribute(s) missing from ...

[ https://issues.apache.org/jira/browse/SPARK-31160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059841#comment-17059841 ] L. C. Hsieh commented on SPARK-31160: - This is not a bug. It is because your pass old attribute

[jira] [Commented] (SPARK-31156) DataFrameStatFunctions API is not consistent with respect to Column type

[ https://issues.apache.org/jira/browse/SPARK-31156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059834#comment-17059834 ] L. C. Hsieh commented on SPARK-31156: - Can you provide more reasons other than consistency why the

[jira] [Commented] (SPARK-31156) DataFrameStatFunctions API is not consistent with respect to Column type

[ https://issues.apache.org/jira/browse/SPARK-31156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059828#comment-17059828 ] Oleksii Kachaiev commented on SPARK-31156: -- [~viirya] Yes, they all accept Strings. I propose

[jira] [Updated] (SPARK-30983) Support more than 5 typed column in typed Dataset.select API

[ https://issues.apache.org/jira/browse/SPARK-30983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-30983: Description: Because Dataset only provides overloading typed select API to at most 5 typed

[jira] [Commented] (SPARK-31074) Avro serializer should not fail when a nullable Spark field is written to a non-null Avro column

[ https://issues.apache.org/jira/browse/SPARK-31074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059827#comment-17059827 ] L. C. Hsieh commented on SPARK-31074: - I think it is the same issue. You could check SPARK-27838 and

[jira] [Commented] (SPARK-31156) DataFrameStatFunctions API is not consistent with respect to Column type

[ https://issues.apache.org/jira/browse/SPARK-31156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059824#comment-17059824 ] L. C. Hsieh commented on SPARK-31156: - I checked current DataFrameStatFunctions. Both bloomFilter

[jira] [Commented] (SPARK-31074) Avro serializer should not fail when a nullable Spark field is written to a non-null Avro column

[ https://issues.apache.org/jira/browse/SPARK-31074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059823#comment-17059823 ] Kyrill Alyoshin commented on SPARK-31074: - Hmm... If you can't reproduce it, then it is likely

[jira] [Resolved] (SPARK-31074) Avro serializer should not fail when a nullable Spark field is written to a non-null Avro column

[ https://issues.apache.org/jira/browse/SPARK-31074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-31074. - Resolution: Duplicate > Avro serializer should not fail when a nullable Spark field is written

[jira] [Commented] (SPARK-31074) Avro serializer should not fail when a nullable Spark field is written to a non-null Avro column

[ https://issues.apache.org/jira/browse/SPARK-31074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059820#comment-17059820 ] L. C. Hsieh commented on SPARK-31074: - I think this is already resolved by SPARK-27838. > Avro

[jira] [Updated] (SPARK-29280) DataFrameReader should support a compression option

[ https://issues.apache.org/jira/browse/SPARK-29280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-29280: - Affects Version/s: (was: 3.0.0) 3.1.0 > DataFrameReader

[jira] [Assigned] (SPARK-31120) Support enabling maven profiles for importing via sbt on Intellij IDEA.

[ https://issues.apache.org/jira/browse/SPARK-31120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-31120: Assignee: Prashant Sharma > Support enabling maven profiles for importing via sbt on

[jira] [Resolved] (SPARK-31120) Support enabling maven profiles for importing via sbt on Intellij IDEA.

[ https://issues.apache.org/jira/browse/SPARK-31120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31120. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27878

[jira] [Updated] (SPARK-31160) Resolved attribute(s) missing from ...

[ https://issues.apache.org/jira/browse/SPARK-31160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yves updated SPARK-31160: - Description: When a fillna is used on a column and then you try to apply a udf on the same column you get a

[jira] [Updated] (SPARK-31160) Resolved attribute(s) missing from ...

[ https://issues.apache.org/jira/browse/SPARK-31160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yves updated SPARK-31160: - Priority: Critical (was: Major) > Resolved attribute(s) missing from ... >

[jira] [Created] (SPARK-31160) Resolved attribute(s) missing from ...

Yves created SPARK-31160: Summary: Resolved attribute(s) missing from ... Key: SPARK-31160 URL: https://issues.apache.org/jira/browse/SPARK-31160 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059681#comment-17059681 ] Felix Kizhakkel Jose commented on SPARK-17495: -- Is this going to available in the coming

[jira] [Commented] (SPARK-31159) Incompatible Parquet dates/timestamps with Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-31159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059617#comment-17059617 ] Maxim Gekk commented on SPARK-31159: [~cloud_fan] FYI > Incompatible Parquet dates/timestamps with

[jira] [Created] (SPARK-31159) Incompatible Parquet dates/timestamps with Spark 2.4

Maxim Gekk created SPARK-31159: -- Summary: Incompatible Parquet dates/timestamps with Spark 2.4 Key: SPARK-31159 URL: https://issues.apache.org/jira/browse/SPARK-31159 Project: Spark Issue Type: