[jira] [Commented] (SPARK-23812) DFS should be removed from unsupportedHiveNativeCommands in SqlBase.g4

[ https://issues.apache.org/jira/browse/SPARK-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418550#comment-16418550 ] Apache Spark commented on SPARK-23812: -- User 'wangtao605' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23812) DFS should be removed from unsupportedHiveNativeCommands in SqlBase.g4

[ https://issues.apache.org/jira/browse/SPARK-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23812: Assignee: Apache Spark > DFS should be removed from unsupportedHiveNativeCommands in

[jira] [Assigned] (SPARK-23812) DFS should be removed from unsupportedHiveNativeCommands in SqlBase.g4

[ https://issues.apache.org/jira/browse/SPARK-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23812: Assignee: (was: Apache Spark) > DFS should be removed from

[jira] [Commented] (SPARK-23812) DFS should be removed from unsupportedHiveNativeCommands in SqlBase.g4

[ https://issues.apache.org/jira/browse/SPARK-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418546#comment-16418546 ] wangtao93 commented on SPARK-23812: --- [~q79969786] yes, i'm working on this > DFS should be removed

[jira] [Commented] (SPARK-23784) Cannot use custom Aggregator with groupBy/agg

[ https://issues.apache.org/jira/browse/SPARK-23784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418558#comment-16418558 ] Liang-Chi Hsieh commented on SPARK-23784: - I think your question is already replied on

[jira] [Resolved] (SPARK-23784) Cannot use custom Aggregator with groupBy/agg

[ https://issues.apache.org/jira/browse/SPARK-23784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh resolved SPARK-23784. - Resolution: Not A Problem > Cannot use custom Aggregator with groupBy/agg >

[jira] [Assigned] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23785: -- Assignee: Sahil Takiar > LauncherBackend doesn't check state of connection before

[jira] [Resolved] (SPARK-23639) SparkSQL CLI fails talk to Kerberized metastore when use proxy user

[ https://issues.apache.org/jira/browse/SPARK-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23639. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by

[jira] [Assigned] (SPARK-23823) ResolveReferences loses correct origin

[ https://issues.apache.org/jira/browse/SPARK-23823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23823: Assignee: Apache Spark > ResolveReferences loses correct origin >

[jira] [Commented] (SPARK-23823) ResolveReferences loses correct origin

[ https://issues.apache.org/jira/browse/SPARK-23823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419482#comment-16419482 ] Apache Spark commented on SPARK-23823: -- User 'JiahuiJiang' has created a pull request for this

[jira] [Issue Comment Deleted] (SPARK-20297) Parquet Decimal(12,2) written by Spark is unreadable by Hive and Impala

[ https://issues.apache.org/jira/browse/SPARK-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Ivanfi updated SPARK-20297: -- Comment: was deleted (was: Could you please clarify how those DECIMALS were written in the

[jira] [Commented] (SPARK-20297) Parquet Decimal(12,2) written by Spark is unreadable by Hive and Impala

[ https://issues.apache.org/jira/browse/SPARK-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419335#comment-16419335 ] Zoltan Ivanfi commented on SPARK-20297: --- Could you please clarify how those DECIMALS were written

[jira] [Assigned] (SPARK-23823) ResolveReferences loses correct origin

[ https://issues.apache.org/jira/browse/SPARK-23823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23823: Assignee: (was: Apache Spark) > ResolveReferences loses correct origin >

[jira] [Assigned] (SPARK-23639) SparkSQL CLI fails talk to Kerberized metastore when use proxy user

[ https://issues.apache.org/jira/browse/SPARK-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23639: -- Assignee: Kent Yao > SparkSQL CLI fails talk to Kerberized metastore when use proxy

[jira] [Created] (SPARK-23821) Collection functions: flatten

Marek Novotny created SPARK-23821: - Summary: Collection functions: flatten Key: SPARK-23821 URL: https://issues.apache.org/jira/browse/SPARK-23821 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-23822) Improve error message for Parquet schema mismatches

Yuchen Huo created SPARK-23822: -- Summary: Improve error message for Parquet schema mismatches Key: SPARK-23822 URL: https://issues.apache.org/jira/browse/SPARK-23822 Project: Spark Issue Type:

[jira] [Updated] (SPARK-23822) Improve error message for Parquet schema mismatches

[ https://issues.apache.org/jira/browse/SPARK-23822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuchen Huo updated SPARK-23822: --- Description: If a user attempts to read Parquet files with mismatched schemas and schema merging is

[jira] [Updated] (SPARK-20498) RandomForestRegressionModel should expose getMaxDepth in PySpark

[ https://issues.apache.org/jira/browse/SPARK-20498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20498: -- Target Version/s: (was: 2.4.0) > RandomForestRegressionModel should expose

[jira] [Commented] (SPARK-20498) RandomForestRegressionModel should expose getMaxDepth in PySpark

[ https://issues.apache.org/jira/browse/SPARK-20498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419397#comment-16419397 ] Joseph K. Bradley commented on SPARK-20498: --- I'll close this since Bryan's PR mostly solved

[jira] [Resolved] (SPARK-20498) RandomForestRegressionModel should expose getMaxDepth in PySpark

[ https://issues.apache.org/jira/browse/SPARK-20498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-20498. --- Resolution: Fixed Fix Version/s: 2.3.0 > RandomForestRegressionModel should

[jira] [Updated] (SPARK-23821) Collection function: flatten

[ https://issues.apache.org/jira/browse/SPARK-23821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Novotny updated SPARK-23821: -- Summary: Collection function: flatten (was: Collection functions: flatten) > Collection

[jira] [Assigned] (SPARK-23821) Collection functions: flatten

[ https://issues.apache.org/jira/browse/SPARK-23821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23821: Assignee: (was: Apache Spark) > Collection functions: flatten >

[jira] [Assigned] (SPARK-23821) Collection functions: flatten

[ https://issues.apache.org/jira/browse/SPARK-23821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23821: Assignee: Apache Spark > Collection functions: flatten > - >

[jira] [Created] (SPARK-23823) ResolveReferences loses correct origin

Jiahui Jiang created SPARK-23823: Summary: ResolveReferences loses correct origin Key: SPARK-23823 URL: https://issues.apache.org/jira/browse/SPARK-23823 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23821) Collection functions: flatten

[ https://issues.apache.org/jira/browse/SPARK-23821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419426#comment-16419426 ] Apache Spark commented on SPARK-23821: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23785. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by

[jira] [Commented] (SPARK-20169) Groupby Bug with Sparksql

[ https://issues.apache.org/jira/browse/SPARK-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419808#comment-16419808 ] Maryann Xue commented on SPARK-20169: - [~smilegator], I think this is also caused by SPARK-23368, so

[jira] [Updated] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Vogelbacher updated SPARK-23825: -- Description: We currently request {{spark.{driver,executor}.memory}} as memory from

[jira] [Created] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

David Vogelbacher created SPARK-23825: - Summary: [K8s] Spark pods should request memory + memoryOverhead as resources Key: SPARK-23825 URL: https://issues.apache.org/jira/browse/SPARK-23825

[jira] [Commented] (SPARK-20297) Parquet Decimal(12,2) written by Spark is unreadable by Hive and Impala

[ https://issues.apache.org/jira/browse/SPARK-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419338#comment-16419338 ] Zoltan Ivanfi commented on SPARK-20297: --- Sorry, commented to the wrong JIRA. > Parquet

[jira] [Issue Comment Deleted] (SPARK-20297) Parquet Decimal(12,2) written by Spark is unreadable by Hive and Impala

[ https://issues.apache.org/jira/browse/SPARK-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Ivanfi updated SPARK-20297: -- Comment: was deleted (was: Sorry, commented to the wrong JIRA.) > Parquet Decimal(12,2)

[jira] [Updated] (SPARK-23704) PySpark access of individual trees in random forest is slow

[ https://issues.apache.org/jira/browse/SPARK-23704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-23704: - Component/s: PySpark > PySpark access of individual trees in random forest is slow >

[jira] [Updated] (SPARK-20498) RandomForestRegressionModel should expose getMaxDepth in PySpark

[ https://issues.apache.org/jira/browse/SPARK-20498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20498: -- Shepherd: (was: Joseph K. Bradley) > RandomForestRegressionModel should expose

[jira] [Resolved] (SPARK-23333) SparkML VectorAssembler.transform slow when needing to invoke .first() on sorted DataFrame

[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] V Luong resolved SPARK-2. - Resolution: Won't Fix > SparkML VectorAssembler.transform slow when needing to invoke .first() on >

[jira] [Commented] (SPARK-23333) SparkML VectorAssembler.transform slow when needing to invoke .first() on sorted DataFrame

[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419745#comment-16419745 ] V Luong commented on SPARK-2: - [~bago.amirbekian] thank you, that is indeed a good solution available

[jira] [Commented] (SPARK-23784) Cannot use custom Aggregator with groupBy/agg

[ https://issues.apache.org/jira/browse/SPARK-23784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419764#comment-16419764 ] Joshua Howard commented on SPARK-23784: --- Correct. It had been answered, but I am just now closing

[jira] [Assigned] (SPARK-23429) Add executor memory metrics to heartbeat and expose in executors REST API

[ https://issues.apache.org/jira/browse/SPARK-23429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23429: Assignee: Apache Spark > Add executor memory metrics to heartbeat and expose in executors

[jira] [Assigned] (SPARK-23429) Add executor memory metrics to heartbeat and expose in executors REST API

[ https://issues.apache.org/jira/browse/SPARK-23429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23429: Assignee: (was: Apache Spark) > Add executor memory metrics to heartbeat and expose

[jira] [Commented] (SPARK-23429) Add executor memory metrics to heartbeat and expose in executors REST API

[ https://issues.apache.org/jira/browse/SPARK-23429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419861#comment-16419861 ] Apache Spark commented on SPARK-23429: -- User 'edwinalu' has created a pull request for this issue:

[jira] [Created] (SPARK-23826) TestHiveSparkSession should set default session

Jose Torres created SPARK-23826: --- Summary: TestHiveSparkSession should set default session Key: SPARK-23826 URL: https://issues.apache.org/jira/browse/SPARK-23826 Project: Spark Issue Type:

[jira] [Closed] (SPARK-23784) Cannot use custom Aggregator with groupBy/agg

[ https://issues.apache.org/jira/browse/SPARK-23784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua Howard closed SPARK-23784. - See SO link. > Cannot use custom Aggregator with groupBy/agg >

[jira] [Updated] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Vogelbacher updated SPARK-23825: -- Description: We currently request {{spark.[driver,executor].memory}} as memory from

[jira] [Commented] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419857#comment-16419857 ] David Vogelbacher commented on SPARK-23825: --- Will make a PR shortly, cc [~mcheah] > [K8s]

[jira] [Created] (SPARK-23824) Make inpurityStats publicly accessible in ml.tree.Node

Barry Becker created SPARK-23824: Summary: Make inpurityStats publicly accessible in ml.tree.Node Key: SPARK-23824 URL: https://issues.apache.org/jira/browse/SPARK-23824 Project: Spark Issue

[jira] [Updated] (SPARK-23816) FetchFailedException when killing speculative task

[ https://issues.apache.org/jira/browse/SPARK-23816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chen xiao updated SPARK-23816: -- Description: When spark trying to kill speculative tasks because of another attempt has already

[jira] [Updated] (SPARK-23816) FetchFailedException when killing speculative task

[ https://issues.apache.org/jira/browse/SPARK-23816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chen xiao updated SPARK-23816: -- Description: When spark trying to kill speculative tasks because of another attempt has already

[jira] [Updated] (SPARK-23816) FetchFailedException when killing speculative task

[ https://issues.apache.org/jira/browse/SPARK-23816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chen xiao updated SPARK-23816: -- Description: When spark trying to kill speculative tasks because of another attempt has already

[jira] [Created] (SPARK-23817) Migrate ORC file format read path to data source V2

Gengliang Wang created SPARK-23817: -- Summary: Migrate ORC file format read path to data source V2 Key: SPARK-23817 URL: https://issues.apache.org/jira/browse/SPARK-23817 Project: Spark

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: duibi1.zip) > SQL which has large ‘case when’ expressions may cause code

[jira] [Commented] (SPARK-23291) SparkR : substr : In SparkR dataframe , starting and ending position arguments in "substr" is giving wrong result when the position is greater than 1

[ https://issues.apache.org/jira/browse/SPARK-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418599#comment-16418599 ] Liang-Chi Hsieh commented on SPARK-23291: - Because it is related to behavior change, I'm hesitant

[jira] [Resolved] (SPARK-23806) Broadcast. unpersist can cause fatal exception when used with dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-23806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23806. - Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by pull

[jira] [Assigned] (SPARK-23806) Broadcast. unpersist can cause fatal exception when used with dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-23806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23806: --- Assignee: Thomas Graves > Broadcast. unpersist can cause fatal exception when used with

[jira] [Created] (SPARK-23816) FetchFailedException when killing speculative task

chen xiao created SPARK-23816: - Summary: FetchFailedException when killing speculative task Key: SPARK-23816 URL: https://issues.apache.org/jira/browse/SPARK-23816 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23816) FetchFailedException when killing speculative task

[ https://issues.apache.org/jira/browse/SPARK-23816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chen xiao updated SPARK-23816: -- Description: When spark trying to kill speculative tasks because of another attempt has already

[jira] [Assigned] (SPARK-23817) Migrate ORC file format read path to data source V2

[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23817: Assignee: (was: Apache Spark) > Migrate ORC file format read path to data source V2 >

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: zeppelin-site.xml ZeppelinConfiguration.java shiro.ini

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: !test.JPG! when there are large 'ca !test2.JPG! se when ' expressions in spark sql,the

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: GbdLdapRealm.java) > SQL which has large ‘case when’ expressions may cause

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: GetUserList.java) > SQL which has large ‘case when’ expressions may cause code

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: LoginRestApi.java) > SQL which has large ‘case when’ expressions may cause

[jira] [Assigned] (SPARK-23817) Migrate ORC file format read path to data source V2

[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23817: Assignee: Apache Spark > Migrate ORC file format read path to data source V2 >

[jira] [Commented] (SPARK-23817) Migrate ORC file format read path to data source V2

[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418641#comment-16418641 ] Apache Spark commented on SPARK-23817: -- User 'gengliangwang' has created a pull request for this

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: duibi2.zip) > SQL which has large ‘case when’ expressions may cause code

[jira] [Comment Edited] (SPARK-22342) refactor schedulerDriver registration

[ https://issues.apache.org/jira/browse/SPARK-22342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418617#comment-16418617 ] Stavros Kontopoulos edited comment on SPARK-22342 at 3/29/18 9:43 AM:

[jira] [Comment Edited] (SPARK-20384) supporting value classes over primitives in DataSets

[ https://issues.apache.org/jira/browse/SPARK-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418691#comment-16418691 ] Furcy Pin edited comment on SPARK-20384 at 3/29/18 10:10 AM: - +1 on this

[jira] [Commented] (SPARK-22342) refactor schedulerDriver registration

[ https://issues.apache.org/jira/browse/SPARK-22342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418617#comment-16418617 ] Stavros Kontopoulos commented on SPARK-22342: - [~susanxhuynh] I guess we need to create a bug

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: (was: !test.JPG! when there are large 'ca !test2.JPG! se when ' expressions in

[jira] [Commented] (SPARK-20384) supporting value classes over primitives in DataSets

[ https://issues.apache.org/jira/browse/SPARK-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418691#comment-16418691 ] Furcy Pin commented on SPARK-20384: --- +1 on this issue. I think the generic use case is that the

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: ZeppelinConfiguration.java) > SQL which has large ‘case when’ expressions may

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: SecurityUtils.java) > SQL which has large ‘case when’ expressions may cause

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: shiro.ini) > SQL which has large ‘case when’ expressions may cause code

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: SecurityRestApi.java) > SQL which has large ‘case when’ expressions may cause

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: zeppelin-site.xml) > SQL which has large ‘case when’ expressions may cause

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: duibi1.zip > SQL which has large ‘case when’ expressions may cause code generation

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: duibi2.zip > SQL which has large ‘case when’ expressions may cause code generation

[jira] [Commented] (SPARK-23818) an official UDF interface for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418754#comment-16418754 ] Apache Spark commented on SPARK-23818: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-23818) an official UDF interface for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23818: Assignee: (was: Apache Spark) > an official UDF interface for Spark SQL >

[jira] [Assigned] (SPARK-23819) InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats

[ https://issues.apache.org/jira/browse/SPARK-23819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23819: Assignee: (was: Apache Spark) > InMemoryTableScanExec prunes orderable complex types

[jira] [Commented] (SPARK-23819) InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats

[ https://issues.apache.org/jira/browse/SPARK-23819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418777#comment-16418777 ] Apache Spark commented on SPARK-23819: -- User 'pwoody' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23819) InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats

[ https://issues.apache.org/jira/browse/SPARK-23819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23819: Assignee: Apache Spark > InMemoryTableScanExec prunes orderable complex types due to out

[jira] [Assigned] (SPARK-23770) Expose repartitionByRange in SparkR

[ https://issues.apache.org/jira/browse/SPARK-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23770: Assignee: Hyukjin Kwon > Expose repartitionByRange in SparkR >

[jira] [Resolved] (SPARK-23770) Expose repartitionByRange in SparkR

[ https://issues.apache.org/jira/browse/SPARK-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23770. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20902

[jira] [Created] (SPARK-23818) an official UDF interface for Spark SQL

Wenchen Fan created SPARK-23818: --- Summary: an official UDF interface for Spark SQL Key: SPARK-23818 URL: https://issues.apache.org/jira/browse/SPARK-23818 Project: Spark Issue Type:

[jira] [Created] (SPARK-23819) InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats

Patrick Woody created SPARK-23819: - Summary: InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats Key: SPARK-23819 URL: https://issues.apache.org/jira/browse/SPARK-23819

[jira] [Updated] (SPARK-23811) FetchFailed comes before Success of same task will cause child stage never succeed

[ https://issues.apache.org/jira/browse/SPARK-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-23811: Summary: FetchFailed comes before Success of same task will cause child stage never succeed (was:

[jira] [Assigned] (SPARK-23818) an official UDF interface for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23818: Assignee: Apache Spark > an official UDF interface for Spark SQL >

[jira] [Resolved] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

[ https://issues.apache.org/jira/browse/SPARK-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-22711. -- Resolution: Workaround Closing this because it seems wordnet is not serializable with

[jira] [Updated] (SPARK-23822) Improve error message for Parquet schema mismatches

[ https://issues.apache.org/jira/browse/SPARK-23822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuchen Huo updated SPARK-23822: --- Component/s: (was: Input/Output) SQL > Improve error message for Parquet schema

[jira] [Assigned] (SPARK-23827) StreamingJoinExec should ensure that input data is partitioned into specific number of partitions

[ https://issues.apache.org/jira/browse/SPARK-23827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23827: Assignee: Apache Spark (was: Tathagata Das) > StreamingJoinExec should ensure that input

[jira] [Commented] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419950#comment-16419950 ] David Vogelbacher commented on SPARK-23825: --- addressed by

[jira] [Assigned] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23825: Assignee: (was: Apache Spark) > [K8s] Spark pods should request memory +

[jira] [Assigned] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23825: Assignee: Apache Spark > [K8s] Spark pods should request memory + memoryOverhead as

[jira] [Updated] (SPARK-23429) Add executor memory metrics to heartbeat and expose in executors REST API

[ https://issues.apache.org/jira/browse/SPARK-23429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edwina Lu updated SPARK-23429: -- Description: Add new executor level memory metrics ( jvmUsedMemory, onHeapExecutionMemory,

[jira] [Assigned] (SPARK-23827) StreamingJoinExec should ensure that input data is partitioned into specific number of partitions

[ https://issues.apache.org/jira/browse/SPARK-23827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23827: Assignee: Tathagata Das (was: Apache Spark) > StreamingJoinExec should ensure that input

[jira] [Commented] (SPARK-23827) StreamingJoinExec should ensure that input data is partitioned into specific number of partitions

[ https://issues.apache.org/jira/browse/SPARK-23827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419925#comment-16419925 ] Apache Spark commented on SPARK-23827: -- User 'tdas' has created a pull request for this issue:

[jira] [Commented] (SPARK-15009) PySpark CountVectorizerModel should be able to construct from vocabulary list

[ https://issues.apache.org/jira/browse/SPARK-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419939#comment-16419939 ] Apache Spark commented on SPARK-15009: -- User 'BryanCutler' has created a pull request for this

[jira] [Commented] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419995#comment-16419995 ] Apache Spark commented on SPARK-23825: -- User 'dvogelbacher' has created a pull request for this

[jira] [Created] (SPARK-23829) spark-sql-kafka source in spark 2.3 causes reading stream failure frequently

Norman Bai created SPARK-23829: -- Summary: spark-sql-kafka source in spark 2.3 causes reading stream failure frequently Key: SPARK-23829 URL: https://issues.apache.org/jira/browse/SPARK-23829 Project:

[jira] [Commented] (SPARK-23805) support vector-size validation and Inference

[ https://issues.apache.org/jira/browse/SPARK-23805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16420113#comment-16420113 ] zhengruifeng commented on SPARK-23805: -- [~sethah] [~josephkb] Are you interested in this? >

[jira] [Created] (SPARK-23827) StreamingJoinExec should ensure that input data is partitioned into specific number of partitions

Tathagata Das created SPARK-23827: - Summary: StreamingJoinExec should ensure that input data is partitioned into specific number of partitions Key: SPARK-23827 URL:

  1   2   >