[jira] [Created] (SPARK-23829) spark-sql-kafka source in spark 2.3 causes reading stream failure frequently

Norman Bai created SPARK-23829: -- Summary: spark-sql-kafka source in spark 2.3 causes reading stream failure frequently Key: SPARK-23829 URL: https://issues.apache.org/jira/browse/SPARK-23829 Project:

[jira] [Resolved] (SPARK-23808) Test spark sessions should set default session

[ https://issues.apache.org/jira/browse/SPARK-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23808. - Resolution: Fixed Assignee: Jose Torres > Test spark sessions should set default session >

[jira] [Updated] (SPARK-23808) Test spark sessions should set default session

[ https://issues.apache.org/jira/browse/SPARK-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23808: Fix Version/s: 2.4.0 2.3.1 > Test spark sessions should set default session >

[jira] [Commented] (SPARK-23805) support vector-size validation and Inference

[ https://issues.apache.org/jira/browse/SPARK-23805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16420113#comment-16420113 ] zhengruifeng commented on SPARK-23805: -- [~sethah] [~josephkb] Are you interested in this? >

[jira] [Issue Comment Deleted] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuliang updated SPARK-5928: --- Comment: was deleted (was: !image-2018-03-29-11-52-32-075.png|width=741,height=189!   >From the picture, The

[jira] [Commented] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419995#comment-16419995 ] Apache Spark commented on SPARK-23825: -- User 'dvogelbacher' has created a pull request for this

[jira] [Assigned] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23825: Assignee: Apache Spark > [K8s] Spark pods should request memory + memoryOverhead as

[jira] [Assigned] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23825: Assignee: (was: Apache Spark) > [K8s] Spark pods should request memory +

[jira] [Updated] (SPARK-23822) Improve error message for Parquet schema mismatches

[ https://issues.apache.org/jira/browse/SPARK-23822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuchen Huo updated SPARK-23822: --- Component/s: (was: Input/Output) SQL > Improve error message for Parquet schema

[jira] [Commented] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419950#comment-16419950 ] David Vogelbacher commented on SPARK-23825: --- addressed by

[jira] [Created] (SPARK-23828) PySpark StringIndexerModel should have constructor from labels

Bryan Cutler created SPARK-23828: Summary: PySpark StringIndexerModel should have constructor from labels Key: SPARK-23828 URL: https://issues.apache.org/jira/browse/SPARK-23828 Project: Spark

[jira] [Commented] (SPARK-15009) PySpark CountVectorizerModel should be able to construct from vocabulary list

[ https://issues.apache.org/jira/browse/SPARK-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419939#comment-16419939 ] Apache Spark commented on SPARK-15009: -- User 'BryanCutler' has created a pull request for this

[jira] [Assigned] (SPARK-23827) StreamingJoinExec should ensure that input data is partitioned into specific number of partitions

[ https://issues.apache.org/jira/browse/SPARK-23827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23827: Assignee: Apache Spark (was: Tathagata Das) > StreamingJoinExec should ensure that input

[jira] [Commented] (SPARK-23827) StreamingJoinExec should ensure that input data is partitioned into specific number of partitions

[ https://issues.apache.org/jira/browse/SPARK-23827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419925#comment-16419925 ] Apache Spark commented on SPARK-23827: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23827) StreamingJoinExec should ensure that input data is partitioned into specific number of partitions

[ https://issues.apache.org/jira/browse/SPARK-23827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23827: Assignee: Tathagata Das (was: Apache Spark) > StreamingJoinExec should ensure that input

[jira] [Resolved] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

[ https://issues.apache.org/jira/browse/SPARK-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-22711. -- Resolution: Workaround Closing this because it seems wordnet is not serializable with

[jira] [Updated] (SPARK-23429) Add executor memory metrics to heartbeat and expose in executors REST API

[ https://issues.apache.org/jira/browse/SPARK-23429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edwina Lu updated SPARK-23429: -- Description: Add new executor level memory metrics ( jvmUsedMemory, onHeapExecutionMemory,

[jira] [Created] (SPARK-23827) StreamingJoinExec should ensure that input data is partitioned into specific number of partitions

Tathagata Das created SPARK-23827: - Summary: StreamingJoinExec should ensure that input data is partitioned into specific number of partitions Key: SPARK-23827 URL:

[jira] [Created] (SPARK-23826) TestHiveSparkSession should set default session

Jose Torres created SPARK-23826: --- Summary: TestHiveSparkSession should set default session Key: SPARK-23826 URL: https://issues.apache.org/jira/browse/SPARK-23826 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-23429) Add executor memory metrics to heartbeat and expose in executors REST API

[ https://issues.apache.org/jira/browse/SPARK-23429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23429: Assignee: (was: Apache Spark) > Add executor memory metrics to heartbeat and expose

[jira] [Commented] (SPARK-23429) Add executor memory metrics to heartbeat and expose in executors REST API

[ https://issues.apache.org/jira/browse/SPARK-23429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419861#comment-16419861 ] Apache Spark commented on SPARK-23429: -- User 'edwinalu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23429) Add executor memory metrics to heartbeat and expose in executors REST API

[ https://issues.apache.org/jira/browse/SPARK-23429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23429: Assignee: Apache Spark > Add executor memory metrics to heartbeat and expose in executors

[jira] [Commented] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419857#comment-16419857 ] David Vogelbacher commented on SPARK-23825: --- Will make a PR shortly, cc [~mcheah] > [K8s]

[jira] [Updated] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Vogelbacher updated SPARK-23825: -- Description: We currently request {{spark.[driver,executor].memory}} as memory from

[jira] [Updated] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

[ https://issues.apache.org/jira/browse/SPARK-23825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Vogelbacher updated SPARK-23825: -- Description: We currently request {{spark.{driver,executor}.memory}} as memory from

[jira] [Created] (SPARK-23825) [K8s] Spark pods should request memory + memoryOverhead as resources

David Vogelbacher created SPARK-23825: - Summary: [K8s] Spark pods should request memory + memoryOverhead as resources Key: SPARK-23825 URL: https://issues.apache.org/jira/browse/SPARK-23825

[jira] [Commented] (SPARK-20169) Groupby Bug with Sparksql

[ https://issues.apache.org/jira/browse/SPARK-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419808#comment-16419808 ] Maryann Xue commented on SPARK-20169: - [~smilegator], I think this is also caused by SPARK-23368, so

[jira] [Commented] (SPARK-23784) Cannot use custom Aggregator with groupBy/agg

[ https://issues.apache.org/jira/browse/SPARK-23784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419764#comment-16419764 ] Joshua Howard commented on SPARK-23784: --- Correct. It had been answered, but I am just now closing

[jira] [Closed] (SPARK-23784) Cannot use custom Aggregator with groupBy/agg

[ https://issues.apache.org/jira/browse/SPARK-23784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua Howard closed SPARK-23784. - See SO link. > Cannot use custom Aggregator with groupBy/agg >

[jira] [Resolved] (SPARK-23333) SparkML VectorAssembler.transform slow when needing to invoke .first() on sorted DataFrame

[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] V Luong resolved SPARK-2. - Resolution: Won't Fix > SparkML VectorAssembler.transform slow when needing to invoke .first() on >

[jira] [Commented] (SPARK-23333) SparkML VectorAssembler.transform slow when needing to invoke .first() on sorted DataFrame

[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419745#comment-16419745 ] V Luong commented on SPARK-2: - [~bago.amirbekian] thank you, that is indeed a good solution available

[jira] [Created] (SPARK-23824) Make inpurityStats publicly accessible in ml.tree.Node

Barry Becker created SPARK-23824: Summary: Make inpurityStats publicly accessible in ml.tree.Node Key: SPARK-23824 URL: https://issues.apache.org/jira/browse/SPARK-23824 Project: Spark Issue

[jira] [Assigned] (SPARK-23823) ResolveReferences loses correct origin

[ https://issues.apache.org/jira/browse/SPARK-23823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23823: Assignee: (was: Apache Spark) > ResolveReferences loses correct origin >

[jira] [Assigned] (SPARK-23823) ResolveReferences loses correct origin

[ https://issues.apache.org/jira/browse/SPARK-23823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23823: Assignee: Apache Spark > ResolveReferences loses correct origin >

[jira] [Commented] (SPARK-23823) ResolveReferences loses correct origin

[ https://issues.apache.org/jira/browse/SPARK-23823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419482#comment-16419482 ] Apache Spark commented on SPARK-23823: -- User 'JiahuiJiang' has created a pull request for this

[jira] [Created] (SPARK-23823) ResolveReferences loses correct origin

Jiahui Jiang created SPARK-23823: Summary: ResolveReferences loses correct origin Key: SPARK-23823 URL: https://issues.apache.org/jira/browse/SPARK-23823 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-23639) SparkSQL CLI fails talk to Kerberized metastore when use proxy user

[ https://issues.apache.org/jira/browse/SPARK-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23639. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by

[jira] [Assigned] (SPARK-23639) SparkSQL CLI fails talk to Kerberized metastore when use proxy user

[ https://issues.apache.org/jira/browse/SPARK-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23639: -- Assignee: Kent Yao > SparkSQL CLI fails talk to Kerberized metastore when use proxy

[jira] [Assigned] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23785: -- Assignee: Sahil Takiar > LauncherBackend doesn't check state of connection before

[jira] [Resolved] (SPARK-23785) LauncherBackend doesn't check state of connection before setting state

[ https://issues.apache.org/jira/browse/SPARK-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23785. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by

[jira] [Updated] (SPARK-23821) Collection function: flatten

[ https://issues.apache.org/jira/browse/SPARK-23821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Novotny updated SPARK-23821: -- Summary: Collection function: flatten (was: Collection functions: flatten) > Collection

[jira] [Assigned] (SPARK-23821) Collection functions: flatten

[ https://issues.apache.org/jira/browse/SPARK-23821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23821: Assignee: (was: Apache Spark) > Collection functions: flatten >

[jira] [Assigned] (SPARK-23821) Collection functions: flatten

[ https://issues.apache.org/jira/browse/SPARK-23821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23821: Assignee: Apache Spark > Collection functions: flatten > - >

[jira] [Commented] (SPARK-23821) Collection functions: flatten

[ https://issues.apache.org/jira/browse/SPARK-23821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419426#comment-16419426 ] Apache Spark commented on SPARK-23821: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Updated] (SPARK-23822) Improve error message for Parquet schema mismatches

[ https://issues.apache.org/jira/browse/SPARK-23822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuchen Huo updated SPARK-23822: --- Description: If a user attempts to read Parquet files with mismatched schemas and schema merging is

[jira] [Resolved] (SPARK-20498) RandomForestRegressionModel should expose getMaxDepth in PySpark

[ https://issues.apache.org/jira/browse/SPARK-20498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-20498. --- Resolution: Fixed Fix Version/s: 2.3.0 > RandomForestRegressionModel should

[jira] [Updated] (SPARK-20498) RandomForestRegressionModel should expose getMaxDepth in PySpark

[ https://issues.apache.org/jira/browse/SPARK-20498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20498: -- Target Version/s: (was: 2.4.0) > RandomForestRegressionModel should expose

[jira] [Commented] (SPARK-20498) RandomForestRegressionModel should expose getMaxDepth in PySpark

[ https://issues.apache.org/jira/browse/SPARK-20498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419397#comment-16419397 ] Joseph K. Bradley commented on SPARK-20498: --- I'll close this since Bryan's PR mostly solved

[jira] [Updated] (SPARK-20498) RandomForestRegressionModel should expose getMaxDepth in PySpark

[ https://issues.apache.org/jira/browse/SPARK-20498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20498: -- Shepherd: (was: Joseph K. Bradley) > RandomForestRegressionModel should expose

[jira] [Created] (SPARK-23821) Collection functions: flatten

Marek Novotny created SPARK-23821: - Summary: Collection functions: flatten Key: SPARK-23821 URL: https://issues.apache.org/jira/browse/SPARK-23821 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-23822) Improve error message for Parquet schema mismatches

Yuchen Huo created SPARK-23822: -- Summary: Improve error message for Parquet schema mismatches Key: SPARK-23822 URL: https://issues.apache.org/jira/browse/SPARK-23822 Project: Spark Issue Type:

[jira] [Issue Comment Deleted] (SPARK-20297) Parquet Decimal(12,2) written by Spark is unreadable by Hive and Impala

[ https://issues.apache.org/jira/browse/SPARK-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Ivanfi updated SPARK-20297: -- Comment: was deleted (was: Sorry, commented to the wrong JIRA.) > Parquet Decimal(12,2)

[jira] [Updated] (SPARK-23704) PySpark access of individual trees in random forest is slow

[ https://issues.apache.org/jira/browse/SPARK-23704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-23704: - Component/s: PySpark > PySpark access of individual trees in random forest is slow >

[jira] [Commented] (SPARK-20297) Parquet Decimal(12,2) written by Spark is unreadable by Hive and Impala

[ https://issues.apache.org/jira/browse/SPARK-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419338#comment-16419338 ] Zoltan Ivanfi commented on SPARK-20297: --- Sorry, commented to the wrong JIRA. > Parquet

[jira] [Issue Comment Deleted] (SPARK-20297) Parquet Decimal(12,2) written by Spark is unreadable by Hive and Impala

[ https://issues.apache.org/jira/browse/SPARK-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Ivanfi updated SPARK-20297: -- Comment: was deleted (was: Could you please clarify how those DECIMALS were written in the

[jira] [Commented] (SPARK-20297) Parquet Decimal(12,2) written by Spark is unreadable by Hive and Impala

[ https://issues.apache.org/jira/browse/SPARK-20297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419335#comment-16419335 ] Zoltan Ivanfi commented on SPARK-20297: --- Could you please clarify how those DECIMALS were written

[jira] [Commented] (SPARK-23503) continuous execution should sequence committed epochs

[ https://issues.apache.org/jira/browse/SPARK-23503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419136#comment-16419136 ] Efim Poberezkin commented on SPARK-23503: - [~joseph.torres] Good day Jose. From what I've figured

[jira] [Commented] (SPARK-23723) New charset option for json datasource

[ https://issues.apache.org/jira/browse/SPARK-23723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419123#comment-16419123 ] Apache Spark commented on SPARK-23723: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-23724) Custom record separator for jsons in charsets different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419124#comment-16419124 ] Apache Spark commented on SPARK-23724: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-23503) continuous execution should sequence committed epochs

[ https://issues.apache.org/jira/browse/SPARK-23503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419111#comment-16419111 ] Apache Spark commented on SPARK-23503: -- User 'efimpoberezkin' has created a pull request for this

[jira] [Assigned] (SPARK-23503) continuous execution should sequence committed epochs

[ https://issues.apache.org/jira/browse/SPARK-23503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23503: Assignee: (was: Apache Spark) > continuous execution should sequence committed epochs

[jira] [Assigned] (SPARK-23503) continuous execution should sequence committed epochs

[ https://issues.apache.org/jira/browse/SPARK-23503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23503: Assignee: Apache Spark > continuous execution should sequence committed epochs >

[jira] [Commented] (SPARK-21960) Spark Streaming Dynamic Allocation should respect spark.executor.instances

[ https://issues.apache.org/jira/browse/SPARK-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419104#comment-16419104 ] Leonel Atencio commented on SPARK-21960: This is a very important issue, because right now,

[jira] [Created] (SPARK-23820) Allow the long form of call sites to be recorded in the log

Michael Mior created SPARK-23820: Summary: Allow the long form of call sites to be recorded in the log Key: SPARK-23820 URL: https://issues.apache.org/jira/browse/SPARK-23820 Project: Spark

[jira] [Commented] (SPARK-991) Report call sites of operators in Python

[ https://issues.apache.org/jira/browse/SPARK-991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419099#comment-16419099 ] Michael Mior commented on SPARK-991: Maybe I'm missing something here, but I'm not seeing a Python

[jira] [Commented] (SPARK-22618) RDD.unpersist can cause fatal exception when used with dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419062#comment-16419062 ] Brad commented on SPARK-22618: -- Yeah the fix for broadcaset unpersist should be basically the same. Thanks

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419053#comment-16419053 ] Steve Loughran commented on SPARK-23534: [~nchammas] bq. Cloudera still ships 2.6 and EMR is on

[jira] [Updated] (SPARK-23807) Add Hadoop 3 profile with relevant POM fix ups, cloud-storage artifacts and binding

[ https://issues.apache.org/jira/browse/SPARK-23807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-23807: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-23534 > Add Hadoop 3 profile

[jira] [Commented] (SPARK-23807) Add Hadoop 3 profile with relevant POM fix ups, cloud-storage artifacts and binding

[ https://issues.apache.org/jira/browse/SPARK-23807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419017#comment-16419017 ] Steve Loughran commented on SPARK-23807: yes, this profile is part of the hadoop 3 support,

[jira] [Commented] (SPARK-15125) CSV data source recognizes empty quoted strings in the input as null.

[ https://issues.apache.org/jira/browse/SPARK-15125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418925#comment-16418925 ] Max Murphy commented on SPARK-15125: [~snanda] That would not allow ,, to be distinguished from ,"",

[jira] [Updated] (SPARK-23811) FetchFailed comes before Success of same task will cause child stage never succeed

[ https://issues.apache.org/jira/browse/SPARK-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-23811: Summary: FetchFailed comes before Success of same task will cause child stage never succeed (was:

[jira] [Assigned] (SPARK-23819) InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats

[ https://issues.apache.org/jira/browse/SPARK-23819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23819: Assignee: Apache Spark > InMemoryTableScanExec prunes orderable complex types due to out

[jira] [Assigned] (SPARK-23819) InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats

[ https://issues.apache.org/jira/browse/SPARK-23819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23819: Assignee: (was: Apache Spark) > InMemoryTableScanExec prunes orderable complex types

[jira] [Commented] (SPARK-23819) InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats

[ https://issues.apache.org/jira/browse/SPARK-23819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418777#comment-16418777 ] Apache Spark commented on SPARK-23819: -- User 'pwoody' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23818) an official UDF interface for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23818: Assignee: Apache Spark > an official UDF interface for Spark SQL >

[jira] [Commented] (SPARK-23818) an official UDF interface for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418754#comment-16418754 ] Apache Spark commented on SPARK-23818: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-23818) an official UDF interface for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23818: Assignee: (was: Apache Spark) > an official UDF interface for Spark SQL >

[jira] [Created] (SPARK-23819) InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats

Patrick Woody created SPARK-23819: - Summary: InMemoryTableScanExec prunes orderable complex types due to out of date ColumnStats Key: SPARK-23819 URL: https://issues.apache.org/jira/browse/SPARK-23819

[jira] [Created] (SPARK-23818) an official UDF interface for Spark SQL

Wenchen Fan created SPARK-23818: --- Summary: an official UDF interface for Spark SQL Key: SPARK-23818 URL: https://issues.apache.org/jira/browse/SPARK-23818 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-23770) Expose repartitionByRange in SparkR

[ https://issues.apache.org/jira/browse/SPARK-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23770: Assignee: Hyukjin Kwon > Expose repartitionByRange in SparkR >

[jira] [Resolved] (SPARK-23770) Expose repartitionByRange in SparkR

[ https://issues.apache.org/jira/browse/SPARK-23770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23770. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20902

[jira] [Comment Edited] (SPARK-20384) supporting value classes over primitives in DataSets

[ https://issues.apache.org/jira/browse/SPARK-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418691#comment-16418691 ] Furcy Pin edited comment on SPARK-20384 at 3/29/18 10:10 AM: - +1 on this

[jira] [Commented] (SPARK-20384) supporting value classes over primitives in DataSets

[ https://issues.apache.org/jira/browse/SPARK-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418691#comment-16418691 ] Furcy Pin commented on SPARK-20384: --- +1 on this issue. I think the generic use case is that the

[jira] [Comment Edited] (SPARK-22342) refactor schedulerDriver registration

[ https://issues.apache.org/jira/browse/SPARK-22342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418617#comment-16418617 ] Stavros Kontopoulos edited comment on SPARK-22342 at 3/29/18 9:43 AM:

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: duibi1.zip) > SQL which has large ‘case when’ expressions may cause code

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: duibi2.zip) > SQL which has large ‘case when’ expressions may cause code

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: duibi2.zip > SQL which has large ‘case when’ expressions may cause code generation

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: duibi1.zip > SQL which has large ‘case when’ expressions may cause code generation

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: shiro.ini) > SQL which has large ‘case when’ expressions may cause code

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: SecurityRestApi.java) > SQL which has large ‘case when’ expressions may cause

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: zeppelin-site.xml) > SQL which has large ‘case when’ expressions may cause

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: ZeppelinConfiguration.java) > SQL which has large ‘case when’ expressions may

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: SecurityUtils.java) > SQL which has large ‘case when’ expressions may cause

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: LoginRestApi.java) > SQL which has large ‘case when’ expressions may cause

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: GetUserList.java) > SQL which has large ‘case when’ expressions may cause code

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: (was: GbdLdapRealm.java) > SQL which has large ‘case when’ expressions may cause

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: (was: !test.JPG! when there are large 'ca !test2.JPG! se when ' expressions in

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Attachment: zeppelin-site.xml ZeppelinConfiguration.java shiro.ini

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fengchaoge updated SPARK-21337: --- Description: !test.JPG! when there are large 'ca !test2.JPG! se when ' expressions in spark sql,the

[jira] [Assigned] (SPARK-23817) Migrate ORC file format read path to data source V2

[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23817: Assignee: (was: Apache Spark) > Migrate ORC file format read path to data source V2 >

  1   2   >