[jira] [Commented] (SPARK-25452) Query with where clause is giving unexpected result in case of float column

[ https://issues.apache.org/jira/browse/SPARK-25452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628525#comment-16628525 ] Ayush Anubhava commented on SPARK-25452: Hi HyukjiKwon This issue does not seems to be

[jira] [Created] (SPARK-25544) Slow/failed convergence in Spark ML models due to internal predictor scaling

Andrew Crosby created SPARK-25544: - Summary: Slow/failed convergence in Spark ML models due to internal predictor scaling Key: SPARK-25544 URL: https://issues.apache.org/jira/browse/SPARK-25544

[jira] [Updated] (SPARK-25392) [Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page

[ https://issues.apache.org/jira/browse/SPARK-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-25392: - Description: Steps: 1.Enable spark.scheduler.mode = FAIR 2.Submitted beeline

[jira] [Comment Edited] (SPARK-16859) History Server storage information is missing

[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628554#comment-16628554 ] t oo edited comment on SPARK-16859 at 9/26/18 10:43 AM: bump @shahid was

[jira] [Assigned] (SPARK-25379) Improve ColumnPruning performance

[ https://issues.apache.org/jira/browse/SPARK-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25379: --- Assignee: Marco Gaido > Improve ColumnPruning performance >

[jira] [Resolved] (SPARK-25379) Improve ColumnPruning performance

[ https://issues.apache.org/jira/browse/SPARK-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25379. - Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22364

[jira] [Comment Edited] (SPARK-16859) History Server storage information is missing

[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628554#comment-16628554 ] t oo edited comment on SPARK-16859 at 9/26/18 10:46 AM: bump was (Author:

[jira] [Commented] (SPARK-25502) [Spark Job History] Empty Page when page number exceeds the reatinedTask size

[ https://issues.apache.org/jira/browse/SPARK-25502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628561#comment-16628561 ] t oo commented on SPARK-25502: -- related https://jira.apache.org/jira/browse/SPARK-16859 ? > [Spark Job

[jira] [Comment Edited] (SPARK-16859) History Server storage information is missing

[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628554#comment-16628554 ] t oo edited comment on SPARK-16859 at 9/26/18 10:45 AM: bump [~ashahid] was

[jira] [Commented] (SPARK-23401) Improve test cases for all supported types and unsupported types

[ https://issues.apache.org/jira/browse/SPARK-23401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628727#comment-16628727 ] Aleksandr Koriagin commented on SPARK-23401: I will take a look > Improve test cases for

[jira] [Commented] (SPARK-25392) [Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page

[ https://issues.apache.org/jira/browse/SPARK-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628742#comment-16628742 ] sandeep katta commented on SPARK-25392: --- [~abhishek.akg] as per current design pool details are

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628744#comment-16628744 ] Wenchen Fan commented on SPARK-25538: - cc [~kiszk] as well > incorrect row counts after distinct()

[jira] [Commented] (SPARK-24440) When use constant as column we may get wrong answer versus impala

[ https://issues.apache.org/jira/browse/SPARK-24440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628608#comment-16628608 ] Marco Gaido commented on SPARK-24440: - Can you provide a sample repro which can be run in order to

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628721#comment-16628721 ] Felix Cheung commented on SPARK-21291: -- The PR did not have bucketBy? > R bucketBy partitionBy

[jira] [Commented] (SPARK-25502) [Spark Job History] Empty Page when page number exceeds the reatinedTask size

[ https://issues.apache.org/jira/browse/SPARK-25502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628588#comment-16628588 ] shahid commented on SPARK-25502: [~toopt4] No. please refer the PR, to see the fix > [Spark Job

[jira] [Resolved] (SPARK-25541) CaseInsensitiveMap should be serializable after '-' or 'filterKeys'

[ https://issues.apache.org/jira/browse/SPARK-25541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25541. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 2.5.0 >

[jira] [Updated] (SPARK-25544) Slow/failed convergence in Spark ML models due to internal predictor scaling

[ https://issues.apache.org/jira/browse/SPARK-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Crosby updated SPARK-25544: -- Description: The LinearRegression and LogisticRegression estimators in Spark ML can take a

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628852#comment-16628852 ] Eugeniu commented on SPARK-18112: - This issue should be reopened. As already commented by [~Tavis]

[jira] [Updated] (SPARK-25544) Slow/failed convergence in Spark ML models due to internal predictor scaling

[ https://issues.apache.org/jira/browse/SPARK-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Crosby updated SPARK-25544: -- Description: The LinearRegression and LogisticRegression estimators in Spark ML can take a

[jira] [Created] (SPARK-25545) CSV loading with DROPMALFORMED mode doesn't correctly drop rows that do not confirm to non-nullable schema fields

Steven Bakhtiari created SPARK-25545: Summary: CSV loading with DROPMALFORMED mode doesn't correctly drop rows that do not confirm to non-nullable schema fields Key: SPARK-25545 URL:

[jira] [Commented] (SPARK-25545) CSV loading with DROPMALFORMED mode doesn't correctly drop rows that do not confirm to non-nullable schema fields

[ https://issues.apache.org/jira/browse/SPARK-25545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628949#comment-16628949 ] Steven Bakhtiari commented on SPARK-25545: -- Somebody on SO pointed me to this older ticket that

[jira] [Resolved] (SPARK-25509) SHS V2 cannot enabled in Windows, because POSIX permissions is not support.

[ https://issues.apache.org/jira/browse/SPARK-25509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25509. --- Resolution: Fixed Fix Version/s: 2.4.0 2.3.3 Issue resolved by pull

[jira] [Assigned] (SPARK-25509) SHS V2 cannot enabled in Windows, because POSIX permissions is not support.

[ https://issues.apache.org/jira/browse/SPARK-25509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25509: - Assignee: Rong Tang > SHS V2 cannot enabled in Windows, because POSIX permissions is not

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628865#comment-16628865 ] Hyukjin Kwon commented on SPARK-18112: -- Can you post reproducer steps please before we open this?

[jira] [Updated] (SPARK-25544) Slow/failed convergence in Spark ML models due to internal predictor scaling

[ https://issues.apache.org/jira/browse/SPARK-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Crosby updated SPARK-25544: -- Description: The LinearRegression and LogisticRegression estimators in Spark ML can take a

[jira] [Resolved] (SPARK-20937) Describe spark.sql.parquet.writeLegacyFormat property in Spark SQL, DataFrames and Datasets Guide

[ https://issues.apache.org/jira/browse/SPARK-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-20937. -- Resolution: Fixed Fix Version/s: 2.4.1 2.5.0 Issue resolved by pull

[jira] [Assigned] (SPARK-20937) Describe spark.sql.parquet.writeLegacyFormat property in Spark SQL, DataFrames and Datasets Guide

[ https://issues.apache.org/jira/browse/SPARK-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-20937: Assignee: Chenxiao Mao > Describe spark.sql.parquet.writeLegacyFormat property in Spark

[jira] [Created] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

Marcelo Vanzin created SPARK-25546: -- Summary: RDDInfo uses SparkEnv before it may have been initialized Key: SPARK-25546 URL: https://issues.apache.org/jira/browse/SPARK-25546 Project: Spark

[jira] [Commented] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629216#comment-16629216 ] Apache Spark commented on SPARK-25546: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25546: Assignee: (was: Apache Spark) > RDDInfo uses SparkEnv before it may have been

[jira] [Commented] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629218#comment-16629218 ] Apache Spark commented on SPARK-25546: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629198#comment-16629198 ] Huaxin Gao commented on SPARK-21291: [~felixcheung] I will submit a PR for bucketBy.  bucketBy

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629231#comment-16629231 ] David Spies commented on SPARK-18492: - Ran into this as well. It seems like this is happening

[jira] [Assigned] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25546: Assignee: Apache Spark > RDDInfo uses SparkEnv before it may have been initialized >

[jira] [Assigned] (SPARK-25533) Inconsistent message for Completed Jobs in the JobUI, when there are failed jobs, compared to spark2.2

[ https://issues.apache.org/jira/browse/SPARK-25533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25533: -- Assignee: shahid > Inconsistent message for Completed Jobs in the JobUI, when there

[jira] [Resolved] (SPARK-25318) Add exception handling when wrapping the input stream during the the fetch or stage retry in response to a corrupted block

[ https://issues.apache.org/jira/browse/SPARK-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25318. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22325

[jira] [Updated] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25536: -- Affects Version/s: 2.3.0 2.3.1 > executorSource.METRIC read wrong

[jira] [Assigned] (SPARK-25318) Add exception handling when wrapping the input stream during the the fetch or stage retry in response to a corrupted block

[ https://issues.apache.org/jira/browse/SPARK-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25318: -- Assignee: Reza Safi > Add exception handling when wrapping the input stream during

[jira] [Commented] (SPARK-25535) Work around bad error checking in commons-crypto

[ https://issues.apache.org/jira/browse/SPARK-25535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629174#comment-16629174 ] Apache Spark commented on SPARK-25535: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25535) Work around bad error checking in commons-crypto

[ https://issues.apache.org/jira/browse/SPARK-25535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25535: Assignee: (was: Apache Spark) > Work around bad error checking in commons-crypto >

[jira] [Assigned] (SPARK-25535) Work around bad error checking in commons-crypto

[ https://issues.apache.org/jira/browse/SPARK-25535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25535: Assignee: Apache Spark > Work around bad error checking in commons-crypto >

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629182#comment-16629182 ] Leo Gallucci commented on SPARK-18112: -- And to get things worse Hive is already in version 3. Same

[jira] [Issue Comment Deleted] (SPARK-25546) RDDInfo uses SparkEnv before it may have been initialized

[ https://issues.apache.org/jira/browse/SPARK-25546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-25546: --- Comment: was deleted (was: User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629000#comment-16629000 ] Eugeniu commented on SPARK-18112: - I can only describe my situation. I am using AWS EMR 5.17.0 with

[jira] [Commented] (SPARK-25533) Inconsistent message for Completed Jobs in the JobUI, when there are failed jobs, compared to spark2.2

[ https://issues.apache.org/jira/browse/SPARK-25533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629243#comment-16629243 ] Marcelo Vanzin commented on SPARK-25533: This is merged to master. I'll backport it to 2.4 and

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629281#comment-16629281 ] Kazuaki Ishizaki commented on SPARK-25538: -- Hi [~Steven Rand], would it be possible to share

[jira] [Commented] (SPARK-17952) SparkSession createDataFrame method throws exception for nested JavaBeans

[ https://issues.apache.org/jira/browse/SPARK-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629321#comment-16629321 ] Michal Šenkýř commented on SPARK-17952: --- Implemented nested bean support in pull request. Arrays

[jira] [Commented] (SPARK-25501) Kafka delegation token support

[ https://issues.apache.org/jira/browse/SPARK-25501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629320#comment-16629320 ] Mingjie Tang commented on SPARK-25501: -- [~gsomogyi] Thanks for your reply. At first, what my PR

[jira] [Commented] (SPARK-25531) new write APIs for data source v2

[ https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629418#comment-16629418 ] Ryan Blue commented on SPARK-25531: --- [~cloud_fan], what was the intent for this umbrella issue? You

[jira] [Created] (SPARK-25547) Pluggable jdbc connection factory

Frank Sauer created SPARK-25547: --- Summary: Pluggable jdbc connection factory Key: SPARK-25547 URL: https://issues.apache.org/jira/browse/SPARK-25547 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-25547) Pluggable jdbc connection factory

[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25547: Assignee: Apache Spark > Pluggable jdbc connection factory >

[jira] [Commented] (SPARK-25547) Pluggable jdbc connection factory

[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629425#comment-16629425 ] Apache Spark commented on SPARK-25547: -- User 'fsauer65' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25547) Pluggable jdbc connection factory

[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25547: Assignee: (was: Apache Spark) > Pluggable jdbc connection factory >

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629420#comment-16629420 ] t oo commented on SPARK-18112: -- here, here! > Spark2.x does not support read data from Hive 2.x metastore

[jira] [Updated] (SPARK-24285) Flaky test: ContinuousSuite.query without test harness

[ https://issues.apache.org/jira/browse/SPARK-24285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24285: -- Description: *2.5.0-SNAPSHOT* -

[jira] [Updated] (SPARK-24285) Flaky test: ContinuousSuite.query without test harness

[ https://issues.apache.org/jira/browse/SPARK-24285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24285: -- Description: *2.5.0-SNAPSHOT* -

[jira] [Resolved] (SPARK-25372) Deprecate Yarn-specific configs in regards to keytab login for SparkSubmit

[ https://issues.apache.org/jira/browse/SPARK-25372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25372. Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22362

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629561#comment-16629561 ] Steven Rand commented on SPARK-25538: - [~kiszk], yes, the schema is:   {code} scala>

[jira] [Assigned] (SPARK-25372) Deprecate Yarn-specific configs in regards to keytab login for SparkSubmit

[ https://issues.apache.org/jira/browse/SPARK-25372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25372: -- Assignee: Ilan Filonenko > Deprecate Yarn-specific configs in regards to keytab

[jira] [Resolved] (SPARK-25454) Division between operands with negative scale can cause precision loss

[ https://issues.apache.org/jira/browse/SPARK-25454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25454. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.4.0 2.3.3 >

[jira] [Assigned] (SPARK-25540) Make HiveContext in PySpark behave as the same as Scala.

[ https://issues.apache.org/jira/browse/SPARK-25540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25540: --- Assignee: Takuya Ueshin > Make HiveContext in PySpark behave as the same as Scala. >

[jira] [Resolved] (SPARK-25540) Make HiveContext in PySpark behave as the same as Scala.

[ https://issues.apache.org/jira/browse/SPARK-25540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25540. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22552

[jira] [Commented] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629655#comment-16629655 ] Apache Spark commented on SPARK-25548: -- User 'eatoncys' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25548: Assignee: Apache Spark > In the PruneFileSourcePartitions optimizer, replace the

[jira] [Assigned] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25548: Assignee: Apache Spark > In the PruneFileSourcePartitions optimizer, replace the

[jira] [Assigned] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

[ https://issues.apache.org/jira/browse/SPARK-25548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25548: Assignee: (was: Apache Spark) > In the PruneFileSourcePartitions optimizer, replace

[jira] [Commented] (SPARK-25531) new write APIs for data source v2

[ https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629550#comment-16629550 ] Wenchen Fan commented on SPARK-25531: - I want to have a more structured view of the data source v2

[jira] [Commented] (SPARK-25351) Handle Pandas category type when converting from Python with Arrow

[ https://issues.apache.org/jira/browse/SPARK-25351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629572#comment-16629572 ] Bryan Cutler commented on SPARK-25351: -- Hi [~pgadige], yes please go ahead with this issue! When

[jira] [Reopened] (SPARK-25454) Division between operands with negative scale can cause precision loss

[ https://issues.apache.org/jira/browse/SPARK-25454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-25454: - Assignee: (was: Wenchen Fan) I'm reopening it, since the bug is not fully fixed. But we

[jira] [Updated] (SPARK-25454) Division between operands with negative scale can cause precision loss

[ https://issues.apache.org/jira/browse/SPARK-25454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25454: Fix Version/s: (was: 2.3.3) (was: 2.4.0) > Division between operands

[jira] [Created] (SPARK-25548) In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned

eaton created SPARK-25548: - Summary: In the PruneFileSourcePartitions optimizer, replace the nonPartitionOps field with true in the And(partitionOps, nonPartitionOps) to make the partition can be pruned Key: SPARK-25548

[jira] [Commented] (SPARK-16859) History Server storage information is missing

[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628554#comment-16628554 ] t oo commented on SPARK-16859: -- bump > History Server storage information is missing >

[jira] [Updated] (SPARK-25541) CaseInsensitiveMap should be serializable after '-' operator

[ https://issues.apache.org/jira/browse/SPARK-25541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-25541: --- Summary: CaseInsensitiveMap should be serializable after '-' operator (was:

[jira] [Commented] (SPARK-25549) High level API to collect RDD statistics

[ https://issues.apache.org/jira/browse/SPARK-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629702#comment-16629702 ] Liang-Chi Hsieh commented on SPARK-25549: - cc [~cloud_fan]   > High level API to collect RDD

[jira] [Resolved] (SPARK-25481) Refactor ColumnarBatchBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25481. --- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22490

[jira] [Assigned] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25536: - Assignee: shahid > executorSource.METRIC read wrong record in Executor.scala Line444 >

[jira] [Commented] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629740#comment-16629740 ] Dongjoon Hyun commented on SPARK-25536: --- Issue resolved by pull request 22555

[jira] [Resolved] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25536. --- Resolution: Fixed > executorSource.METRIC read wrong record in Executor.scala Line444 >

[jira] [Updated] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25536: -- Fix Version/s: 2.4.0 2.3.3 > executorSource.METRIC read wrong record in

[jira] [Updated] (SPARK-25540) Make HiveContext in PySpark behave as the same as Scala.

[ https://issues.apache.org/jira/browse/SPARK-25540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25540: - Fix Version/s: (was: 2.4.0) 2.5.0 > Make HiveContext in PySpark behave

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629749#comment-16629749 ] Hyukjin Kwon commented on SPARK-18112: -- Hive 3 support. See

[jira] [Resolved] (SPARK-25468) Highlight current page index in the history server

[ https://issues.apache.org/jira/browse/SPARK-25468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25468. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22516

[jira] [Commented] (SPARK-25541) CaseInsensitiveMap should be serializable after '-' or 'filterKeys'

[ https://issues.apache.org/jira/browse/SPARK-25541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629690#comment-16629690 ] Apache Spark commented on SPARK-25541: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-24341) Codegen compile error from predicate subquery

[ https://issues.apache.org/jira/browse/SPARK-24341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629698#comment-16629698 ] Apache Spark commented on SPARK-24341: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-25549) High level API to collect RDD statistics

[ https://issues.apache.org/jira/browse/SPARK-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629700#comment-16629700 ] Liang-Chi Hsieh commented on SPARK-25549: - The design doc is at:

[jira] [Created] (SPARK-25549) High level API to collect RDD statistics

Liang-Chi Hsieh created SPARK-25549: --- Summary: High level API to collect RDD statistics Key: SPARK-25549 URL: https://issues.apache.org/jira/browse/SPARK-25549 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-25481) Refactor ColumnarBatchBenchmark to use main method

[ https://issues.apache.org/jira/browse/SPARK-25481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25481: - Assignee: yucai > Refactor ColumnarBatchBenchmark to use main method >

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629742#comment-16629742 ] Hyukjin Kwon commented on SPARK-18112: -- Hive 3 support is blocked by Hadoop 3 profile. See

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629743#comment-16629743 ] Hyukjin Kwon commented on SPARK-18112: -- Re:

[jira] [Assigned] (SPARK-25525) Do not update conf for existing SparkContext in SparkSession.getOrCreate.

[ https://issues.apache.org/jira/browse/SPARK-25525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25525: Assignee: Takuya Ueshin > Do not update conf for existing SparkContext in

[jira] [Resolved] (SPARK-25525) Do not update conf for existing SparkContext in SparkSession.getOrCreate.

[ https://issues.apache.org/jira/browse/SPARK-25525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25525. -- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22545

[jira] [Comment Edited] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629742#comment-16629742 ] Hyukjin Kwon edited comment on SPARK-18112 at 9/27/18 4:42 AM: --- Hadoop 3

[jira] [Assigned] (SPARK-25468) Highlight current page index in the history server

[ https://issues.apache.org/jira/browse/SPARK-25468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25468: - Assignee: Adam Wang > Highlight current page index in the history server >

[jira] [Assigned] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25536: Assignee: (was: Apache Spark) > executorSource.METRIC read wrong record in

[jira] [Assigned] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25536: Assignee: Apache Spark > executorSource.METRIC read wrong record in Executor.scala

[jira] [Commented] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628339#comment-16628339 ] Apache Spark commented on SPARK-25536: -- User 'shahidki31' has created a pull request for this

[jira] [Comment Edited] (SPARK-25536) executorSource.METRIC read wrong record in Executor.scala Line444

[ https://issues.apache.org/jira/browse/SPARK-25536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628267#comment-16628267 ] shahid edited comment on SPARK-25536 at 9/26/18 7:18 AM: - Thanks. I will raise a

[jira] [Updated] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-25538: Priority: Major (was: Blocker) > incorrect row counts after distinct() >

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628369#comment-16628369 ] Marco Gaido commented on SPARK-25538: - Please do not use Blocker and Critical when reporting issues

[jira] [Updated] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-25538: Labels: correctness (was: ) > incorrect row counts after distinct() >

  1   2   >