[jira] [Commented] (SPARK-36280) Remove redundant aliases after RewritePredicateSubquery

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386646#comment-17386646 ] Apache Spark commented on SPARK-36280: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36280) Remove redundant aliases after RewritePredicateSubquery

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36280: Assignee: Apache Spark > Remove redundant aliases after RewritePredicateSubquery >

[jira] [Commented] (SPARK-36280) Remove redundant aliases after RewritePredicateSubquery

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386645#comment-17386645 ] Apache Spark commented on SPARK-36280: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36280) Remove redundant aliases after RewritePredicateSubquery

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36280: Assignee: (was: Apache Spark) > Remove redundant aliases after

[jira] [Created] (SPARK-36280) Remove redundant aliases after RewritePredicateSubquery

2021-07-23 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36280: --- Summary: Remove redundant aliases after RewritePredicateSubquery Key: SPARK-36280 URL: https://issues.apache.org/jira/browse/SPARK-36280 Project: Spark Issue

[jira] [Resolved] (SPARK-36274) Fix equality comparison of unordered Categoricals

2021-07-23 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36274. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Comment Edited] (SPARK-32797) Install mypy on the Jenkins CI workers

2021-07-23 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386522#comment-17386522 ] Shane Knapp edited comment on SPARK-32797 at 7/24/21, 1:05 AM: ---

[jira] [Resolved] (SPARK-36264) Add reorder_categories to CategoricalAccessor and CategoricalIndex.

2021-07-23 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36264. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Assigned] (SPARK-36058) Support replicasets/job API

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36058: Assignee: (was: Apache Spark) > Support replicasets/job API >

[jira] [Assigned] (SPARK-36058) Support replicasets/job API

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36058: Assignee: Apache Spark > Support replicasets/job API > --- > >

[jira] [Commented] (SPARK-36058) Support replicasets/job API

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386596#comment-17386596 ] Apache Spark commented on SPARK-36058: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36279) Fix lint-python to work with Python 3.9

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36279: Assignee: Apache Spark > Fix lint-python to work with Python 3.9 >

[jira] [Assigned] (SPARK-36279) Fix lint-python to work with Python 3.9

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36279: Assignee: (was: Apache Spark) > Fix lint-python to work with Python 3.9 >

[jira] [Commented] (SPARK-36279) Fix lint-python to work with Python 3.9

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386586#comment-17386586 ] Apache Spark commented on SPARK-36279: -- User 'ueshin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-35956) Support auto-assigning labels to less important pods (e.g. decommissioning pods)

2021-07-23 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-35956. -- Fix Version/s: 3.3.0 Target Version/s: 3.3.0 Resolution: Fixed > Support

[jira] [Created] (SPARK-36279) Fix lint-python to work with Python 3.9

2021-07-23 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36279: - Summary: Fix lint-python to work with Python 3.9 Key: SPARK-36279 URL: https://issues.apache.org/jira/browse/SPARK-36279 Project: Spark Issue Type:

[jira] [Created] (SPARK-36278) Make the dev-run-integration-tests & docker-image-tool able to use sbt build output

2021-07-23 Thread Holden Karau (Jira)
Holden Karau created SPARK-36278: Summary: Make the dev-run-integration-tests & docker-image-tool able to use sbt build output Key: SPARK-36278 URL: https://issues.apache.org/jira/browse/SPARK-36278

[jira] [Commented] (SPARK-36260) Add set_categories to CategoricalAccessor and CategoricalIndex

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386569#comment-17386569 ] Apache Spark commented on SPARK-36260: -- User 'xinrong-databricks' has created a pull request for

[jira] [Commented] (SPARK-36260) Add set_categories to CategoricalAccessor and CategoricalIndex

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386568#comment-17386568 ] Apache Spark commented on SPARK-36260: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-36260) Add set_categories to CategoricalAccessor and CategoricalIndex

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36260: Assignee: Apache Spark > Add set_categories to CategoricalAccessor and CategoricalIndex

[jira] [Assigned] (SPARK-36260) Add set_categories to CategoricalAccessor and CategoricalIndex

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36260: Assignee: (was: Apache Spark) > Add set_categories to CategoricalAccessor and

[jira] [Commented] (SPARK-36213) Normalize PartitionSpec for DescTable with PartitionSpec

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386565#comment-17386565 ] Apache Spark commented on SPARK-36213: -- User 'viirya' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36276) Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43

2021-07-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36276. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33500

[jira] [Assigned] (SPARK-36276) Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43

2021-07-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36276: - Assignee: William Hyun > Update maven-checkstyle-plugin to 3.1.2 and checkstyle to

[jira] [Commented] (SPARK-36267) Clean up CategoricalAccessor and CategoricalIndex.

2021-07-23 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386534#comment-17386534 ] Takuya Ueshin commented on SPARK-36267: --- I'll work on this. > Clean up CategoricalAccessor and

[jira] [Updated] (SPARK-36277) Issue with record count of data frame while reading in DropMalformed mode

2021-07-23 Thread anju (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anju updated SPARK-36277: - Description: While reading the dataframe in malformed mode ,I am not getting right record count.

[jira] [Updated] (SPARK-36277) Issue with record count of data frame while reading in DropMalformed mode

2021-07-23 Thread anju (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anju updated SPARK-36277: - Description: While reading the dataframe in malformed mode ,I am not getting right record count.

[jira] [Updated] (SPARK-36277) Issue with record count of data frame while reading in DropMalformed mode

2021-07-23 Thread anju (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anju updated SPARK-36277: - Description:  I am trying to read the two dataframes from one file using with/without predefined schema. Issue

[jira] [Updated] (SPARK-36277) Issue with record count of data frame while reading in DropMalformed mode

2021-07-23 Thread anju (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anju updated SPARK-36277: - Description:  I am trying to read the two dataframes from one file using with/without predefined schema. Issue

[jira] [Updated] (SPARK-36277) Issue with record count of data frame while reading in DropMalformed mode

2021-07-23 Thread anju (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anju updated SPARK-36277: - Attachment: 111.PNG > Issue with record count of data frame while reading in DropMalformed mode >

[jira] [Updated] (SPARK-36277) Issue with record count of data frame while reading in DropMalformed mode

2021-07-23 Thread anju (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anju updated SPARK-36277: - Summary: Issue with record count of data frame while reading in DropMalformed mode (was: Issue record count of

[jira] [Created] (SPARK-36277) Issue record count of data frame while reading in DropMalformed mode

2021-07-23 Thread anju (Jira)
anju created SPARK-36277: Summary: Issue record count of data frame while reading in DropMalformed mode Key: SPARK-36277 URL: https://issues.apache.org/jira/browse/SPARK-36277 Project: Spark Issue

[jira] [Commented] (SPARK-32797) Install mypy on the Jenkins CI workers

2021-07-23 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386524#comment-17386524 ] Shane Knapp commented on SPARK-32797: - i've temporarily disabled mypy on the workers: for x in `cat

[jira] [Commented] (SPARK-32797) Install mypy on the Jenkins CI workers

2021-07-23 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386522#comment-17386522 ] Shane Knapp commented on SPARK-32797: - [~hyukjin.kwon] this is now causing all builds to fail

[jira] [Commented] (SPARK-32797) Install mypy on the Jenkins CI workers

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386515#comment-17386515 ] Apache Spark commented on SPARK-32797: -- User 'shaneknapp' has created a pull request for this

[jira] [Assigned] (SPARK-32797) Install mypy on the Jenkins CI workers

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32797: Assignee: Shane Knapp (was: Apache Spark) > Install mypy on the Jenkins CI workers >

[jira] [Commented] (SPARK-32797) Install mypy on the Jenkins CI workers

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386514#comment-17386514 ] Apache Spark commented on SPARK-32797: -- User 'shaneknapp' has created a pull request for this

[jira] [Assigned] (SPARK-32797) Install mypy on the Jenkins CI workers

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32797: Assignee: Apache Spark (was: Shane Knapp) > Install mypy on the Jenkins CI workers >

[jira] [Reopened] (SPARK-32797) Install mypy on the Jenkins CI workers

2021-07-23 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp reopened SPARK-32797: - the version of mypy i installed wasn't current enough for the tests to run...  fixing that now. >

[jira] [Commented] (SPARK-36270) Change memory settings for enabling GA

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386503#comment-17386503 ] Apache Spark commented on SPARK-36270: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-36270) Change memory settings for enabling GA

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386502#comment-17386502 ] Apache Spark commented on SPARK-36270: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-36270) Change memory settings for enabling GA

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386496#comment-17386496 ] Apache Spark commented on SPARK-36270: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-36270) Change memory settings for enabling GA

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386497#comment-17386497 ] Apache Spark commented on SPARK-36270: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36276) Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36276: Assignee: Apache Spark > Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43

[jira] [Commented] (SPARK-36276) Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386489#comment-17386489 ] Apache Spark commented on SPARK-36276: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-36276) Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36276: Assignee: (was: Apache Spark) > Update maven-checkstyle-plugin to 3.1.2 and

[jira] [Created] (SPARK-36276) Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43

2021-07-23 Thread William Hyun (Jira)
William Hyun created SPARK-36276: Summary: Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43 Key: SPARK-36276 URL: https://issues.apache.org/jira/browse/SPARK-36276 Project: Spark

[jira] [Commented] (SPARK-36264) Add reorder_categories to CategoricalAccessor and CategoricalIndex.

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386445#comment-17386445 ] Apache Spark commented on SPARK-36264: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36264) Add reorder_categories to CategoricalAccessor and CategoricalIndex.

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36264: Assignee: (was: Apache Spark) > Add reorder_categories to CategoricalAccessor and

[jira] [Assigned] (SPARK-36264) Add reorder_categories to CategoricalAccessor and CategoricalIndex.

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36264: Assignee: Apache Spark > Add reorder_categories to CategoricalAccessor and

[jira] [Commented] (SPARK-36275) ResolveAggregateFunctions should work with nested fields

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386444#comment-17386444 ] Apache Spark commented on SPARK-36275: -- User 'allisonwang-db' has created a pull request for this

[jira] [Commented] (SPARK-36275) ResolveAggregateFunctions should work with nested fields

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386443#comment-17386443 ] Apache Spark commented on SPARK-36275: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-36275) ResolveAggregateFunctions should work with nested fields

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36275: Assignee: (was: Apache Spark) > ResolveAggregateFunctions should work with nested

[jira] [Assigned] (SPARK-36275) ResolveAggregateFunctions should work with nested fields

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36275: Assignee: Apache Spark > ResolveAggregateFunctions should work with nested fields >

[jira] [Created] (SPARK-36275) ResolveAggregateFunctions should work with nested fields

2021-07-23 Thread Allison Wang (Jira)
Allison Wang created SPARK-36275: Summary: ResolveAggregateFunctions should work with nested fields Key: SPARK-36275 URL: https://issues.apache.org/jira/browse/SPARK-36275 Project: Spark

[jira] [Resolved] (SPARK-36273) Comparison of identical values

2021-07-23 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36273. -- Fix Version/s: (was: 3.1.3) 3.2.0 Target Version/s: (was:

[jira] [Assigned] (SPARK-35561) partition result is incorrect when insert into partition table with int datatype partition column

2021-07-23 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35561: Assignee: YuanGuanhu > partition result is incorrect when insert into partition table

[jira] [Resolved] (SPARK-35561) partition result is incorrect when insert into partition table with int datatype partition column

2021-07-23 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35561. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33291

[jira] [Updated] (SPARK-36108) Refactor a few query parsing errors to use error classes

2021-07-23 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36108: --- Description: Refactor some exceptions in

[jira] [Assigned] (SPARK-36274) Fix equality comparison of unordered Categoricals

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36274: Assignee: (was: Apache Spark) > Fix equality comparison of unordered Categoricals >

[jira] [Assigned] (SPARK-36274) Fix equality comparison of unordered Categoricals

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36274: Assignee: Apache Spark > Fix equality comparison of unordered Categoricals >

[jira] [Commented] (SPARK-36274) Fix equality comparison of unordered Categoricals

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386416#comment-17386416 ] Apache Spark commented on SPARK-36274: -- User 'xinrong-databricks' has created a pull request for

[jira] [Updated] (SPARK-36107) Refactor a few query execution errors to use error classes

2021-07-23 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36107: --- Description: Refactor some exceptions in

[jira] [Created] (SPARK-36274) Fix equality comparison of unordered Categoricals

2021-07-23 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36274: Summary: Fix equality comparison of unordered Categoricals Key: SPARK-36274 URL: https://issues.apache.org/jira/browse/SPARK-36274 Project: Spark Issue

[jira] [Reopened] (SPARK-35997) Implement comparison operators for CategoricalDtype in pandas API on Spark

2021-07-23 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reopened SPARK-35997: -- > Implement comparison operators for CategoricalDtype in pandas API on Spark >

[jira] [Commented] (SPARK-36270) Change memory settings for enabling GA

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386354#comment-17386354 ] Apache Spark commented on SPARK-36270: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-36270) Change memory settings for enabling GA

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386353#comment-17386353 ] Apache Spark commented on SPARK-36270: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386346#comment-17386346 ] Apache Spark commented on SPARK-32920: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36242) Ensure spill file closed before set success to true in ExternalSorter.spillMemoryIteratorToDisk method

2021-07-23 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36242. -- Fix Version/s: 3.3.0 3.2.0 Assignee: Yang Jie Resolution: Fixed Issue

[jira] [Resolved] (SPARK-30186) support Dynamic Partition Pruning in Adaptive Execution

2021-07-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-30186. - Fix Version/s: 3.2.0 Assignee: Ke Jia Resolution: Fixed > support Dynamic

[jira] [Commented] (SPARK-36272) Re-enable SPARK-32629 ShuffledHashJoin(full outer) metrics

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386180#comment-17386180 ] Apache Spark commented on SPARK-36272: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36272) Re-enable SPARK-32629 ShuffledHashJoin(full outer) metrics

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36272: Assignee: Apache Spark > Re-enable SPARK-32629 ShuffledHashJoin(full outer) metrics >

[jira] [Assigned] (SPARK-36272) Re-enable SPARK-32629 ShuffledHashJoin(full outer) metrics

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36272: Assignee: (was: Apache Spark) > Re-enable SPARK-32629 ShuffledHashJoin(full outer)

[jira] [Commented] (SPARK-36272) Re-enable SPARK-32629 ShuffledHashJoin(full outer) metrics

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386181#comment-17386181 ] Apache Spark commented on SPARK-36272: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36273) Comparison of identical values

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36273: Assignee: Apache Spark > Comparison of identical values > --

[jira] [Assigned] (SPARK-36273) Comparison of identical values

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36273: Assignee: (was: Apache Spark) > Comparison of identical values >

[jira] [Commented] (SPARK-36273) Comparison of identical values

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386166#comment-17386166 ] Apache Spark commented on SPARK-36273: -- User 'almogtavor' has created a pull request for this

[jira] [Updated] (SPARK-36273) Comparison of identical values

2021-07-23 Thread Almog Tavor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Almog Tavor updated SPARK-36273: Description: This PR fixes the use of the "o.appAttemptId" variable instead of the mistaken

[jira] [Created] (SPARK-36273) Comparison of identical values

2021-07-23 Thread Almog Tavor (Jira)
Almog Tavor created SPARK-36273: --- Summary: Comparison of identical values Key: SPARK-36273 URL: https://issues.apache.org/jira/browse/SPARK-36273 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-36226) improve python docstring links to other pyspark classes

2021-07-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36226. -- Fix Version/s: 3.1.3 3.2.0 Assignee: Dominik Gehl

[jira] [Commented] (SPARK-36226) improve python docstring links to other pyspark classes

2021-07-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386141#comment-17386141 ] Hyukjin Kwon commented on SPARK-36226: -- Fixed in https://github.com/apache/spark/pull/33440 >

[jira] [Resolved] (SPARK-36258) Export functionExists in pyspark catalog

2021-07-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36258. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33481

[jira] [Assigned] (SPARK-36258) Export functionExists in pyspark catalog

2021-07-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36258: Assignee: Dominik Gehl > Export functionExists in pyspark catalog >

[jira] [Resolved] (SPARK-36270) Change memory settings for enabling GA

2021-07-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36270. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33447

[jira] [Assigned] (SPARK-36270) Change memory settings for enabling GA

2021-07-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36270: Assignee: L. C. Hsieh > Change memory settings for enabling GA >

[jira] [Commented] (SPARK-36052) Introduce pending pod limit for Spark on K8s

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386102#comment-17386102 ] Apache Spark commented on SPARK-36052: -- User 'attilapiros' has created a pull request for this

[jira] [Assigned] (SPARK-36052) Introduce pending pod limit for Spark on K8s

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36052: Assignee: Attila Zsolt Piros (was: Apache Spark) > Introduce pending pod limit for

[jira] [Assigned] (SPARK-36052) Introduce pending pod limit for Spark on K8s

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36052: Assignee: Apache Spark (was: Attila Zsolt Piros) > Introduce pending pod limit for

[jira] [Comment Edited] (SPARK-35933) PartitionFilters and pushFilters not applied to window functions

2021-07-23 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386086#comment-17386086 ] Saurabh Chawla edited comment on SPARK-35933 at 7/23/21, 8:34 AM: -- This

[jira] [Commented] (SPARK-35933) PartitionFilters and pushFilters not applied to window functions

2021-07-23 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386086#comment-17386086 ] Saurabh Chawla commented on SPARK-35933: This is not the bug,  its the actual functionality to

[jira] [Created] (SPARK-36272) Re-enable SPARK-32629 ShuffledHashJoin(full outer) metrics

2021-07-23 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-36272: --- Summary: Re-enable SPARK-32629 ShuffledHashJoin(full outer) metrics Key: SPARK-36272 URL: https://issues.apache.org/jira/browse/SPARK-36272 Project: Spark

[jira] [Resolved] (SPARK-30694) If exception occured while fetching blocks by ExternalBlockClient, fail early when External Shuffle Service is not alive

2021-07-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-30694. --- Resolution: Won't Fix > If exception occured while fetching blocks by ExternalBlockClient, fail

[jira] [Resolved] (SPARK-29769) Spark SQL cannot handle "exists/not exists" condition when using "JOIN"

2021-07-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-29769. --- Resolution: Implemented > Spark SQL cannot handle "exists/not exists" condition when using "JOIN" >

[jira] [Resolved] (SPARK-29018) Build spark thrift server on it's own code based on protocol v11

2021-07-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-29018. --- Resolution: Won't Fix > Build spark thrift server on it's own code based on protocol v11 >

[jira] [Resolved] (SPARK-28419) A patch for SparkThriftServer support multi-tenant authentication

2021-07-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-28419. --- Resolution: Won't Fix > A patch for SparkThriftServer support multi-tenant authentication >

[jira] [Commented] (SPARK-34851) Add scope tag in configuration

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386034#comment-17386034 ] Apache Spark commented on SPARK-34851: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-34851) Add scope tag in configuration

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34851: Assignee: Apache Spark > Add scope tag in configuration > --

[jira] [Assigned] (SPARK-34851) Add scope tag in configuration

2021-07-23 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34851: Assignee: (was: Apache Spark) > Add scope tag in configuration >

[jira] [Resolved] (SPARK-33912) Refactor DependencyUtils ivy property parameter

2021-07-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-33912. --- Resolution: Won't Fix > Refactor DependencyUtils ivy property parameter >

[jira] [Resolved] (SPARK-32818) Make hive metastore convert config can be changed session level

2021-07-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-32818. --- Resolution: Not A Problem > Make hive metastore convert config can be changed session level >

  1   2   >