[jira] [Commented] (SPARK-31851) Redesign PySpark documentation

2020-07-10 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155885#comment-17155885 ] Jijo Sunny commented on SPARK-31851: Sure, let me know here when we are good to start SPARK-32180 

[jira] [Commented] (SPARK-24985) Executing SQL with "Full Outer Join" on top of large tables when there is data skew met OOM

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155839#comment-17155839 ] Apache Spark commented on SPARK-24985: -- User 'sidedoorleftroad' has created a pull request for this

[jira] [Updated] (SPARK-32278) Install PyPy3 on Jenkins to enable PySpark tests with PyPy

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32278: - Description: Current PyPy installed in Jenkins is too old, which is Python 2 compatible.

[jira] [Updated] (SPARK-32279) Install Sphinx in Python 3 on Jenkins machines

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32279: - Description: Currently Sphinx is only installed in Python 2. We should install it in Python 3

[jira] [Updated] (SPARK-32278) Install PyPy3 on Jenkins to enable PySpark tests with PyPy

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32278: - Summary: Install PyPy3 on Jenkins to enable PySpark tests with PyPy (was: Enable the PySpark

[jira] [Assigned] (SPARK-32279) Install Sphinx in Python 3 on Jenkins machines

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32279: Assignee: Shane Knapp > Install Sphinx in Python 3 on Jenkins machines >

[jira] [Updated] (SPARK-32278) Enable the PySpark tests with PyPy3 on Jenkins

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32278: - Description: Current PyPy installed in Jenkins is too old, which is Python 2 compatible.

[jira] [Assigned] (SPARK-32278) Enable the PySpark tests with PyPy3 on Jenkins

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32278: Assignee: Shane Knapp > Enable the PySpark tests with PyPy3 on Jenkins >

[jira] [Commented] (SPARK-32278) Enable the PySpark tests with PyPy3 on Jenkins

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155823#comment-17155823 ] Hyukjin Kwon commented on SPARK-32278: -- Hey [~shaneknapp] would you mind taking a look when you

[jira] [Commented] (SPARK-32279) Install Sphinx in Python 3 on Jenkins machines

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155824#comment-17155824 ] Hyukjin Kwon commented on SPARK-32279: -- Hey [~shaneknapp] would you mind taking a look when you

[jira] [Updated] (SPARK-32279) Install Sphinx in Python 3 on Jenkins machines

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32279: - Description: Currently Sphinx is only installed in Python 2. We should install it in Python 3

[jira] [Created] (SPARK-32279) Install Sphinx in Python 3 on Jenkins machines

2020-07-10 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32279: Summary: Install Sphinx in Python 3 on Jenkins machines Key: SPARK-32279 URL: https://issues.apache.org/jira/browse/SPARK-32279 Project: Spark Issue Type:

[jira] [Updated] (SPARK-32266) Run smoke tests after a commit is pushed

2020-07-10 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-32266: --- Description: Run linter/sbt build/maven build/doc generation on commit pushed. > Run smoke

[jira] [Updated] (SPARK-32266) Run smoke tests after a commit is pushed

2020-07-10 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-32266: --- Affects Version/s: (was: 2.4.6) (was: 3.0.0) > Run smoke

[jira] [Created] (SPARK-32278) Enable the PySpark tests with PyPy3 on Jenkins

2020-07-10 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32278: Summary: Enable the PySpark tests with PyPy3 on Jenkins Key: SPARK-32278 URL: https://issues.apache.org/jira/browse/SPARK-32278 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32245) Implement the base to run Spark tests in GitHun Actions

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155808#comment-17155808 ] Hyukjin Kwon commented on SPARK-32245: -- Sure, let's merge this in master only for now and see how

[jira] [Updated] (SPARK-32244) Build and run the Spark with test cases in Github Actions

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32244: - Affects Version/s: 2.4.6 3.0.0 > Build and run the Spark with test cases

[jira] [Commented] (SPARK-32244) Build and run the Spark with test cases in Github Actions

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155807#comment-17155807 ] Hyukjin Kwon commented on SPARK-32244: -- [~dongjoon] that's filed in SPARK-32249. If we're going to

[jira] [Updated] (SPARK-23258) Should not split Arrow record batches based on row count

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23258: - Labels: (was: bulk-closed) > Should not split Arrow record batches based on row count >

[jira] [Updated] (SPARK-23258) Should not split Arrow record batches based on row count

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23258: - Affects Version/s: (was: 2.3.0) 3.1.0 > Should not split Arrow

[jira] [Reopened] (SPARK-23258) Should not split Arrow record batches based on row count

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-23258: -- Assignee: Bryan Cutler > Should not split Arrow record batches based on row count >

[jira] [Updated] (SPARK-32277) Memory limit exception on high usage of direct buffer pool

2020-07-10 Thread Sathyaprakash Govindasamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathyaprakash Govindasamy updated SPARK-32277: -- Description: We have a spark application that uses jdbc to read a

[jira] [Updated] (SPARK-32277) Memory limit exception on high usage of direct buffer pool

2020-07-10 Thread Sathyaprakash Govindasamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathyaprakash Govindasamy updated SPARK-32277: -- Description: We have a spark application that uses jdbc to read a

[jira] [Updated] (SPARK-32277) Memory limit exception on high usage of direct buffer pool

2020-07-10 Thread Sathyaprakash Govindasamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathyaprakash Govindasamy updated SPARK-32277: -- Description: We have a spark application that uses jdbc to read a

[jira] [Updated] (SPARK-32277) Memory limit exception on high usage of direct buffer pool

2020-07-10 Thread Sathyaprakash Govindasamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathyaprakash Govindasamy updated SPARK-32277: -- Description: We have a spark application that uses jdbc to read a

[jira] [Updated] (SPARK-32277) Memory limit exception on high usage of direct buffer pool

2020-07-10 Thread Sathyaprakash Govindasamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathyaprakash Govindasamy updated SPARK-32277: -- Description: We have a spark application that uses jdbc to read a

[jira] [Created] (SPARK-32277) Memory limit exception on high usage of direct buffer pool

2020-07-10 Thread Sathyaprakash Govindasamy (Jira)
Sathyaprakash Govindasamy created SPARK-32277: - Summary: Memory limit exception on high usage of direct buffer pool Key: SPARK-32277 URL: https://issues.apache.org/jira/browse/SPARK-32277

[jira] [Updated] (SPARK-31226) SizeBasedCoalesce logic error

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31226: -- Component/s: Tests > SizeBasedCoalesce logic error > - > >

[jira] [Updated] (SPARK-31226) Fix SizeBasedCoalesce in tests

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31226: -- Summary: Fix SizeBasedCoalesce in tests (was: SizeBasedCoalesce logic error) > Fix

[jira] [Commented] (SPARK-32220) Cartesian Product Hint cause data error

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155786#comment-17155786 ] Apache Spark commented on SPARK-32220: -- User 'AngersZh' has created a pull request for this

[jira] [Updated] (SPARK-32276) Remove redundant sorts before repartition nodes

2020-07-10 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-32276: - Summary: Remove redundant sorts before repartition nodes (was: Remove redundant sorts

[jira] [Created] (SPARK-32276) Remove redundant sorts before repartition/repartitionByExpression/coalesce

2020-07-10 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-32276: Summary: Remove redundant sorts before repartition/repartitionByExpression/coalesce Key: SPARK-32276 URL: https://issues.apache.org/jira/browse/SPARK-32276

[jira] [Resolved] (SPARK-32251) fix SQL keyword document

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32251. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-32251) fix SQL keyword document

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32251: -- Issue Type: Bug (was: Improvement) > fix SQL keyword document > > >

[jira] [Closed] (SPARK-32254) Reenable SparkSQLEnvSuite's "external listeners should be initialized with Spark classloader"

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-32254. - > Reenable SparkSQLEnvSuite's "external listeners should be initialized with > Spark classloader"

[jira] [Commented] (SPARK-32244) Build and run the Spark with test cases in Github Actions

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155720#comment-17155720 ] Dongjoon Hyun commented on SPARK-32244: --- Please target this `3.1.0` first. After we stabilize and

[jira] [Commented] (SPARK-32245) Implement the base to run Spark tests in GitHun Actions

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155719#comment-17155719 ] Dongjoon Hyun commented on SPARK-32245: --- This should be applied `master` branch only. We need to

[jira] [Updated] (SPARK-32244) Build and run the Spark with test cases in Github Actions

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32244: -- Affects Version/s: (was: 2.4.6) (was: 3.0.0) > Build and run

[jira] [Updated] (SPARK-32245) Implement the base to run Spark tests in GitHun Actions

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32245: -- Affects Version/s: (was: 2.4.6) (was: 3.0.0) > Implement the

[jira] [Assigned] (SPARK-32103) Spark support IPV6 in yarn mode

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32103: - Assignee: pavithra ramachandran > Spark support IPV6 in yarn mode >

[jira] [Resolved] (SPARK-32103) Spark support IPV6 in yarn mode

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32103. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28931

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-07-10 Thread Shivaram Venkataraman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155697#comment-17155697 ] Shivaram Venkataraman commented on SPARK-31918: --- Yes – this is the reason that SparkR has

[jira] [Commented] (SPARK-31831) Flaky test: org.apache.spark.sql.hive.thriftserver.HiveSessionImplSuite.(It is not a test it is a sbt.testing.SuiteSelector)

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155688#comment-17155688 ] Apache Spark commented on SPARK-31831: -- User 'frankyin-factual' has created a pull request for this

[jira] [Commented] (SPARK-27892) Saving/loading stages in PipelineModel should be parallel

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155684#comment-17155684 ] Apache Spark commented on SPARK-27892: -- User 'Moovlin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-27892) Saving/loading stages in PipelineModel should be parallel

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27892: Assignee: Apache Spark > Saving/loading stages in PipelineModel should be parallel >

[jira] [Assigned] (SPARK-27892) Saving/loading stages in PipelineModel should be parallel

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27892: Assignee: (was: Apache Spark) > Saving/loading stages in PipelineModel should be

[jira] [Commented] (SPARK-27892) Saving/loading stages in PipelineModel should be parallel

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155682#comment-17155682 ] Apache Spark commented on SPARK-27892: -- User 'Moovlin' has created a pull request for this issue:

[jira] [Created] (SPARK-32275) "None.org.apache.spark.api.java.JavaSparkContext" Issue With Spark-Mllib Algorithm and JDBC Connectors

2020-07-10 Thread Luke Chu (Jira)
Luke Chu created SPARK-32275: Summary: "None.org.apache.spark.api.java.JavaSparkContext" Issue With Spark-Mllib Algorithm and JDBC Connectors Key: SPARK-32275 URL: https://issues.apache.org/jira/browse/SPARK-32275

[jira] [Resolved] (SPARK-31447) DATE_PART functions produces incorrect result

2020-07-10 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31447. -- Resolution: Won't Fix > DATE_PART functions produces incorrect result >

[jira] [Resolved] (SPARK-31592) bufferPoolsBySize in HeapMemoryAllocator should be thread safe

2020-07-10 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31592. -- Resolution: Cannot Reproduce > bufferPoolsBySize in HeapMemoryAllocator should be thread safe

[jira] [Commented] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155661#comment-17155661 ] Apache Spark commented on SPARK-32274: -- User 'revans2' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32274: Assignee: (was: Apache Spark) > Add in the ability for a user to replace the

[jira] [Assigned] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32274: Assignee: Apache Spark > Add in the ability for a user to replace the serialization

[jira] [Commented] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

2020-07-10 Thread Robert Joseph Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155660#comment-17155660 ] Robert Joseph Evans commented on SPARK-32274: - I filed

[jira] [Assigned] (SPARK-23889) DataSourceV2: Add interfaces to pass required sorting and clustering for writes

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23889: Assignee: (was: Apache Spark) > DataSourceV2: Add interfaces to pass required

[jira] [Commented] (SPARK-23889) DataSourceV2: Add interfaces to pass required sorting and clustering for writes

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155656#comment-17155656 ] Apache Spark commented on SPARK-23889: -- User 'aokolnychyi' has created a pull request for this

[jira] [Assigned] (SPARK-23889) DataSourceV2: Add interfaces to pass required sorting and clustering for writes

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23889: Assignee: Apache Spark > DataSourceV2: Add interfaces to pass required sorting and

[jira] [Updated] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

2020-07-10 Thread Robert Joseph Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated SPARK-32274: Description: Caching a dataset or dataframe can be a very expensive operation,

[jira] [Commented] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

2020-07-10 Thread Robert Joseph Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155654#comment-17155654 ] Robert Joseph Evans commented on SPARK-32274: - If someone could assign this to me that would

[jira] [Created] (SPARK-32274) Add in the ability for a user to replace the serialization format of the cache

2020-07-10 Thread Robert Joseph Evans (Jira)
Robert Joseph Evans created SPARK-32274: --- Summary: Add in the ability for a user to replace the serialization format of the cache Key: SPARK-32274 URL: https://issues.apache.org/jira/browse/SPARK-32274

[jira] [Assigned] (SPARK-32133) Forbid time field steps for date start/end in Sequence

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32133: - Assignee: JinxinTang > Forbid time field steps for date start/end in Sequence >

[jira] [Resolved] (SPARK-32133) Forbid time field steps for date start/end in Sequence

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32133. --- Resolution: Fixed Issue resolved by pull request 28926

[jira] [Commented] (SPARK-32268) Bloom Filter Join

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155580#comment-17155580 ] Apache Spark commented on SPARK-32268: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-32268) Bloom Filter Join

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155581#comment-17155581 ] Apache Spark commented on SPARK-32268: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32268) Bloom Filter Join

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32268: Assignee: Yuming Wang (was: Apache Spark) > Bloom Filter Join > - > >

[jira] [Assigned] (SPARK-32268) Bloom Filter Join

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32268: Assignee: Apache Spark (was: Yuming Wang) > Bloom Filter Join > - > >

[jira] [Resolved] (SPARK-32220) Cartesian Product Hint cause data error

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32220. --- Fix Version/s: 3.1.0 Assignee: angerszhu Resolution: Fixed This is resolved

[jira] [Updated] (SPARK-32238) Use Utils.getSimpleName to avoid hitting Malformed class name in ScalaUDF

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32238: -- Affects Version/s: 2.0.2 > Use Utils.getSimpleName to avoid hitting Malformed class name in

[jira] [Updated] (SPARK-32238) Use Utils.getSimpleName to avoid hitting Malformed class name in ScalaUDF

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32238: -- Affects Version/s: 2.1.3 > Use Utils.getSimpleName to avoid hitting Malformed class name in

[jira] [Updated] (SPARK-32238) Use Utils.getSimpleName to avoid hitting Malformed class name in ScalaUDF

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32238: -- Affects Version/s: 2.2.3 > Use Utils.getSimpleName to avoid hitting Malformed class name in

[jira] [Updated] (SPARK-32238) Use Utils.getSimpleName to avoid hitting Malformed class name in ScalaUDF

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32238: -- Affects Version/s: 2.3.4 > Use Utils.getSimpleName to avoid hitting Malformed class name in

[jira] [Updated] (SPARK-32238) Use Utils.getSimpleName to avoid hitting Malformed class name in ScalaUDF

2020-07-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32238: -- Affects Version/s: (was: 3.1.0) 2.4.6 3.0.0

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-07-10 Thread Michael Chirico (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155512#comment-17155512 ] Michael Chirico commented on SPARK-31918: - Hey folks, I just saw SparkR was removed from CRAN, I

[jira] [Resolved] (SPARK-32091) Ignore timeout error when remove blocks on the lost executor

2020-07-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32091. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28924

[jira] [Assigned] (SPARK-32091) Ignore timeout error when remove blocks on the lost executor

2020-07-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32091: --- Assignee: wuyi > Ignore timeout error when remove blocks on the lost executor >

[jira] [Commented] (SPARK-32229) Application entry parsing fails because DriverWrapper registered instead of the normal driver

2020-07-10 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155465#comment-17155465 ] Gabor Somogyi commented on SPARK-32229: --- The solution is clear but the change is depending on the

[jira] [Resolved] (SPARK-32256) Hive may fail to detect Hadoop version when using isolated classloader

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32256. -- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-32273) Support ANSI dialect in MAKE_DATE and MAKE_TIMESTAMP

2020-07-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155398#comment-17155398 ] Wenchen Fan commented on SPARK-32273: - I think there are a lot more datetime functions to be

[jira] [Assigned] (SPARK-32272) SET TIME ZONE standard sql command support

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32272: Assignee: (was: Apache Spark) > SET TIME ZONE standard sql command support >

[jira] [Assigned] (SPARK-32272) SET TIME ZONE standard sql command support

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32272: Assignee: Apache Spark > SET TIME ZONE standard sql command support >

[jira] [Commented] (SPARK-32272) SET TIME ZONE standard sql command support

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155348#comment-17155348 ] Apache Spark commented on SPARK-32272: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-32273) Support ANSI dialect in MAKE_DATE and MAKE_TIMESTAMP

2020-07-10 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155337#comment-17155337 ] Maxim Gekk commented on SPARK-32273: [~Samwel] [~cloud_fan] I haven't found an umbrella ticket for

[jira] [Created] (SPARK-32273) Support ANSI dialect in MAKE_DATE and MAKE_TIMESTAMP

2020-07-10 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-32273: -- Summary: Support ANSI dialect in MAKE_DATE and MAKE_TIMESTAMP Key: SPARK-32273 URL: https://issues.apache.org/jira/browse/SPARK-32273 Project: Spark Issue

[jira] [Created] (SPARK-32272) SET TIME ZONE standard sql command support

2020-07-10 Thread Kent Yao (Jira)
Kent Yao created SPARK-32272: Summary: SET TIME ZONE standard sql command support Key: SPARK-32272 URL: https://issues.apache.org/jira/browse/SPARK-32272 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-32271) Update CrossValidator to parallelize fit method across folds

2020-07-10 Thread Austin Jordan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Austin Jordan updated SPARK-32271: -- Description: Currently, fitting a CrossValidator is only parallelized across models. This

[jira] [Created] (SPARK-32271) Update CrossValidator to parallelize fit method across folds

2020-07-10 Thread Austin Jordan (Jira)
Austin Jordan created SPARK-32271: - Summary: Update CrossValidator to parallelize fit method across folds Key: SPARK-32271 URL: https://issues.apache.org/jira/browse/SPARK-32271 Project: Spark

[jira] [Commented] (SPARK-32270) Use text file sources in CSV's schema inference even in different encoding

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155288#comment-17155288 ] Apache Spark commented on SPARK-32270: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-32270) Use text file sources in CSV's schema inference even in different encoding

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32270: Assignee: Apache Spark > Use text file sources in CSV's schema inference even in

[jira] [Assigned] (SPARK-32270) Use text file sources in CSV's schema inference even in different encoding

2020-07-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32270: Assignee: (was: Apache Spark) > Use text file sources in CSV's schema inference even

[jira] [Updated] (SPARK-32270) Use text file sources in CSV's schema inference even in different encoding

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32270: - Summary: Use text file sources in CSV's schema inference even in different encoding (was: Use

[jira] [Updated] (SPARK-32270) Use text file sources in CSV's schema inference for consistency

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32270: - Issue Type: Improvement (was: Bug) > Use text file sources in CSV's schema inference for

[jira] [Updated] (SPARK-32270) Use text file sources in CSV's schema inference for consistency

2020-07-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32270: - Issue Type: Bug (was: Improvement) > Use text file sources in CSV's schema inference for

[jira] [Created] (SPARK-32270) Use text file sources in CSV's schema inference for consistency

2020-07-10 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32270: Summary: Use text file sources in CSV's schema inference for consistency Key: SPARK-32270 URL: https://issues.apache.org/jira/browse/SPARK-32270 Project: Spark

[jira] [Updated] (SPARK-32269) Failed to rename delta file on checkpoint path

2020-07-10 Thread Elvis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elvis updated SPARK-32269: -- Description: Hi Team,  I got an exception that : {code:java} // code placeholder Aborting task

[jira] [Created] (SPARK-32269) Failed to rename delta file on checkpoint path

2020-07-10 Thread Elvis (Jira)
Elvis created SPARK-32269: - Summary: Failed to rename delta file on checkpoint path Key: SPARK-32269 URL: https://issues.apache.org/jira/browse/SPARK-32269 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-32250) Reenable MasterSuite's "Master should avoid dead loop while launching executor failed in Worker"

2020-07-10 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155201#comment-17155201 ] wuyi commented on SPARK-32250: -- [~hyukjin.kwon]Thanks for ping me. I'll take a look later. > Reenable

[jira] [Updated] (SPARK-32268) Bloom Filter Join

2020-07-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32268: Description: We can improve the performance of some joins by pre-filtering one side of a join

[jira] [Updated] (SPARK-32268) Bloom Filter Join

2020-07-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32268: Description: We can improve the performance of some joins by pre-filtering one side of a join

[jira] [Updated] (SPARK-32268) Bloom Filter Join

2020-07-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32268: Description: We can improve the performance of some joins by pre-filtering one side of a join

[jira] [Updated] (SPARK-32268) Bloom Filter Join

2020-07-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32268: Attachment: (was: q16-default.png) > Bloom Filter Join > - > >

  1   2   >