[jira] [Commented] (SPARK-34007) Downgrade scala-maven-plugin to 4.3.0

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258736#comment-17258736 ] Apache Spark commented on SPARK-34007: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-34007) Downgrade scala-maven-plugin to 4.3.0

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34007: Assignee: Apache Spark > Downgrade scala-maven-plugin to 4.3.0 >

[jira] [Assigned] (SPARK-34007) Downgrade scala-maven-plugin to 4.3.0

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34007: Assignee: (was: Apache Spark) > Downgrade scala-maven-plugin to 4.3.0 >

[jira] [Commented] (SPARK-33948) branch-3.1 jenkins test failed in Scala 2.13

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258734#comment-17258734 ] Wenchen Fan commented on SPARK-33948: - SPARK-33619 improved the codegen test coverage of Spark

[jira] [Commented] (SPARK-34006) [spark.sql.hive.convertMetastoreOrc]This parameter can solve orc format table insert overwrite read table, it should be stated in the document

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258732#comment-17258732 ] Apache Spark commented on SPARK-34006: -- User 'dh20' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34006) [spark.sql.hive.convertMetastoreOrc]This parameter can solve orc format table insert overwrite read table, it should be stated in the document

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34006: Assignee: Apache Spark > [spark.sql.hive.convertMetastoreOrc]This parameter can solve

[jira] [Commented] (SPARK-34006) [spark.sql.hive.convertMetastoreOrc]This parameter can solve orc format table insert overwrite read table, it should be stated in the document

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258733#comment-17258733 ] Apache Spark commented on SPARK-34006: -- User 'dh20' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34006) [spark.sql.hive.convertMetastoreOrc]This parameter can solve orc format table insert overwrite read table, it should be stated in the document

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34006: Assignee: (was: Apache Spark) > [spark.sql.hive.convertMetastoreOrc]This parameter

[jira] [Updated] (SPARK-34007) Downgrade scala-maven-plugin to 4.3.0

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34007: - Target Version/s: 3.1.0 > Downgrade scala-maven-plugin to 4.3.0 >

[jira] [Updated] (SPARK-33980) invalidate char/varchar in spark.readStream.schema

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33980: - Fix Version/s: (was: 3.1.1) 3.1.0 > invalidate char/varchar in

[jira] [Assigned] (SPARK-34005) Update peak memory metrics for each Executor on task end.

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34005: Assignee: Kousuke Saruta (was: Apache Spark) > Update peak memory metrics for each

[jira] [Commented] (SPARK-34005) Update peak memory metrics for each Executor on task end.

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258727#comment-17258727 ] Apache Spark commented on SPARK-34005: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-33833) Allow Spark Structured Streaming report Kafka Lag through Burrow

2021-01-04 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258726#comment-17258726 ] L. C. Hsieh commented on SPARK-33833: - Yea, but this can be easily overcome here. We just need to

[jira] [Assigned] (SPARK-34005) Update peak memory metrics for each Executor on task end.

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34005: Assignee: Apache Spark (was: Kousuke Saruta) > Update peak memory metrics for each

[jira] [Updated] (SPARK-33992) resolveOperatorsUpWithNewOutput should wrap allowInvokingTransformsInAnalyzer

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33992: - Fix Version/s: (was: 3.1.1) 3.1.0 > resolveOperatorsUpWithNewOutput

[jira] [Updated] (SPARK-33894) Word2VecSuite failed for Scala 2.13

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33894: - Fix Version/s: (was: 3.1.1) 3.1.0 > Word2VecSuite failed for Scala 2.13

[jira] [Updated] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34000: - Fix Version/s: (was: 3.1.1) 3.1.0 > ExecutorAllocationListener threw an

[jira] [Commented] (SPARK-33950) ALTER TABLE .. DROP PARTITION doesn't refresh cache

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258725#comment-17258725 ] Hyukjin Kwon commented on SPARK-33950: -- I need to recreate rc1 tag. I failed to create a RC due to

[jira] [Comment Edited] (SPARK-33980) invalidate char/varchar in spark.readStream.schema

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258724#comment-17258724 ] Hyukjin Kwon edited comment on SPARK-33980 at 1/5/21, 7:43 AM: --- I need to

[jira] [Updated] (SPARK-33950) ALTER TABLE .. DROP PARTITION doesn't refresh cache

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33950: - Fix Version/s: (was: 3.1.1) 3.1.0 > ALTER TABLE .. DROP PARTITION

[jira] [Commented] (SPARK-33980) invalidate char/varchar in spark.readStream.schema

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258724#comment-17258724 ] Hyukjin Kwon commented on SPARK-33980: -- I need to recreate rc1 tag. I failed to create a RC due to

[jira] [Created] (SPARK-34007) Downgrade scala-maven-plugin to 4.3.0

2021-01-04 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34007: Summary: Downgrade scala-maven-plugin to 4.3.0 Key: SPARK-34007 URL: https://issues.apache.org/jira/browse/SPARK-34007 Project: Spark Issue Type:

[jira] [Created] (SPARK-34006) [spark.sql.hive.convertMetastoreOrc]This parameter can solve orc format table insert overwrite read table, it should be stated in the document

2021-01-04 Thread hao (Jira)
hao created SPARK-34006: --- Summary: [spark.sql.hive.convertMetastoreOrc]This parameter can solve orc format table insert overwrite read table, it should be stated in the document Key: SPARK-34006 URL:

[jira] [Updated] (SPARK-33989) Strip auto-generated cast when using Cast.sql

2021-01-04 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33989: Summary: Strip auto-generated cast when using Cast.sql (was: Strip auto-generated cast when

[jira] [Assigned] (SPARK-34003) Rule conflicts between PaddingAndLengthCheckForCharVarchar and ResolveAggregateFunctions

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34003: Assignee: (was: Apache Spark) > Rule conflicts between

[jira] [Assigned] (SPARK-34003) Rule conflicts between PaddingAndLengthCheckForCharVarchar and ResolveAggregateFunctions

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34003: Assignee: Apache Spark > Rule conflicts between PaddingAndLengthCheckForCharVarchar and

[jira] [Assigned] (SPARK-34004) Change FrameLessOffsetWindowFunction as sealed abstract class

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34004: Assignee: Apache Spark > Change FrameLessOffsetWindowFunction as sealed abstract class >

[jira] [Commented] (SPARK-34004) Change FrameLessOffsetWindowFunction as sealed abstract class

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258722#comment-17258722 ] Apache Spark commented on SPARK-34004: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34004) Change FrameLessOffsetWindowFunction as sealed abstract class

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34004: Assignee: (was: Apache Spark) > Change FrameLessOffsetWindowFunction as sealed

[jira] [Commented] (SPARK-34003) Rule conflicts between PaddingAndLengthCheckForCharVarchar and ResolveAggregateFunctions

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258723#comment-17258723 ] Apache Spark commented on SPARK-34003: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-32017) Make Pyspark Hadoop 3.2+ Variant available in PyPI

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258721#comment-17258721 ] Apache Spark commented on SPARK-32017: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-34005) Update peak memory metrics for each Executor on task end.

2021-01-04 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34005: -- Summary: Update peak memory metrics for each Executor on task end. Key: SPARK-34005 URL: https://issues.apache.org/jira/browse/SPARK-34005 Project: Spark

[jira] [Updated] (SPARK-34005) Update peak memory metrics for each Executor on task end.

2021-01-04 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34005: --- Issue Type: Improvement (was: Bug) > Update peak memory metrics for each Executor on task

[jira] [Resolved] (SPARK-33919) Unify v1 and v2 SHOW NAMESPACES tests

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33919. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30937

[jira] [Assigned] (SPARK-33919) Unify v1 and v2 SHOW NAMESPACES tests

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33919: --- Assignee: Maxim Gekk > Unify v1 and v2 SHOW NAMESPACES tests >

[jira] [Commented] (SPARK-33995) Make datetime addition easier for years, weeks, hours, minutes, and seconds

2021-01-04 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258718#comment-17258718 ] Maxim Gekk commented on SPARK-33995: > Option 1: Single make_interval function that takes 7

[jira] [Commented] (SPARK-33833) Allow Spark Structured Streaming report Kafka Lag through Burrow

2021-01-04 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258717#comment-17258717 ] Jungtaek Lim commented on SPARK-33833: -- That’s available with serious caution. Spark has to have

[jira] [Commented] (SPARK-33833) Allow Spark Structured Streaming report Kafka Lag through Burrow

2021-01-04 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258716#comment-17258716 ] L. C. Hsieh commented on SPARK-33833: - I read though the comments in the previous PR. The approach

[jira] [Commented] (SPARK-33833) Allow Spark Structured Streaming report Kafka Lag through Burrow

2021-01-04 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258707#comment-17258707 ] L. C. Hsieh commented on SPARK-33833: - Btw, thanks for providing the useful link to previous

[jira] [Created] (SPARK-34004) Change FrameLessOffsetWindowFunction as sealed abstract class

2021-01-04 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-34004: -- Summary: Change FrameLessOffsetWindowFunction as sealed abstract class Key: SPARK-34004 URL: https://issues.apache.org/jira/browse/SPARK-34004 Project: Spark

[jira] [Resolved] (SPARK-33935) Fix CBOs cost function

2021-01-04 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-33935. -- Fix Version/s: 3.2.0 3.1.0 Assignee: Tanel Kiis

[jira] [Commented] (SPARK-33833) Allow Spark Structured Streaming report Kafka Lag through Burrow

2021-01-04 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258705#comment-17258705 ] L. C. Hsieh commented on SPARK-33833: - I think SS allows users to specify custom group id, isn't it?

[jira] [Created] (SPARK-34003) Rule conflicts between PaddingAndLengthCheckForCharVarchar and ResolveAggregateFunctions

2021-01-04 Thread Kent Yao (Jira)
Kent Yao created SPARK-34003: Summary: Rule conflicts between PaddingAndLengthCheckForCharVarchar and ResolveAggregateFunctions Key: SPARK-34003 URL: https://issues.apache.org/jira/browse/SPARK-34003

[jira] [Resolved] (SPARK-33100) Support parse the sql statements with c-style comments

2021-01-04 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-33100. -- Fix Version/s: 3.2.0 3.1.0 Assignee: feiwang (was: Apache

[jira] [Commented] (SPARK-33833) Allow Spark Structured Streaming report Kafka Lag through Burrow

2021-01-04 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258695#comment-17258695 ] Jungtaek Lim commented on SPARK-33833: -- For SS, consumer group is randomly generated by intention,

[jira] [Comment Edited] (SPARK-33833) Allow Spark Structured Streaming report Kafka Lag through Burrow

2021-01-04 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258692#comment-17258692 ] L. C. Hsieh edited comment on SPARK-33833 at 1/5/21, 6:27 AM: -- Hmm, I did a

[jira] [Commented] (SPARK-33833) Allow Spark Structured Streaming report Kafka Lag through Burrow

2021-01-04 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258693#comment-17258693 ] L. C. Hsieh commented on SPARK-33833: - [~samdvr] Can you help elaborate the question above? Thanks.

[jira] [Commented] (SPARK-33833) Allow Spark Structured Streaming report Kafka Lag through Burrow

2021-01-04 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258692#comment-17258692 ] L. C. Hsieh commented on SPARK-33833: - Hmm, I did a few test locally. Does Burrow work only if Spark

[jira] [Updated] (SPARK-34002) Broken UDF Encoding

2021-01-04 Thread Mark Hamilton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamilton updated SPARK-34002: -- Description: UDFs can behave differently depending on if a dataframe is cached, despite the

[jira] [Updated] (SPARK-34002) Broken UDF Encoding

2021-01-04 Thread Mark Hamilton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamilton updated SPARK-34002: -- Description: UDFs can behave differently depending on if a dataframe is cached, despite the

[jira] [Updated] (SPARK-34002) Broken UDF Encoding

2021-01-04 Thread Mark Hamilton (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Hamilton updated SPARK-34002: -- Description: UDFs can behave differently depending on if a dataframe is cached, despite the

[jira] [Updated] (SPARK-32085) Migrate to NumPy documentation style

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32085: - Fix Version/s: 3.1.0 > Migrate to NumPy documentation style >

[jira] [Resolved] (SPARK-32085) Migrate to NumPy documentation style

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32085. -- Resolution: Done > Migrate to NumPy documentation style >

[jira] [Created] (SPARK-34002) Broken UDF behavior

2021-01-04 Thread Mark Hamilton (Jira)
Mark Hamilton created SPARK-34002: - Summary: Broken UDF behavior Key: SPARK-34002 URL: https://issues.apache.org/jira/browse/SPARK-34002 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-33242) Install numpydoc in Jenkins machines

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33242: - Parent: (was: SPARK-32085) Issue Type: Test (was: Sub-task) > Install numpydoc in

[jira] [Updated] (SPARK-33992) resolveOperatorsUpWithNewOutput should wrap allowInvokingTransformsInAnalyzer

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33992: -- Fix Version/s: (was: 3.1.0) 3.1.1 > resolveOperatorsUpWithNewOutput

[jira] [Updated] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34000: -- Fix Version/s: (was: 3.1.0) 3.1.1 > ExecutorAllocationListener threw

[jira] [Assigned] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34000: - Assignee: Lantao Jin > ExecutorAllocationListener threw an exception

[jira] [Resolved] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34000. --- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-33992) resolveOperatorsUpWithNewOutput should wrap allowInvokingTransformsInAnalyzer

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33992: --- Assignee: Kent Yao > resolveOperatorsUpWithNewOutput should wrap

[jira] [Commented] (SPARK-34001) Remove unused runShowTablesSql() in DataSourceV2SQLSuite.scala

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258676#comment-17258676 ] Apache Spark commented on SPARK-34001: -- User 'imback82' has created a pull request for this issue:

[jira] [Resolved] (SPARK-33992) resolveOperatorsUpWithNewOutput should wrap allowInvokingTransformsInAnalyzer

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33992. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 31013

[jira] [Commented] (SPARK-34001) Remove unused runShowTablesSql() in DataSourceV2SQLSuite.scala

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258675#comment-17258675 ] Apache Spark commented on SPARK-34001: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34001) Remove unused runShowTablesSql() in DataSourceV2SQLSuite.scala

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34001: - Assignee: Terry Kim > Remove unused runShowTablesSql() in DataSourceV2SQLSuite.scala >

[jira] [Resolved] (SPARK-34001) Remove unused runShowTablesSql() in DataSourceV2SQLSuite.scala

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34001. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31022

[jira] [Resolved] (SPARK-33998) Refactor v2CommandExec to provide an API to create an InternalRow

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33998. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31020

[jira] [Assigned] (SPARK-33998) Refactor v2CommandExec to provide an API to create an InternalRow

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33998: --- Assignee: Terry Kim > Refactor v2CommandExec to provide an API to create an InternalRow >

[jira] [Updated] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-34000: --- Affects Version/s: 3.0.1 > ExecutorAllocationListener threw an exception

[jira] [Created] (SPARK-34001) Remove unused runShowTablesSql() in DataSourceV2SQLSuite.scala

2021-01-04 Thread Terry Kim (Jira)
Terry Kim created SPARK-34001: - Summary: Remove unused runShowTablesSql() in DataSourceV2SQLSuite.scala Key: SPARK-34001 URL: https://issues.apache.org/jira/browse/SPARK-34001 Project: Spark

[jira] [Updated] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-34000: --- Affects Version/s: (was: 3.0.1) > ExecutorAllocationListener threw an exception

[jira] [Comment Edited] (SPARK-31786) Exception on submitting Spark-Pi to Kubernetes 1.17.3

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258670#comment-17258670 ] Dongjoon Hyun edited comment on SPARK-31786 at 1/5/21, 5:24 AM: Yes, you

[jira] [Commented] (SPARK-31786) Exception on submitting Spark-Pi to Kubernetes 1.17.3

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258670#comment-17258670 ] Dongjoon Hyun commented on SPARK-31786: --- Yes, you are correct. # `export` is only required for

[jira] [Resolved] (SPARK-33794) next_day function should throw runtime exception when receiving invalid input under ANSI mode

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33794. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30807

[jira] [Assigned] (SPARK-33794) next_day function should throw runtime exception when receiving invalid input under ANSI mode

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33794: --- Assignee: Chongguang LIU > next_day function should throw runtime exception when receiving

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258667#comment-17258667 ] Dongjoon Hyun commented on SPARK-25075: --- [~smarter]. Sorry, but unfortunately, from my assesement,

[jira] [Updated] (SPARK-33950) ALTER TABLE .. DROP PARTITION doesn't refresh cache

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33950: -- Fix Version/s: (was: 3.1.0) 3.1.1 > ALTER TABLE .. DROP PARTITION

[jira] [Updated] (SPARK-33894) Word2VecSuite failed for Scala 2.13

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33894: -- Fix Version/s: (was: 3.1.0) 3.1.1 > Word2VecSuite failed for Scala

[jira] [Updated] (SPARK-33980) invalidate char/varchar in spark.readStream.schema

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33980: -- Fix Version/s: (was: 3.1.0) 3.1.1 > invalidate char/varchar in

[jira] [Assigned] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34000: Assignee: (was: Apache Spark) > ExecutorAllocationListener threw an exception

[jira] [Assigned] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34000: Assignee: Apache Spark > ExecutorAllocationListener threw an exception

[jira] [Commented] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258640#comment-17258640 ] Apache Spark commented on SPARK-34000: -- User 'LantaoJin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33979) Filter predicate reorder

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33979: Assignee: Apache Spark > Filter predicate reorder > > >

[jira] [Assigned] (SPARK-33979) Filter predicate reorder

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33979: Assignee: (was: Apache Spark) > Filter predicate reorder >

[jira] [Commented] (SPARK-33979) Filter predicate reorder

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258639#comment-17258639 ] Apache Spark commented on SPARK-33979: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-34000) ExecutorAllocationListener threw an exception java.util.NoSuchElementException

2021-01-04 Thread Lantao Jin (Jira)
Lantao Jin created SPARK-34000: -- Summary: ExecutorAllocationListener threw an exception java.util.NoSuchElementException Key: SPARK-34000 URL: https://issues.apache.org/jira/browse/SPARK-34000 Project:

[jira] [Commented] (SPARK-33999) Make sbt unidoc success with JDK11

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258634#comment-17258634 ] Apache Spark commented on SPARK-33999: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33999) Make sbt unidoc success with JDK11

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33999: Assignee: Kousuke Saruta (was: Apache Spark) > Make sbt unidoc success with JDK11 >

[jira] [Assigned] (SPARK-33999) Make sbt unidoc success with JDK11

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33999: Assignee: Apache Spark (was: Kousuke Saruta) > Make sbt unidoc success with JDK11 >

[jira] [Created] (SPARK-33999) Make sbt unidoc success with JDK11

2021-01-04 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-33999: -- Summary: Make sbt unidoc success with JDK11 Key: SPARK-33999 URL: https://issues.apache.org/jira/browse/SPARK-33999 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-16087) Spark Hangs When Using Union With Persisted Hadoop RDD

2021-01-04 Thread Rob Russo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Russo updated SPARK-16087: -- Affects Version/s: 3.0.1 > Spark Hangs When Using Union With Persisted Hadoop RDD >

[jira] [Commented] (SPARK-16087) Spark Hangs When Using Union With Persisted Hadoop RDD

2021-01-04 Thread Rob Russo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258627#comment-17258627 ] Rob Russo commented on SPARK-16087: --- I know this ticket is old now but spark 3 seems to have

[jira] [Reopened] (SPARK-16087) Spark Hangs When Using Union With Persisted Hadoop RDD

2021-01-04 Thread Rob Russo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Russo reopened SPARK-16087: --- Reopening as it occurred for us only after upgrading to spark 3.x > Spark Hangs When Using Union With

[jira] [Resolved] (SPARK-33964) Combine distinct unions in more cases

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33964. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30996

[jira] [Assigned] (SPARK-33964) Combine distinct unions in more cases

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33964: Assignee: Tanel Kiis > Combine distinct unions in more cases >

[jira] [Commented] (SPARK-33382) Unify v1 and v2 SHOW TABLES tests

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258617#comment-17258617 ] Apache Spark commented on SPARK-33382: -- User 'imback82' has created a pull request for this issue:

[jira] [Resolved] (SPARK-33997) Running bin/spark-sql gives NoSuchMethodError

2021-01-04 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-33997. Resolution: Cannot Reproduce Rebuilt Spark locally and the error was gone. > Running bin/spark-sql gives

[jira] [Commented] (SPARK-33998) Refactor v2CommandExec to provide an API to create an InternalRow

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258606#comment-17258606 ] Apache Spark commented on SPARK-33998: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33998) Refactor v2CommandExec to provide an API to create an InternalRow

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33998: Assignee: (was: Apache Spark) > Refactor v2CommandExec to provide an API to create

[jira] [Commented] (SPARK-33998) Refactor v2CommandExec to provide an API to create an InternalRow

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258605#comment-17258605 ] Apache Spark commented on SPARK-33998: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33998) Refactor v2CommandExec to provide an API to create an InternalRow

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33998: Assignee: Apache Spark > Refactor v2CommandExec to provide an API to create an

  1   2   3   >