[jira] [Commented] (SPARK-33935) Fix CBOs cost function

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312908#comment-17312908 ] Apache Spark commented on SPARK-33935: -- User 'viirya' has created a pull request fo

[jira] [Updated] (SPARK-34799) Return UDT from Pandas UDF: @pandas_udf(UDT)

2021-03-31 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darcy Shen updated SPARK-34799: --- Summary: Return UDT from Pandas UDF: @pandas_udf(UDT) (was: Return User-defined types from Pandas U

[jira] [Updated] (SPARK-34799) Return UDT from Pandas UDF: @pandas_udf(UDT)

2021-03-31 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darcy Shen updated SPARK-34799: --- Description: Focus on a simpler case first: `@pandas_udf(UDT)`. Because Pandas UDF uses pyarrow

[jira] [Commented] (SPARK-34881) New SQL Function: TRY_CAST

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312853#comment-17312853 ] Apache Spark commented on SPARK-34881: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-34881) New SQL Function: TRY_CAST

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312852#comment-17312852 ] Apache Spark commented on SPARK-34881: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

2021-03-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312851#comment-17312851 ] Attila Zsolt Piros commented on SPARK-34738: Shane, as Docker is available o

[jira] [Created] (SPARK-34930) Install PyArrow and pandas on Jenkins

2021-03-31 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34930: Summary: Install PyArrow and pandas on Jenkins Key: SPARK-34930 URL: https://issues.apache.org/jira/browse/SPARK-34930 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-34354) CostBasedJoinReorder can fail on self-join

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34354: Assignee: (was: Apache Spark) > CostBasedJoinReorder can fail on self-join >

[jira] [Assigned] (SPARK-34354) CostBasedJoinReorder can fail on self-join

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34354: Assignee: Apache Spark > CostBasedJoinReorder can fail on self-join > ---

[jira] [Reopened] (SPARK-34354) CostBasedJoinReorder can fail on self-join

2021-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-34354: -- Assignee: (was: wuyi) Reverted in https://github.com/apache/spark/commit/cc451c16a3d28b

[jira] [Updated] (SPARK-34354) CostBasedJoinReorder can fail on self-join

2021-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34354: - Fix Version/s: (was: 3.2.0) > CostBasedJoinReorder can fail on self-join > -

[jira] [Updated] (SPARK-34929) MapStatusesSerDeserBenchmark causes a side effect to other benchmarks with tasks being too big (JDK 11)

2021-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34929: - Description: In JDK 11, MapStatusesSerDeserBenchmark (being started failed) seems affecting oth

[jira] [Created] (SPARK-34929) MapStatusesSerDeserBenchmark causes a side effect to other benchmarks with tasks being too big (JDK 11)

2021-03-31 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34929: Summary: MapStatusesSerDeserBenchmark causes a side effect to other benchmarks with tasks being too big (JDK 11) Key: SPARK-34929 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Updated] (SPARK-34928) CTE Execution fails for Sql Server

2021-03-31 Thread Supun De Silva (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Supun De Silva updated SPARK-34928: --- Description: h2. Issue We have a simple Sql statement that we intend to execute on SQL Serv

[jira] [Updated] (SPARK-34928) CTE Execution fails for Sql Server

2021-03-31 Thread Supun De Silva (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Supun De Silva updated SPARK-34928: --- Priority: Minor (was: Major) > CTE Execution fails for Sql Server > ---

[jira] [Updated] (SPARK-34928) CTE Execution fails for Sql Server

2021-03-31 Thread Supun De Silva (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Supun De Silva updated SPARK-34928: --- Description: h2. Issue We have a simple Sql statement that we intend to execute on SQL Serv

[jira] [Updated] (SPARK-34928) CTE Execution fails for Sql Server

2021-03-31 Thread Supun De Silva (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Supun De Silva updated SPARK-34928: --- Description: h2. Issue We have a simple Sql statement that we intend to execute on SQL Serv

[jira] [Created] (SPARK-34928) CTE Execution fails for Sql Server

2021-03-31 Thread Supun De Silva (Jira)
Supun De Silva created SPARK-34928: -- Summary: CTE Execution fails for Sql Server Key: SPARK-34928 URL: https://issues.apache.org/jira/browse/SPARK-34928 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-34927) Support TPCDSQueryBenchmark in Benchmarks

2021-03-31 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34927: Summary: Support TPCDSQueryBenchmark in Benchmarks Key: SPARK-34927 URL: https://issues.apache.org/jira/browse/SPARK-34927 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-34927) Support TPCDSQueryBenchmark in Benchmarks

2021-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34927: - Priority: Minor (was: Major) > Support TPCDSQueryBenchmark in Benchmarks >

[jira] [Commented] (SPARK-34926) PartitionUtils.getPathFragment should handle null value

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312816#comment-17312816 ] Apache Spark commented on SPARK-34926: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-34926) PartitionUtils.getPathFragment should handle null value

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34926: Assignee: Apache Spark > PartitionUtils.getPathFragment should handle null value > --

[jira] [Assigned] (SPARK-34926) PartitionUtils.getPathFragment should handle null value

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34926: Assignee: (was: Apache Spark) > PartitionUtils.getPathFragment should handle null val

[jira] [Commented] (SPARK-34926) PartitionUtils.getPathFragment should handle null value

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312815#comment-17312815 ] Apache Spark commented on SPARK-34926: -- User 'AngersZh' has created a pull requ

[jira] [Created] (SPARK-34926) PartitionUtils.getPathFragment should handle null value

2021-03-31 Thread angerszhu (Jira)
angerszhu created SPARK-34926: - Summary: PartitionUtils.getPathFragment should handle null value Key: SPARK-34926 URL: https://issues.apache.org/jira/browse/SPARK-34926 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34925) Spark shell failed to access external file to read content

2021-03-31 Thread czh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] czh updated SPARK-34925: Attachment: 微信截图_20210401093951.png > Spark shell failed to access external file to read content > ---

[jira] [Created] (SPARK-34925) Spark shell failed to access external file to read content

2021-03-31 Thread czh (Jira)
czh created SPARK-34925: --- Summary: Spark shell failed to access external file to read content Key: SPARK-34925 URL: https://issues.apache.org/jira/browse/SPARK-34925 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-23977) Add commit protocol binding to Hadoop 3.1 PathOutputCommitter mechanism

2021-03-31 Thread Daniel Zhi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311818#comment-17311818 ] Daniel Zhi edited comment on SPARK-23977 at 3/31/21, 11:59 PM: ---

[jira] [Comment Edited] (SPARK-23977) Add commit protocol binding to Hadoop 3.1 PathOutputCommitter mechanism

2021-03-31 Thread Daniel Zhi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311818#comment-17311818 ] Daniel Zhi edited comment on SPARK-23977 at 3/31/21, 11:57 PM: ---

[jira] [Comment Edited] (SPARK-34910) JDBC - Add an option for different stride orders

2021-03-31 Thread Jason Yarbrough (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312702#comment-17312702 ] Jason Yarbrough edited comment on SPARK-34910 at 3/31/21, 11:51 PM: --

[jira] [Resolved] (SPARK-34882) RewriteDistinctAggregates can cause a bug if the aggregator does not ignore NULLs

2021-03-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-34882. -- Fix Version/s: 3.2.0 Assignee: Tanel Kiis Resolution: Fixed Resolved b

[jira] [Commented] (SPARK-34910) JDBC - Add an option for different stride orders

2021-03-31 Thread Jason Yarbrough (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312749#comment-17312749 ] Jason Yarbrough commented on SPARK-34910: - It's not that the records/rows themse

[jira] [Commented] (SPARK-34923) Metadata output should not always be propagated

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312743#comment-17312743 ] Apache Spark commented on SPARK-34923: -- User 'karenfeng' has created a pull request

[jira] [Assigned] (SPARK-34923) Metadata output should not always be propagated

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34923: Assignee: Apache Spark > Metadata output should not always be propagated > --

[jira] [Assigned] (SPARK-34923) Metadata output should not always be propagated

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34923: Assignee: (was: Apache Spark) > Metadata output should not always be propagated > ---

[jira] [Commented] (SPARK-34910) JDBC - Add an option for different stride orders

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312708#comment-17312708 ] Sean R. Owen commented on SPARK-34910: -- In this example, how are the rows 'skewed'?

[jira] [Commented] (SPARK-34910) JDBC - Add an option for different stride orders

2021-03-31 Thread Jason Yarbrough (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312702#comment-17312702 ] Jason Yarbrough commented on SPARK-34910: - Hi [~srowen], thanks for asking. The

[jira] [Comment Edited] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

2021-03-31 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312696#comment-17312696 ] Shane Knapp edited comment on SPARK-34738 at 3/31/21, 8:38 PM: ---

[jira] [Created] (SPARK-34924) Add nested column scan in DataSourceReadBenchmark

2021-03-31 Thread Cheng Su (Jira)
Cheng Su created SPARK-34924: Summary: Add nested column scan in DataSourceReadBenchmark Key: SPARK-34924 URL: https://issues.apache.org/jira/browse/SPARK-34924 Project: Spark Issue Type: Sub-tas

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

2021-03-31 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312696#comment-17312696 ] Shane Knapp commented on SPARK-34738: - managed to snag the logs from the pod when it

[jira] [Updated] (SPARK-33638) Full support of V2 table creation in Structured Streaming writer path

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-33638: - Priority: Major (was: Blocker) > Full support of V2 table creation in Structured Streaming writ

[jira] [Resolved] (SPARK-33694) Auditing the API changes and behavior changes in Spark 3.1

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-33694. -- Resolution: Done > Auditing the API changes and behavior changes in Spark 3.1 > --

[jira] [Updated] (SPARK-34425) Improve error message in default spark session catalog

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-34425: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Blocker) Feel free to revert th

[jira] [Created] (SPARK-34923) Metadata output should not always be propagated

2021-03-31 Thread Karen Feng (Jira)
Karen Feng created SPARK-34923: -- Summary: Metadata output should not always be propagated Key: SPARK-34923 URL: https://issues.apache.org/jira/browse/SPARK-34923 Project: Spark Issue Type: Impro

[jira] [Resolved] (SPARK-34904) Old import of LZ4 package inside CompressionCodec.scala

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34904. -- Resolution: Not A Problem > Old import of LZ4 package inside CompressionCodec.scala > ---

[jira] [Assigned] (SPARK-34903) Return day-time interval from timestamps subtraction

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34903: Assignee: Max Gekk (was: Apache Spark) > Return day-time interval from timestamps subtra

[jira] [Commented] (SPARK-34903) Return day-time interval from timestamps subtraction

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312568#comment-17312568 ] Apache Spark commented on SPARK-34903: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-34903) Return day-time interval from timestamps subtraction

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312569#comment-17312569 ] Apache Spark commented on SPARK-34903: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-34903) Return day-time interval from timestamps subtraction

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34903: Assignee: Apache Spark (was: Max Gekk) > Return day-time interval from timestamps subtra

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

2021-03-31 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312548#comment-17312548 ] Shane Knapp commented on SPARK-34738: - fwiw, the pod is mounting the local FS correc

[jira] [Commented] (SPARK-34904) Old import of LZ4 package inside CompressionCodec.scala

2021-03-31 Thread Michal Zeman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312547#comment-17312547 ] Michal Zeman commented on SPARK-34904: -- [~srowen] Nope, sorry embarrassing. I have

[jira] [Updated] (SPARK-34904) Old import of LZ4 package inside CompressionCodec.scala

2021-03-31 Thread Michal Zeman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michal Zeman updated SPARK-34904: - Issue Type: Question (was: Bug) > Old import of LZ4 package inside CompressionCodec.scala > --

[jira] [Comment Edited] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

2021-03-31 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312517#comment-17312517 ] Shane Knapp edited comment on SPARK-34738 at 3/31/21, 4:43 PM: ---

[jira] [Comment Edited] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

2021-03-31 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312517#comment-17312517 ] Shane Knapp edited comment on SPARK-34738 at 3/31/21, 4:20 PM: ---

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

2021-03-31 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312518#comment-17312518 ] Shane Knapp commented on SPARK-34738: - [~skonto] any insights? > Upgrade Minikube a

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes cluster version on Jenkins

2021-03-31 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312517#comment-17312517 ] Shane Knapp commented on SPARK-34738: - alright, sometimes these things go smoothly,

[jira] [Commented] (SPARK-34883) Setting CSV reader option "multiLine" to "true" causes URISyntaxException when colon is in file path

2021-03-31 Thread Brady Tello (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312504#comment-17312504 ] Brady Tello commented on SPARK-34883: - Sure thing.  This Scala code results in the J

[jira] [Commented] (SPARK-34779) ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

2021-03-31 Thread Baohe Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312498#comment-17312498 ] Baohe Zhang commented on SPARK-34779: - Thanks for pointing it out! I didn't aware th

[jira] [Resolved] (SPARK-34684) Hadoop config could not be successfully serilized from driver pods to executor pods

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34684. -- Resolution: Not A Problem > Hadoop config could not be successfully serilized from driver pods

[jira] [Resolved] (SPARK-34751) Parquet with invalid chars on column name reads double as null when a clean schema is applied

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34751. -- Resolution: Duplicate > Parquet with invalid chars on column name reads double as null when a

[jira] [Resolved] (SPARK-34865) cannot resolve methods when & col in spark java

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34865. -- Resolution: Duplicate > cannot resolve methods when & col in spark java >

[jira] [Resolved] (SPARK-34864) cannot resolve methods when & col in spark java

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34864. -- Resolution: Invalid > cannot resolve methods when & col in spark java > --

[jira] [Resolved] (SPARK-34866) cannot resolve method when of Column() spark java

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34866. -- Resolution: Invalid > cannot resolve method when of Column() spark java >

[jira] [Resolved] (SPARK-34867) cannot resolve method when of Column() spark java

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34867. -- Resolution: Duplicate > cannot resolve method when of Column() spark java > --

[jira] [Commented] (SPARK-34883) Setting CSV reader option "multiLine" to "true" causes URISyntaxException when colon is in file path

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312473#comment-17312473 ] Sean R. Owen commented on SPARK-34883: -- I'm guessing it's somehow that a different

[jira] [Commented] (SPARK-34904) Old import of LZ4 package inside CompressionCodec.scala

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312471#comment-17312471 ] Sean R. Owen commented on SPARK-34904: -- [~langustav] I don't see any code in the or

[jira] [Updated] (SPARK-34904) Old import of LZ4 package inside CompressionCodec.scala

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-34904: - Priority: Minor (was: Major) > Old import of LZ4 package inside CompressionCodec.scala > -

[jira] [Commented] (SPARK-34910) JDBC - Add an option for different stride orders

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312466#comment-17312466 ] Sean R. Owen commented on SPARK-34910: -- Out of curiosity, when is that advantageous

[jira] [Resolved] (SPARK-34912) Start spark shell to read file and report an error

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34912. -- Resolution: Invalid Very old Spark + something wrong with your config of AWS libs > Start spa

[jira] [Resolved] (SPARK-34913) Start spark shell to read file and report an error

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34913. -- Resolution: Duplicate > Start spark shell to read file and report an error > -

[jira] [Resolved] (SPARK-34921) Error in spark accessing file

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34921. -- Resolution: Invalid Not nearly enough info > Error in spark accessing file >

[jira] [Commented] (SPARK-34510) .foreachPartition command hangs when ran inside Python package but works when ran from Python file outside the package on EMR

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312460#comment-17312460 ] Sean R. Owen commented on SPARK-34510: -- Got it. Without more detail or errors, or a

[jira] [Resolved] (SPARK-34257) Improve performance for last_value over unbounded window frame

2021-03-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34257. -- Resolution: Won't Fix > Improve performance for last_value over unbounded window frame > -

[jira] [Commented] (SPARK-34806) Helper class for batch Dataset.observe()

2021-03-31 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312403#comment-17312403 ] Enrico Minack commented on SPARK-34806: --- The interaction with {{Observation}} can

[jira] [Assigned] (SPARK-34821) Set up a workflow for developers to run benchmark in their fork

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34821: Assignee: (was: Apache Spark) > Set up a workflow for developers to run benchmark in

[jira] [Commented] (SPARK-34821) Set up a workflow for developers to run benchmark in their fork

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312374#comment-17312374 ] Apache Spark commented on SPARK-34821: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-34821) Set up a workflow for developers to run benchmark in their fork

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34821: Assignee: Apache Spark > Set up a workflow for developers to run benchmark in their fork

[jira] [Resolved] (SPARK-34881) New SQL Function: TRY_CAST

2021-03-31 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-34881. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31982 [https:

[jira] [Assigned] (SPARK-34922) Use better CBO cost function

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34922: Assignee: Apache Spark > Use better CBO cost function > > >

[jira] [Commented] (SPARK-34922) Use better CBO cost function

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312316#comment-17312316 ] Apache Spark commented on SPARK-34922: -- User 'tanelk' has created a pull request fo

[jira] [Assigned] (SPARK-34922) Use better CBO cost function

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34922: Assignee: (was: Apache Spark) > Use better CBO cost function > --

[jira] [Created] (SPARK-34922) Use better CBO cost function

2021-03-31 Thread Tanel Kiis (Jira)
Tanel Kiis created SPARK-34922: -- Summary: Use better CBO cost function Key: SPARK-34922 URL: https://issues.apache.org/jira/browse/SPARK-34922 Project: Spark Issue Type: Improvement Co

[jira] [Updated] (SPARK-34915) Cache Maven, SBT and Scala in all jobs that use them

2021-03-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34915: - Fix Version/s: 3.0.3 3.1.2 > Cache Maven, SBT and Scala in all jobs that use

[jira] [Updated] (SPARK-34921) Error in spark accessing file

2021-03-31 Thread czh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] czh updated SPARK-34921: Attachment: 微信截图_20210331183234.png > Error in spark accessing file > - > >

[jira] [Created] (SPARK-34921) Error in spark accessing file

2021-03-31 Thread czh (Jira)
czh created SPARK-34921: --- Summary: Error in spark accessing file Key: SPARK-34921 URL: https://issues.apache.org/jira/browse/SPARK-34921 Project: Spark Issue Type: Bug Components: SparkR

[jira] [Commented] (SPARK-34920) Introduce SQLSTATE and ERRORCODE to SQL Exception

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312258#comment-17312258 ] Apache Spark commented on SPARK-34920: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-34920) Introduce SQLSTATE and ERRORCODE to SQL Exception

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34920: Assignee: Apache Spark > Introduce SQLSTATE and ERRORCODE to SQL Exception >

[jira] [Assigned] (SPARK-34920) Introduce SQLSTATE and ERRORCODE to SQL Exception

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34920: Assignee: (was: Apache Spark) > Introduce SQLSTATE and ERRORCODE to SQL Exception > -

[jira] [Updated] (SPARK-34920) Introduce SQLSTATE and ERRORCODE to SQL Exception

2021-03-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34920: Description: SQLSTATE is SQL standard state. Please see was: SQLSTATE is SQL standard state

[jira] [Created] (SPARK-34920) Introduce SQLSTATE and ERRORCODE to SQL Exception

2021-03-31 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-34920: --- Summary: Introduce SQLSTATE and ERRORCODE to SQL Exception Key: SPARK-34920 URL: https://issues.apache.org/jira/browse/SPARK-34920 Project: Spark Issue Type: N

[jira] [Resolved] (SPARK-34915) Cache Maven, SBT and Scala in all jobs that use them

2021-03-31 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-34915. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32011 [https:

[jira] [Assigned] (SPARK-34915) Cache Maven, SBT and Scala in all jobs that use them

2021-03-31 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-34915: -- Assignee: Hyukjin Kwon > Cache Maven, SBT and Scala in all jobs that use them > -

[jira] [Commented] (SPARK-34919) Change partitioning to SinglePartition if partition number is 1

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312249#comment-17312249 ] Apache Spark commented on SPARK-34919: -- User 'ulysses-you' has created a pull reque

[jira] [Commented] (SPARK-34919) Change partitioning to SinglePartition if partition number is 1

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312250#comment-17312250 ] Apache Spark commented on SPARK-34919: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-34919) Change partitioning to SinglePartition if partition number is 1

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34919: Assignee: (was: Apache Spark) > Change partitioning to SinglePartition if partition n

[jira] [Assigned] (SPARK-34919) Change partitioning to SinglePartition if partition number is 1

2021-03-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34919: Assignee: Apache Spark > Change partitioning to SinglePartition if partition number is 1

[jira] [Created] (SPARK-34919) Change partitioning to SinglePartition if partition number is 1

2021-03-31 Thread ulysses you (Jira)
ulysses you created SPARK-34919: --- Summary: Change partitioning to SinglePartition if partition number is 1 Key: SPARK-34919 URL: https://issues.apache.org/jira/browse/SPARK-34919 Project: Spark

[jira] [Updated] (SPARK-34771) Support UDT for Pandas/Spark convertion with Arrow support Enabled

2021-03-31 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darcy Shen updated SPARK-34771: --- Summary: Support UDT for Pandas/Spark convertion with Arrow support Enabled (was: Support UDT for P

[jira] [Updated] (SPARK-34771) Support UDT for Pandas/Spark conversion with Arrow support Enabled

2021-03-31 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darcy Shen updated SPARK-34771: --- Summary: Support UDT for Pandas/Spark conversion with Arrow support Enabled (was: Support UDT for P

[jira] [Updated] (SPARK-34771) Support UDT for Pandas/Spark convertion with Arrow Enabled

2021-03-31 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darcy Shen updated SPARK-34771: --- Summary: Support UDT for Pandas/Spark convertion with Arrow Enabled (was: Support UDT for Pandas/Sp

  1   2   >