[jira] [Commented] (SPARK-36786) SPIP: Improving the compile time performance, by improving a couple of rules, from 24 hrs to under 8 minutes

2023-11-01 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781966#comment-17781966 ] Abhinav Kumar commented on SPARK-36786: --- [~ashahid7] [~adou...@sqli.com] where are we on this one?

[jira] [Commented] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

2023-11-01 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781959#comment-17781959 ] Abhinav Kumar commented on SPARK-33164: --- I see value in some use cases like [~arnaud.nauwynck]

[jira] [Commented] (SPARK-45023) SPIP: Python Stored Procedures

2023-10-22 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778411#comment-17778411 ] Abhinav Kumar commented on SPARK-45023: --- Based on this, I will put another SPIP to create a

[jira] [Commented] (SPARK-44817) SPIP: Incremental Stats Collection

2023-10-18 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776976#comment-17776976 ] Abhinav Kumar commented on SPARK-44817: --- [~rakson] [~gurwls223] [~cloud_fan] - We find this issue

[jira] [Commented] (SPARK-45023) SPIP: Python Stored Procedures

2023-10-18 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776963#comment-17776963 ] Abhinav Kumar commented on SPARK-45023: --- Not sure where we are with this. Looks like we are not

[jira] [Updated] (SPARK-45438) Decimal precision exceeds max precision error when using unary minus on min Decimal values on Scala 2.13 Spark

2023-10-06 Thread Navin Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navin Kumar updated SPARK-45438: Summary: Decimal precision exceeds max precision error when using unary minus on min Decimal

[jira] [Created] (SPARK-45438) Decimal precision exceeds max precision error when using unary minus on min Decimal values

2023-10-06 Thread Navin Kumar (Jira)
Navin Kumar created SPARK-45438: --- Summary: Decimal precision exceeds max precision error when using unary minus on min Decimal values Key: SPARK-45438 URL: https://issues.apache.org/jira/browse/SPARK-45438

[jira] [Updated] (SPARK-44166) Enable dynamicPartitionOverwrite in SaveAsHiveFile for insert overwrite

2023-06-24 Thread Pralabh Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pralabh Kumar updated SPARK-44166: -- Description: Currently in InsertIntoHiveTable.scala , there is no way to pass

[jira] [Updated] (SPARK-44166) Enable dynamicPartitionOverwrite in SaveAsHiveFile for insert overwrite

2023-06-24 Thread Pralabh Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pralabh Kumar updated SPARK-44166: -- Description: Currently in InsertIntoHiveTable.scala , there is no way to pass

[jira] [Updated] (SPARK-44166) Enable dynamicPartitionOverwrite in SaveAsHiveFile for insert overwrite

2023-06-24 Thread Pralabh Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pralabh Kumar updated SPARK-44166: -- Description: Currently in InsertIntoHiveTable.scala , there is no way to pass

[jira] [Created] (SPARK-44166) Enable dynamicPartitionOverwrite in SaveAsHiveFile for insert overwrite

2023-06-24 Thread Pralabh Kumar (Jira)
Pralabh Kumar created SPARK-44166: - Summary: Enable dynamicPartitionOverwrite in SaveAsHiveFile for insert overwrite Key: SPARK-44166 URL: https://issues.apache.org/jira/browse/SPARK-44166 Project:

[jira] [Created] (SPARK-44159) Commands for writting (InsertIntoHadoopFsRelationCommand and InsertIntoHiveTable) should log what they are doing

2023-06-23 Thread Navin Kumar (Jira)
Navin Kumar created SPARK-44159: --- Summary: Commands for writting (InsertIntoHadoopFsRelationCommand and InsertIntoHiveTable) should log what they are doing Key: SPARK-44159 URL:

[jira] [Commented] (SPARK-43235) ClientDistributedCacheManager doesn't set the LocalResourceVisibility.PRIVATE if isPublic throws exception

2023-05-09 Thread Pralabh Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720979#comment-17720979 ] Pralabh Kumar commented on SPARK-43235: --- can any one please look into this . If ok I can create PR

[jira] [Updated] (SPARK-43235) ClientDistributedCacheManager doesn't set the LocalResourceVisibility.PRIVATE if isPublic throws exception

2023-05-01 Thread Pralabh Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pralabh Kumar updated SPARK-43235: -- Description: Hi Spark Team . Currently *ClientDistributedCacheManager* *getVisibility*

[jira] [Commented] (SPARK-43235) ClientDistributedCacheManager doesn't set the LocalResourceVisibility.PRIVATE if isPublic throws exception

2023-05-01 Thread Pralabh Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718163#comment-17718163 ] Pralabh Kumar commented on SPARK-43235: --- Gentle ping to review . I can create a PR for the same 

[jira] [Commented] (SPARK-43235) ClientDistributedCacheManager doesn't set the LocalResourceVisibility.PRIVATE if isPublic throws exception

2023-04-28 Thread Pralabh Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717566#comment-17717566 ] Pralabh Kumar commented on SPARK-43235: --- [~gurwls223] Can u please look into this .  >

[jira] [Created] (SPARK-43235) ClientDistributedCacheManager doesn't set the LocalResourceVisibility.PRIVATE if isPublic throws exception

2023-04-22 Thread Pralabh Kumar (Jira)
Pralabh Kumar created SPARK-43235: - Summary: ClientDistributedCacheManager doesn't set the LocalResourceVisibility.PRIVATE if isPublic throws exception Key: SPARK-43235 URL:

[jira] [Commented] (SPARK-36728) Can't create datetime object from anything other then year column Pyspark - koalas

2023-01-15 Thread Pralabh Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677198#comment-17677198 ] Pralabh Kumar commented on SPARK-36728: --- [~gurwls223] I think this can be closed , as its fixed

[jira] [Commented] (SPARK-36728) Can't create datetime object from anything other then year column Pyspark - koalas

2023-01-15 Thread Pralabh Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677187#comment-17677187 ] Pralabh Kumar commented on SPARK-36728: --- I think this issue is not reproducible on Spark 3.4.

[jira] [Commented] (SPARK-41232) High-order function: array_append

2022-11-30 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641272#comment-17641272 ] Senthil Kumar commented on SPARK-41232: --- [~podongfeng] Shall I work on this? > High-order

[jira] [Commented] (SPARK-23497) Sparklyr Applications doesn't disconnect spark driver in client mode

2022-11-16 Thread bharath kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635096#comment-17635096 ] bharath kumar commented on SPARK-23497: --- This is no longer a problem, tested recently with

[jira] [Commented] (SPARK-40367) Total size of serialized results of 3730 tasks (64.0 GB) is bigger than spark.driver.maxResultSize (64.0 GB)

2022-09-18 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606414#comment-17606414 ] Senthil Kumar commented on SPARK-40367: --- Hi [~jackyjfhu]    Check if you are sending bytes/rows

[jira] [Created] (SPARK-40277) Use DataFrame's column for referring to DDL schema for from_csv() and from_json()

2022-08-30 Thread Jayant Kumar (Jira)
Jayant Kumar created SPARK-40277: Summary: Use DataFrame's column for referring to DDL schema for from_csv() and from_json() Key: SPARK-40277 URL: https://issues.apache.org/jira/browse/SPARK-40277

[jira] [Created] (SPARK-39976) NULL check in ArrayIntersect adds extraneous null from first param

2022-08-03 Thread Navin Kumar (Jira)
Navin Kumar created SPARK-39976: --- Summary: NULL check in ArrayIntersect adds extraneous null from first param Key: SPARK-39976 URL: https://issues.apache.org/jira/browse/SPARK-39976 Project: Spark

[jira] [Updated] (SPARK-39845) 0.0 and -0.0 are not consistent in set operations

2022-07-27 Thread Navin Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navin Kumar updated SPARK-39845: Description: This is a continuation of the issue described in SPARK-32110. When using Array

[jira] [Updated] (SPARK-39845) 0.0 and -0.0 are not consistent in set operations

2022-07-27 Thread Navin Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navin Kumar updated SPARK-39845: Description: This is a continuation of the issue described in SPARK-32110. When using Array

[jira] [Created] (SPARK-39845) 0.0 and -0.0 are not consistent in set operations

2022-07-22 Thread Navin Kumar (Jira)
Navin Kumar created SPARK-39845: --- Summary: 0.0 and -0.0 are not consistent in set operations Key: SPARK-39845 URL: https://issues.apache.org/jira/browse/SPARK-39845 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38213) support Metrics information report to kafkaSink.

2022-02-14 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492391#comment-17492391 ] Senthil Kumar commented on SPARK-38213: --- Working on this > support Metrics information report to

[jira] [Commented] (SPARK-34511) Current Security vulnerabilities in spark libraries

2022-02-09 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489544#comment-17489544 ] Abhinav Kumar commented on SPARK-34511: --- Updating to Spark 3.2.1 does solve most of the issues.

[jira] [Commented] (SPARK-38061) security scan issue with htrace-core4-4.1.0-incubating

2022-02-09 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489537#comment-17489537 ] Abhinav Kumar commented on SPARK-38061: --- [~hyukjin.kwon] [~sujitbiswas] Are we agreeing to track

[jira] (SPARK-34511) Current Security vulnerabilities in spark libraries

2022-02-08 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34511 ] Abhinav Kumar deleted comment on SPARK-34511: --- was (Author: abhinavofficial): Hi [~dongjoon] - We are still seeing json-smart 2.3 in the binaries that are distributed. Is it there by

[jira] [Comment Edited] (SPARK-34511) Current Security vulnerabilities in spark libraries

2022-02-08 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17488719#comment-17488719 ] Abhinav Kumar edited comment on SPARK-34511 at 2/8/22, 10:07 AM: - Hi

[jira] [Commented] (SPARK-34511) Current Security vulnerabilities in spark libraries

2022-02-08 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17488719#comment-17488719 ] Abhinav Kumar commented on SPARK-34511: --- Hi [~dongjoon] - We are still seeing json-smart 2.3 in

[jira] [Commented] (SPARK-37936) Use error classes in the parsing errors of intervals

2022-01-21 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480099#comment-17480099 ] Senthil Kumar commented on SPARK-37936: --- [~maxgekk] ,  I have queries which matches    *

[jira] [Commented] (SPARK-37944) Use error classes in the execution errors of casting

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477551#comment-17477551 ] Senthil Kumar commented on SPARK-37944: --- I will work on this > Use error classes in the execution

[jira] [Commented] (SPARK-37945) Use error classes in the execution errors of arithmetic ops

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477550#comment-17477550 ] Senthil Kumar commented on SPARK-37945: --- I will work on this > Use error classes in the execution

[jira] [Commented] (SPARK-37940) Use error classes in the compilation errors of partitions

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477549#comment-17477549 ] Senthil Kumar commented on SPARK-37940: --- I will work on this > Use error classes in the

[jira] [Commented] (SPARK-37939) Use error classes in the parsing errors of properties

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477548#comment-17477548 ] Senthil Kumar commented on SPARK-37939: --- I will work on this > Use error classes in the parsing

[jira] [Commented] (SPARK-37936) Use error classes in the parsing errors of intervals

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477544#comment-17477544 ] Senthil Kumar commented on SPARK-37936: --- Working on this > Use error classes in the parsing

[jira] [Updated] (SPARK-37475) Add Scale Parameter to Floor and Ceil functions

2021-11-27 Thread Sathiya Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathiya Kumar updated SPARK-37475: -- Description: This feature is proposed in the PR : https://github.com/apache/spark/pull/34593

[jira] [Created] (SPARK-37475) Add Scale Parameter to Floor and Ceil functions

2021-11-27 Thread Sathiya Kumar (Jira)
Sathiya Kumar created SPARK-37475: - Summary: Add Scale Parameter to Floor and Ceil functions Key: SPARK-37475 URL: https://issues.apache.org/jira/browse/SPARK-37475 Project: Spark Issue

[jira] [Created] (SPARK-37433) TimeZoneAwareExpression throws NoSuchElementException: None.get on expr.eval()

2021-11-21 Thread Sathiya Kumar (Jira)
Sathiya Kumar created SPARK-37433: - Summary: TimeZoneAwareExpression throws NoSuchElementException: None.get on expr.eval() Key: SPARK-37433 URL: https://issues.apache.org/jira/browse/SPARK-37433

[jira] [Comment Edited] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-11-17 Thread Siddharth Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445513#comment-17445513 ] Siddharth Kumar edited comment on SPARK-18105 at 11/17/21, 10:28 PM: -

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-11-17 Thread Siddharth Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445513#comment-17445513 ] Siddharth Kumar commented on SPARK-18105: - Hi, I saw a similar failure just as [~vladimir.prus].

[jira] [Updated] (SPARK-37324) Support Decimal RoundingMode.UP, DOWN, HALF_DOWN

2021-11-15 Thread Sathiya Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathiya Kumar updated SPARK-37324: -- Description: Currently we support only Decimal RoundingModes : HALF_UP (round) and HALF_EVEN

[jira] [Updated] (SPARK-37324) Support Decimal RoundingMode.UP, DOWN, HALF_DOWN

2021-11-14 Thread Sathiya Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathiya Kumar updated SPARK-37324: -- Description: Currently we support only Decimal RoundingModes : HALF_UP (round) and HALF_EVEN

[jira] [Updated] (SPARK-37324) Support Decimal RoundingMode.UP, DOWN, HALF_DOWN

2021-11-14 Thread Sathiya Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sathiya Kumar updated SPARK-37324: -- Description: Currently we have support for  > Support Decimal RoundingMode.UP, DOWN,

[jira] [Created] (SPARK-37324) Support Decimal RoundingMode.UP, DOWN, HALF_DOWN

2021-11-14 Thread Sathiya Kumar (Jira)
Sathiya Kumar created SPARK-37324: - Summary: Support Decimal RoundingMode.UP, DOWN, HALF_DOWN Key: SPARK-37324 URL: https://issues.apache.org/jira/browse/SPARK-37324 Project: Spark Issue

[jira] [Commented] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

2021-10-13 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428141#comment-17428141 ] Senthil Kumar commented on SPARK-36996: --- Sample Output after this changes: SQL : mysql> CREATE

[jira] [Commented] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

2021-10-13 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428140#comment-17428140 ] Senthil Kumar commented on SPARK-36996: --- We need to consider 2 scenarios   # maintain NULLABLE

[jira] [Commented] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

2021-10-13 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428104#comment-17428104 ] Senthil Kumar commented on SPARK-36996: --- Based on further analysis, Spark is hard-coding

[jira] [Commented] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

2021-10-13 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428105#comment-17428105 ] Senthil Kumar commented on SPARK-36996: --- I m working on this > fixing "SQL column nullable

[jira] [Created] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

2021-10-13 Thread Senthil Kumar (Jira)
Senthil Kumar created SPARK-36996: - Summary: fixing "SQL column nullable setting not retained as part of spark read" issue Key: SPARK-36996 URL: https://issues.apache.org/jira/browse/SPARK-36996

[jira] [Updated] (SPARK-36925) Overriding parameters

2021-10-04 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhinav Kumar updated SPARK-36925: -- Description: Hello team, One quick question. I read somewhere in the doc - Order of priority

[jira] [Created] (SPARK-36925) Overriding parameters

2021-10-04 Thread Abhinav Kumar (Jira)
Abhinav Kumar created SPARK-36925: - Summary: Overriding parameters Key: SPARK-36925 URL: https://issues.apache.org/jira/browse/SPARK-36925 Project: Spark Issue Type: Question

[jira] [Commented] (SPARK-36238) Spark UI load event timeline too slow for huge stage

2021-10-01 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423419#comment-17423419 ] Senthil Kumar commented on SPARK-36238: --- [~angerszhuuu] Did you try increasing heap memory for

[jira] [Commented] (SPARK-36901) ERROR exchange.BroadcastExchangeExec: Could not execute broadcast in 300 secs

2021-10-01 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423409#comment-17423409 ] Senthil Kumar commented on SPARK-36901: --- [~rangareddy.av...@gmail.com] It looks like normal

[jira] [Comment Edited] (SPARK-36861) Partition columns are overly eagerly parsed as dates

2021-10-01 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423402#comment-17423402 ] Senthil Kumar edited comment on SPARK-36861 at 10/1/21, 7:24 PM: - Yes in

[jira] [Commented] (SPARK-36861) Partition columns are overly eagerly parsed as dates

2021-10-01 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423402#comment-17423402 ] Senthil Kumar commented on SPARK-36861: --- Yes in Spark 3.3 hour column is created as "DateType" but

[jira] [Commented] (SPARK-36861) Partition columns are overly eagerly parsed as dates

2021-09-28 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421328#comment-17421328 ] Senthil Kumar commented on SPARK-36861: --- [~tanelk] This is issue not reproducable even in 3.1.2  

[jira] [Commented] (SPARK-36781) The log could not get the correct line number

2021-09-28 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421281#comment-17421281 ] Senthil Kumar commented on SPARK-36781: --- [~chenxusheng] Could you please share the sample code to

[jira] [Commented] (SPARK-36853) Code failing on checkstyle

2021-09-26 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420258#comment-17420258 ] Abhinav Kumar commented on SPARK-36853: --- This error in thrown in Windows in Maven installing

[jira] [Created] (SPARK-36853) Code failing on checkstyle

2021-09-25 Thread Abhinav Kumar (Jira)
Abhinav Kumar created SPARK-36853: - Summary: Code failing on checkstyle Key: SPARK-36853 URL: https://issues.apache.org/jira/browse/SPARK-36853 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36801) Document change for Spark sql jdbc

2021-09-19 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36801: -- Description: Reading using Spark SQL jdbc DataSource  does not maintain nullable type and

[jira] [Updated] (SPARK-36801) Document change for Spark sql jdbc

2021-09-19 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36801: -- Description: Reading using Spark SQL jdbc DataSource  does not maintain nullable type and

[jira] [Created] (SPARK-36801) Document change for Spark sql jdbc

2021-09-19 Thread Senthil Kumar (Jira)
Senthil Kumar created SPARK-36801: - Summary: Document change for Spark sql jdbc Key: SPARK-36801 URL: https://issues.apache.org/jira/browse/SPARK-36801 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36743) Backporting SPARK-36327 changes into Spark 2.4 version

2021-09-14 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414746#comment-17414746 ] Senthil Kumar commented on SPARK-36743: --- [~hyukjin.kwon], [~dongjoon]. Thanks for the kind and

[jira] [Commented] (SPARK-36743) Backporting SPARK-36327 changes into Spark 2.4 version

2021-09-13 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414242#comment-17414242 ] Senthil Kumar commented on SPARK-36743: --- [~hyukjin.kwon], [~dongjoon] > Backporting SPARK-36327

[jira] [Updated] (SPARK-36743) Backporting SPARK-36327 changes into Spark 2.4 version

2021-09-13 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36743: -- Summary: Backporting SPARK-36327 changes into Spark 2.4 version (was: Backporting changes

[jira] [Created] (SPARK-36743) Backporting changes into Spark 2.4 version

2021-09-13 Thread Senthil Kumar (Jira)
Senthil Kumar created SPARK-36743: - Summary: Backporting changes into Spark 2.4 version Key: SPARK-36743 URL: https://issues.apache.org/jira/browse/SPARK-36743 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

2021-09-02 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408750#comment-17408750 ] Senthil Kumar edited comment on SPARK-35623 at 9/2/21, 11:56 AM: -

[jira] [Commented] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

2021-09-02 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408750#comment-17408750 ] Senthil Kumar commented on SPARK-35623: --- [~dipanjanK] Include me too pls > Volcano resource

[jira] [Updated] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

2021-09-01 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36643: -- Component/s: SQL > Add more information in ERROR log while SparkConf is modified when >

[jira] [Comment Edited] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

2021-09-01 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408282#comment-17408282 ] Senthil Kumar edited comment on SPARK-36643 at 9/1/21, 5:20 PM: New

[jira] [Commented] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

2021-09-01 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408282#comment-17408282 ] Senthil Kumar commented on SPARK-36643: --- New ERROR message will be as below,   {code:java} scala>

[jira] [Commented] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

2021-09-01 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408274#comment-17408274 ] Senthil Kumar commented on SPARK-36643: --- Creating PR for this > Add more information in ERROR log

[jira] [Created] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

2021-09-01 Thread Senthil Kumar (Jira)
Senthil Kumar created SPARK-36643: - Summary: Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set Key: SPARK-36643 URL:

[jira] [Commented] (SPARK-36604) timestamp type column analyze result is wrong

2021-08-30 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407068#comment-17407068 ] Senthil Kumar commented on SPARK-36604: --- [~yghu] I tested this scenario in Spark2.4, but I don't

[jira] [Updated] (SPARK-36412) Add Test Coverage to meet viewFs(Hadoop federation) scenario

2021-08-04 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36412: -- Summary: Add Test Coverage to meet viewFs(Hadoop federation) scenario (was: Create coverage

[jira] [Commented] (SPARK-36412) Create coverage Test to meet viewFs(Hadoop federation) scenario

2021-08-04 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17392976#comment-17392976 ] Senthil Kumar commented on SPARK-36412: --- I am working on this > Create coverage Test to meet

[jira] [Created] (SPARK-36412) Create coverage Test to meet viewFs(Hadoop federation) scenario

2021-08-04 Thread Senthil Kumar (Jira)
Senthil Kumar created SPARK-36412: - Summary: Create coverage Test to meet viewFs(Hadoop federation) scenario Key: SPARK-36412 URL: https://issues.apache.org/jira/browse/SPARK-36412 Project: Spark

[jira] [Commented] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

2021-07-30 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390851#comment-17390851 ] Senthil Kumar commented on SPARK-36327: --- Hi [~sunchao] Hive is creating .staging directories

[jira] [Commented] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

2021-07-29 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389827#comment-17389827 ] Senthil Kumar commented on SPARK-36327: --- Hi [~dongjoon],  [~hyukjin.kwon]   Could you please

[jira] [Updated] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

2021-07-29 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36327: -- Component/s: SQL > Spark sql creates staging dir inside database directory rather than

[jira] [Commented] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

2021-07-29 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389814#comment-17389814 ] Senthil Kumar commented on SPARK-36327: --- Created PR https://github.com/apache/spark/pull/33577 >

[jira] [Commented] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

2021-07-28 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388597#comment-17388597 ] Senthil Kumar commented on SPARK-36327: --- Shall I work on this Jira to fix this issue? > Spark sql

[jira] [Created] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

2021-07-28 Thread Senthil Kumar (Jira)
Senthil Kumar created SPARK-36327: - Summary: Spark sql creates staging dir inside database directory rather than creating inside table directory Key: SPARK-36327 URL:

[jira] [Created] (SPARK-36008) POM errors

2021-07-03 Thread Abhinav Kumar (Jira)
Abhinav Kumar created SPARK-36008: - Summary: POM errors Key: SPARK-36008 URL: https://issues.apache.org/jira/browse/SPARK-36008 Project: Spark Issue Type: Question Components:

[jira] [Resolved] (SPARK-35740) ChromeUIHistoryServerSuite and ChromeUISeleniumSuite failed on Mac 10.15

2021-07-01 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhinav Kumar resolved SPARK-35740. --- Resolution: Not A Problem > ChromeUIHistoryServerSuite and ChromeUISeleniumSuite failed on

[jira] [Comment Edited] (SPARK-35740) ChromeUIHistoryServerSuite and ChromeUISeleniumSuite failed on Mac 10.15

2021-07-01 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372513#comment-17372513 ] Abhinav Kumar edited comment on SPARK-35740 at 7/1/21, 8:16 AM: This can

[jira] [Commented] (SPARK-35740) ChromeUIHistoryServerSuite and ChromeUISeleniumSuite failed on Mac 10.15

2021-07-01 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372513#comment-17372513 ] Abhinav Kumar commented on SPARK-35740: --- This > ChromeUIHistoryServerSuite and

[jira] [Commented] (SPARK-19256) Hive bucketing write support

2021-06-23 Thread Pushkar Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368523#comment-17368523 ] Pushkar Kumar commented on SPARK-19256: --- Thank you [~chengsu]!! > Hive bucketing write support >

[jira] [Commented] (SPARK-19256) Hive bucketing write support

2021-06-23 Thread Pushkar Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368089#comment-17368089 ] Pushkar Kumar commented on SPARK-19256: --- Hi [~chengsu] , Could you please update us here. > Hive

[jira] [Updated] (SPARK-35757) Add bitwise AND operation to BitArray and add intersect AND operation for bloom filters

2021-06-14 Thread Dhruv Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruv Kumar updated SPARK-35757: Issue Type: Improvement (was: New Feature) > Add bitwise AND operation to BitArray and add

[jira] [Created] (SPARK-35757) Add bitwise AND operation to BitArray and add intersect AND operation for bloom filters

2021-06-14 Thread Dhruv Kumar (Jira)
Dhruv Kumar created SPARK-35757: --- Summary: Add bitwise AND operation to BitArray and add intersect AND operation for bloom filters Key: SPARK-35757 URL: https://issues.apache.org/jira/browse/SPARK-35757

[jira] [Updated] (SPARK-35740) ChromeUIHistoryServerSuite and ChromeUISeleniumSuite failed on Mac 10.15

2021-06-11 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhinav Kumar updated SPARK-35740: -- Description: import org.openqa.selenium.chrome.\{ChromeDriver, ChromeOptions}  ChromeDriver

[jira] [Updated] (SPARK-35740) ChromeUIHistoryServerSuite and ChromeUISeleniumSuite failed on Mac 10.15

2021-06-11 Thread Abhinav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhinav Kumar updated SPARK-35740: -- Summary: ChromeUIHistoryServerSuite and ChromeUISeleniumSuite failed on Mac 10.15 (was:

[jira] [Created] (SPARK-35740) ChromeUIHistoryServerSuite failed on Mac 10.15

2021-06-11 Thread Abhinav Kumar (Jira)
Abhinav Kumar created SPARK-35740: - Summary: ChromeUIHistoryServerSuite failed on Mac 10.15 Key: SPARK-35740 URL: https://issues.apache.org/jira/browse/SPARK-35740 Project: Spark Issue Type:

[jira] [Issue Comment Deleted] (SPARK-10388) Public dataset loader interface

2021-05-25 Thread Gaurav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Kumar updated SPARK-10388: - Comment: was deleted (was: I want to work on this issue [~mengxr], yet I am new to opensource.

[jira] [Commented] (SPARK-10388) Public dataset loader interface

2021-05-25 Thread Gaurav Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351097#comment-17351097 ] Gaurav Kumar commented on SPARK-10388: -- I want to work on this issue [~mengxr], yet I am new to

  1   2   3   4   5   6   7   >