[jira] [Comment Edited] (SPARK-24156) Enable no-data micro batches for more eager streaming state clean up

[ https://issues.apache.org/jira/browse/SPARK-24156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17483838#comment-17483838 ] krishna edited comment on SPARK-24156 at 2/3/22, 2:53 PM: -- Hi [~kcsrms] [~tdas]

[jira] [Comment Edited] (SPARK-24156) Enable no-data micro batches for more eager streaming state clean up

[ https://issues.apache.org/jira/browse/SPARK-24156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17483838#comment-17483838 ] krishna edited comment on SPARK-24156 at 2/3/22, 2:48 PM: -- Hi [~kcsrms] [~tdas]

[jira] [Updated] (SPARK-38078) Aggregation with Watermark in AppendMode is holding data beyong water mark boundary.

[ https://issues.apache.org/jira/browse/SPARK-38078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] krishna updated SPARK-38078: Description:  I am struggling with a unique issue. I am not sure if my understanding is wrong or this is

[jira] [Comment Edited] (SPARK-24156) Enable no-data micro batches for more eager streaming state clean up

[ https://issues.apache.org/jira/browse/SPARK-24156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17483838#comment-17483838 ] krishna edited comment on SPARK-24156 at 2/1/22, 9:04 PM: -- Hi [~kcsrms] [~tdas]

[jira] [Created] (SPARK-38078) Aggregation with Watermark in AppendMode is holding data beyong water mark boundary.

krishna created SPARK-38078: --- Summary: Aggregation with Watermark in AppendMode is holding data beyong water mark boundary. Key: SPARK-38078 URL: https://issues.apache.org/jira/browse/SPARK-38078 Project:

[jira] [Commented] (SPARK-24156) Enable no-data micro batches for more eager streaming state clean up

[ https://issues.apache.org/jira/browse/SPARK-24156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17483838#comment-17483838 ] krishna commented on SPARK-24156: - Hi [~kcsrms] [~tdas] ,   I am having the same issue. Is this issue

[jira] [Commented] (SPARK-26054) Creating a computed column applying the spark sql rounding on a column of type decimal affects the orginal column as well.

[ https://issues.apache.org/jira/browse/SPARK-26054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686386#comment-16686386 ] Jaya Krishna commented on SPARK-26054: -- Are you not seeing the issue even with BigDecimal data

[jira] [Commented] (SPARK-26054) Creating a computed column applying the spark sql rounding on a column of type decimal affects the orginal column as well.

[ https://issues.apache.org/jira/browse/SPARK-26054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686373#comment-16686373 ] Jaya Krishna commented on SPARK-26054: -- Sorry for the confusion. I actually joined screen shots of

[jira] [Updated] (SPARK-26054) Creating a computed column applying the spark sql rounding on a column of type decimal affects the orginal column as well.

[ https://issues.apache.org/jira/browse/SPARK-26054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaya Krishna updated SPARK-26054: - Attachment: (was: sparksql-rounding.png) > Creating a computed column applying the spark

[jira] [Updated] (SPARK-26054) Creating a computed column applying the spark sql rounding on a column of type decimal affects the orginal column as well.

[ https://issues.apache.org/jira/browse/SPARK-26054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaya Krishna updated SPARK-26054: - Attachment: sparksql-rounding.png > Creating a computed column applying the spark sql rounding

[jira] [Commented] (SPARK-26054) Creating a computed column applying the spark sql rounding on a column of type decimal affects the orginal column as well.

[ https://issues.apache.org/jira/browse/SPARK-26054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686318#comment-16686318 ] Jaya Krishna commented on SPARK-26054: -- Hmm. There seems to be an issue if we start with a RDD -

[jira] [Updated] (SPARK-26054) Creating a computed column applying the spark sql rounding on a column of type decimal affects the orginal column as well.

[ https://issues.apache.org/jira/browse/SPARK-26054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaya Krishna updated SPARK-26054: - Description: When a computed column that rounds the value is added to a data frame, it is

[jira] [Updated] (SPARK-26054) Creating a computed column applying the spark sql rounding on a column of type decimal affects the orginal column as well.

[ https://issues.apache.org/jira/browse/SPARK-26054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaya Krishna updated SPARK-26054: - Attachment: sparksql-rounding.png > Creating a computed column applying the spark sql rounding

[jira] [Created] (SPARK-26054) Creating a computed column applying the spark sql rounding on a column of type decimal affects the orginal column as well.

Jaya Krishna created SPARK-26054: Summary: Creating a computed column applying the spark sql rounding on a column of type decimal affects the orginal column as well. Key: SPARK-26054 URL:

[jira] [Reopened] (SPARK-24857) required the sample code test the spark steaming job in kubernates and write the data in remote hdfs file system

[ https://issues.apache.org/jira/browse/SPARK-24857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kumpatla murali krishna reopened SPARK-24857: - Hi ,   I am facing issues while running the spark job on eks.  

[jira] [Created] (SPARK-24857) required the sample code test the spark steaming job in kubernates and write the data in remote hdfs file system

kumpatla murali krishna created SPARK-24857: --- Summary: required the sample code test the spark steaming job in kubernates and write the data in remote hdfs file system Key: SPARK-24857 URL:

[jira] [Commented] (SPARK-19268) File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta

[ https://issues.apache.org/jira/browse/SPARK-19268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000374#comment-16000374 ] krishna commented on SPARK-19268: - Hi [~zsxwing] I have tested with spark 2.1.1, still the same problem