[jira] [Comment Edited] (SPARK-20168) Enable kinesis to start stream from Initial position specified by a timestamp

2018-09-04 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16602909#comment-16602909 ] Vladimir Pchelko edited comment on SPARK-20168 at 9/4/18 11:04 AM: ---

[jira] [Commented] (SPARK-20168) Enable kinesis to start stream from Initial position specified by a timestamp

2018-09-04 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16602909#comment-16602909 ] Vladimir Pchelko commented on SPARK-20168: -- [~srowen]  this bug must be covered by unit tests

[jira] [Commented] (SPARK-18805) InternalMapWithStateDStream make java.lang.StackOverflowError

2016-12-22 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15771504#comment-15771504 ] Vladimir Pchelko commented on SPARK-18805: -- I had faced with similar problem ... there are two

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

2016-12-21 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Description: With default settings mapWithState leads to storing up to 10 copies of

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

2016-12-21 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Description: With default settings mapWithState leads to storing up to 10 copies of

[jira] [Comment Edited] (SPARK-18564) mapWithState: add configuration for DEFAULT_CHECKPOINT_DURATION_MULTIPLIER

2016-12-21 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766561#comment-15766561 ] Vladimir Pchelko edited comment on SPARK-18564 at 12/21/16 9:21 AM:

[jira] [Comment Edited] (SPARK-18564) mapWithState: add configuration for DEFAULT_CHECKPOINT_DURATION_MULTIPLIER

2016-12-21 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766561#comment-15766561 ] Vladimir Pchelko edited comment on SPARK-18564 at 12/21/16 9:21 AM:

[jira] [Comment Edited] (SPARK-18564) mapWithState: add configuration for DEFAULT_CHECKPOINT_DURATION_MULTIPLIER

2016-12-21 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766561#comment-15766561 ] Vladimir Pchelko edited comment on SPARK-18564 at 12/21/16 9:20 AM:

[jira] [Commented] (SPARK-18564) mapWithState: add configuration for DEFAULT_CHECKPOINT_DURATION_MULTIPLIER

2016-12-21 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766561#comment-15766561 ] Vladimir Pchelko commented on SPARK-18564: -- Currently user can modify interval of mapWithState

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

2016-12-19 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Description: With default settings mapWithState leads to storing up to 10 copies of

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

2016-12-19 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Description: With default settings mapWithState leads to storing up to 10 copies of

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

2016-12-19 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Attachment: (was: 0001-override-clearMetadata-for-InternalMapWithStateDStre.patch)

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

2016-12-19 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Attachment: 0001-override-clearMetadata-for-InternalMapWithStateDStre.patch > Reduce

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

2016-12-19 Thread Vladimir Pchelko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Affects Version/s: 2.0.0 2.0.1 2.0.2

[jira] [Created] (SPARK-18925) Reduce memory usage of mapWithState

2016-12-19 Thread Vladimir Pchelko (JIRA)
Vladimir Pchelko created SPARK-18925: Summary: Reduce memory usage of mapWithState Key: SPARK-18925 URL: https://issues.apache.org/jira/browse/SPARK-18925 Project: Spark Issue Type: