[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-16929:
---
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png, image-2020-04-06-12-11-01-138.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-56-00-239.png|width=753,height=406!
>
> *Events*
>
> !image-2020-04-01-19-56-27-720.png|width=312,height=383!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-16929:
---
Labels: auto-deprioritized-major stale-minor (was:
auto-deprioritized-major)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Minor
> Labels: auto-deprioritized-major, stale-minor
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png, image-2020-04-06-12-11-01-138.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-56-00-239.png|width=753,height=406!
>
> *Events*
>
> !image-2020-04-01-19-56-27-720.png|width=312,height=383!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-16929:
---
Labels: auto-deprioritized-major (was: stale-major)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Labels: auto-deprioritized-major
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png, image-2020-04-06-12-11-01-138.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-56-00-239.png|width=753,height=406!
>
> *Events*
>
> !image-2020-04-01-19-56-27-720.png|width=312,height=383!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-16929:
---
Priority: Minor (was: Major)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Minor
> Labels: auto-deprioritized-major
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png, image-2020-04-06-12-11-01-138.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-56-00-239.png|width=753,height=406!
>
> *Events*
>
> !image-2020-04-01-19-56-27-720.png|width=312,height=383!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-16929:
---
Labels: stale-major (was: )
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Labels: stale-major
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png, image-2020-04-06-12-11-01-138.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-56-00-239.png|width=753,height=406!
>
> *Events*
>
> !image-2020-04-01-19-56-27-720.png|width=312,height=383!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger updated FLINK-16929:
---
Component/s: API / DataStream
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png, image-2020-04-06-12-11-01-138.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-56-00-239.png|width=753,height=406!
>
> *Events*
>
> !image-2020-04-01-19-56-27-720.png|width=312,height=383!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: image-2020-04-06-12-11-01-138.png
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png, image-2020-04-06-12-11-01-138.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-56-00-239.png|width=753,height=406!
>
> *Events*
>
> !image-2020-04-01-19-56-27-720.png|width=312,height=383!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: image-2020-04-01-19-56-27-720.png
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
>
>
> *Events*
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Description:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
!image-2020-04-01-19-56-00-239.png|width=753,height=406!
*Events*
!image-2020-04-01-19-56-27-720.png|width=312,height=383!
was:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
*Events*
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-56-00-239.png|width=753,height=406!
>
> *Events*
>
> !image-2020-04-01-19-56-27-720.png|width=312,height=383!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: image-2020-04-01-19-56-00-239.png
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-56-00-239.png,
> image-2020-04-01-19-56-27-720.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
>
>
> *Events*
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: image-2020-04-01-19-55-04-886.png
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-50-06-326.png|width=896,height=599!
>
> *Events*
>
> !image-2020-04-01-19-50-23-954.png|width=280,height=617!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: (was: image-2020-04-01-19-55-04-886.png)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
>
>
> *Events*
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: (was: screenshot-1.png)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
>
>
> *Events*
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Description:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
*Events*
was:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
!image-2020-04-01-19-50-06-326.png|width=896,height=599!
*Events*
!image-2020-04-01-19-50-23-954.png|width=280,height=617!
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
>
>
> *Events*
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: (was: image-2020-04-01-19-50-23-954.png)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-50-06-326.png|width=896,height=599!
>
> *Events*
>
> !image-2020-04-01-19-50-23-954.png|width=280,height=617!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: (was: image-2020-04-01-19-43-07-288.png)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-50-23-954.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-50-06-326.png|width=896,height=599!
>
> *Events*
>
> !image-2020-04-01-19-50-23-954.png|width=280,height=617!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: (was: image-2020-04-01-19-50-06-326.png)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-50-23-954.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-50-06-326.png|width=896,height=599!
>
> *Events*
>
> !image-2020-04-01-19-50-23-954.png|width=280,height=617!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: (was: image-2020-04-01-19-49-08-016.png)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-50-23-954.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-50-06-326.png|width=896,height=599!
>
> *Events*
>
> !image-2020-04-01-19-50-23-954.png|width=280,height=617!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: (was: image-2020-04-01-19-48-35-977.png)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-50-23-954.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-50-06-326.png|width=896,height=599!
>
> *Events*
>
> !image-2020-04-01-19-50-23-954.png|width=280,height=617!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: (was: image-2020-04-01-19-44-35-714.png)
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-50-23-954.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-50-06-326.png|width=896,height=599!
>
> *Events*
>
> !image-2020-04-01-19-50-23-954.png|width=280,height=617!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Description:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
!image-2020-04-01-19-50-06-326.png|width=896,height=599!
*Events*
!image-2020-04-01-19-50-23-954.png|width=280,height=617!
was:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
!image-2020-04-01-19-50-06-326.png!
*Events*
!image-2020-04-01-19-50-23-954.png!
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-50-23-954.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-50-06-326.png|width=896,height=599!
>
> *Events*
>
> !image-2020-04-01-19-50-23-954.png|width=280,height=617!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Description:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
!image-2020-04-01-19-50-06-326.png!
*Events*
!image-2020-04-01-19-50-23-954.png!
was:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
!image-2020-04-01-19-48-35-977.png!
*Events*
*!image-2020-04-01-19-49-08-016.png!*
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-43-07-288.png,
> image-2020-04-01-19-44-35-714.png, image-2020-04-01-19-48-35-977.png,
> image-2020-04-01-19-49-08-016.png, image-2020-04-01-19-50-06-326.png,
> image-2020-04-01-19-50-23-954.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-50-06-326.png!
>
> *Events*
>
> !image-2020-04-01-19-50-23-954.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: image-2020-04-01-19-50-23-954.png
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-43-07-288.png,
> image-2020-04-01-19-44-35-714.png, image-2020-04-01-19-48-35-977.png,
> image-2020-04-01-19-49-08-016.png, image-2020-04-01-19-50-06-326.png,
> image-2020-04-01-19-50-23-954.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-48-35-977.png!
>
> *Events*
> *!image-2020-04-01-19-49-08-016.png!*
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: image-2020-04-01-19-50-06-326.png
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-43-07-288.png,
> image-2020-04-01-19-44-35-714.png, image-2020-04-01-19-48-35-977.png,
> image-2020-04-01-19-49-08-016.png, image-2020-04-01-19-50-06-326.png,
> image-2020-04-01-19-50-23-954.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-48-35-977.png!
>
> *Events*
> *!image-2020-04-01-19-49-08-016.png!*
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Description:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
!image-2020-04-01-19-48-35-977.png!
*Events*
*!image-2020-04-01-19-49-08-016.png!*
was:
We have a Flink job which keyBys session ID (sId), and uses a session window
with 30 minutes gap:
{code:java}
inputStream
.keyBy(keySelector)
.window(EventTimeSessionWindows.withGap(Time.minutes(30)))
.allowedLateness(Time.seconds(0L))
{code}
This Flink job reads from Kinesis stream.
Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
sessions, with gaps of several seconds (instead of 30 minutes).
We have no idea why it's happening and suspect a Flink bug or a state backend
bug (we use RocksDB).
I haven't found any indication in the logs except for some read throughput
warnings which were resolved by a backoff.
Attached is a table of derived sessions, and then the raw events
*Sessions*
!image-2020-04-01-19-43-07-288.png!
*Events*
*!image-2020-04-01-19-44-35-714.png!*
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-43-07-288.png,
> image-2020-04-01-19-44-35-714.png, image-2020-04-01-19-48-35-977.png,
> image-2020-04-01-19-49-08-016.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-48-35-977.png!
>
> *Events*
> *!image-2020-04-01-19-49-08-016.png!*
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: image-2020-04-01-19-49-08-016.png
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-43-07-288.png,
> image-2020-04-01-19-44-35-714.png, image-2020-04-01-19-48-35-977.png,
> image-2020-04-01-19-49-08-016.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-43-07-288.png!
>
> *Events*
> *!image-2020-04-01-19-44-35-714.png!*
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: screenshot-1.png
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-43-07-288.png,
> image-2020-04-01-19-44-35-714.png, image-2020-04-01-19-48-35-977.png,
> image-2020-04-01-19-49-08-016.png, screenshot-1.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-48-35-977.png!
>
> *Events*
> *!image-2020-04-01-19-49-08-016.png!*
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16929) Session Window produces sessions randomly
[
https://issues.apache.org/jira/browse/FLINK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ori Popowski updated FLINK-16929:
-
Attachment: image-2020-04-01-19-48-35-977.png
> Session Window produces sessions randomly
> -
>
> Key: FLINK-16929
> URL: https://issues.apache.org/jira/browse/FLINK-16929
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.9.1
>Reporter: Ori Popowski
>Priority: Major
> Attachments: image-2020-04-01-19-43-07-288.png,
> image-2020-04-01-19-44-35-714.png, image-2020-04-01-19-48-35-977.png
>
>
> We have a Flink job which keyBys session ID (sId), and uses a session window
> with 30 minutes gap:
> {code:java}
> inputStream
> .keyBy(keySelector)
> .window(EventTimeSessionWindows.withGap(Time.minutes(30)))
> .allowedLateness(Time.seconds(0L))
> {code}
> This Flink job reads from Kinesis stream.
> Lately (I suspect after upgrading from 1.5.4 to 1.9.1) we get too many
> sessions, with gaps of several seconds (instead of 30 minutes).
> We have no idea why it's happening and suspect a Flink bug or a state backend
> bug (we use RocksDB).
> I haven't found any indication in the logs except for some read throughput
> warnings which were resolved by a backoff.
> Attached is a table of derived sessions, and then the raw events
> *Sessions*
> !image-2020-04-01-19-43-07-288.png!
>
> *Events*
> *!image-2020-04-01-19-44-35-714.png!*
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
