[jira] [Commented] (FLINK-4194) Implement isEndOfStream() for KinesisDeserializationSchema
[
https://issues.apache.org/jira/browse/FLINK-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334318#comment-17334318
]
Flink Jira Bot commented on FLINK-4194:
---
This issue was marked "stale-assigned" and has not received an update in 7
days. It is now automatically unassigned. If you are still working on it, you
can assign it to yourself again. Please also give an update about the status of
the work.
> Implement isEndOfStream() for KinesisDeserializationSchema
> --
>
> Key: FLINK-4194
> URL: https://issues.apache.org/jira/browse/FLINK-4194
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kinesis
>Reporter: Robert Metzger
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
> Labels: stale-assigned
>
> **Original JIRA title: KinesisDeserializationSchema.isEndOfStream() is never
> called. The corresponding part of the code has been commented out with
> reference to this JIRA.**
> The Kinesis connector does not respect the
> {{KinesisDeserializationSchema.isEndOfStream()}} method.
> The purpose of this method is to stop consuming from a source, based on input
> data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-4194) Implement isEndOfStream() for KinesisDeserializationSchema
[
https://issues.apache.org/jira/browse/FLINK-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17323712#comment-17323712
]
Flink Jira Bot commented on FLINK-4194:
---
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> Implement isEndOfStream() for KinesisDeserializationSchema
> --
>
> Key: FLINK-4194
> URL: https://issues.apache.org/jira/browse/FLINK-4194
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kinesis
>Reporter: Robert Metzger
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
> Labels: stale-assigned
>
> **Original JIRA title: KinesisDeserializationSchema.isEndOfStream() is never
> called. The corresponding part of the code has been commented out with
> reference to this JIRA.**
> The Kinesis connector does not respect the
> {{KinesisDeserializationSchema.isEndOfStream()}} method.
> The purpose of this method is to stop consuming from a source, based on input
> data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-4194) Implement isEndOfStream() for KinesisDeserializationSchema
[
https://issues.apache.org/jira/browse/FLINK-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370997#comment-16370997
]
Cristian commented on FLINK-4194:
-
Hi guys. This is a feature we miss.
Given the fact that it's not implemented and will likely not be implemented in
the near future... how else could this be done right now?
Let me explain. Right now I need to run flink applications that read from
Kinesis for a specific period of time (say two days), and I'm trying to figure
out a way for the Flink app to gracefully stop itself after that. Is there a
way to achieve that right now from within the Flink app?
My other option, which I'd like to avoid, is to periodically check for flink
apps to kill from an external worker.
> Implement isEndOfStream() for KinesisDeserializationSchema
> --
>
> Key: FLINK-4194
> URL: https://issues.apache.org/jira/browse/FLINK-4194
> Project: Flink
> Issue Type: New Feature
> Components: Kinesis Connector
>Reporter: Robert Metzger
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
>
> **Original JIRA title: KinesisDeserializationSchema.isEndOfStream() is never
> called. The corresponding part of the code has been commented out with
> reference to this JIRA.**
> The Kinesis connector does not respect the
> {{KinesisDeserializationSchema.isEndOfStream()}} method.
> The purpose of this method is to stop consuming from a source, based on input
> data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-4194) Implement isEndOfStream() for KinesisDeserializationSchema
[
https://issues.apache.org/jira/browse/FLINK-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572052#comment-15572052
]
Robert Metzger commented on FLINK-4194:
---
The method {{KinesisDeserializationSchema.isEndOfStream}} is not part of the
1.1.x release:
https://github.com/apache/flink/blob/release-1.1.3/flink-streaming-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/serialization/KinesisDeserializationSchema.java
So if you want, you can rename it to {{isEndOfShard()}}. I think that renaming
makes sense, because it describes the behavior more accurately.
> Implement isEndOfStream() for KinesisDeserializationSchema
> --
>
> Key: FLINK-4194
> URL: https://issues.apache.org/jira/browse/FLINK-4194
> Project: Flink
> Issue Type: New Feature
> Components: Kinesis Connector
>Reporter: Robert Metzger
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.2.0
>
>
> **Original JIRA title: KinesisDeserializationSchema.isEndOfStream() is never
> called. The corresponding part of the code has been commented out with
> reference to this JIRA.**
> The Kinesis connector does not respect the
> {{KinesisDeserializationSchema.isEndOfStream()}} method.
> The purpose of this method is to stop consuming from a source, based on input
> data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4194) Implement isEndOfStream() for KinesisDeserializationSchema
[
https://issues.apache.org/jira/browse/FLINK-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15461269#comment-15461269
]
Tzu-Li (Gordon) Tai commented on FLINK-4194:
On second thought, I think we ought to discuss whether or not adding the
function back to {{KinesisDeserializationSchema}} and breaking user code is a
good idea now, since the Kinesis connector is now in a released version and
would need a migration plan. Its not a highly requested feature at the moment,
so I would say it is currently not worth the effort. Perhaps a good time to add
this back is when it is actually possible to "stop" parallel consumption across
the subtasks with an atomic checkpoint-and-stop, so an {{isEndOfStream()}}
behaviour can be truly defined.
What do you think, [~rmetzger]?
> Implement isEndOfStream() for KinesisDeserializationSchema
> --
>
> Key: FLINK-4194
> URL: https://issues.apache.org/jira/browse/FLINK-4194
> Project: Flink
> Issue Type: New Feature
> Components: Kinesis Connector
>Reporter: Robert Metzger
>Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.2.0
>
>
> **Original JIRA title: KinesisDeserializationSchema.isEndOfStream() is never
> called. The corresponding part of the code has been commented out with
> reference to this JIRA.**
> The Kinesis connector does not respect the
> {{KinesisDeserializationSchema.isEndOfStream()}} method.
> The purpose of this method is to stop consuming from a source, based on input
> data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4194) Implement isEndOfStream() for KinesisDeserializationSchema
[
https://issues.apache.org/jira/browse/FLINK-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15443628#comment-15443628
]
Tzu-Li (Gordon) Tai commented on FLINK-4194:
I'll implement {{isEndOfShard()}} as an initial PR, so we can start discussing
this from there.
> Implement isEndOfStream() for KinesisDeserializationSchema
> --
>
> Key: FLINK-4194
> URL: https://issues.apache.org/jira/browse/FLINK-4194
> Project: Flink
> Issue Type: New Feature
> Components: Kinesis Connector
>Affects Versions: 1.1.0
>Reporter: Robert Metzger
>Assignee: Tzu-Li (Gordon) Tai
>
> **Original JIRA title: KinesisDeserializationSchema.isEndOfStream() is never
> called. The corresponding part of the code has been commented out with
> reference to this JIRA.**
> The Kinesis connector does not respect the
> {{KinesisDeserializationSchema.isEndOfStream()}} method.
> The purpose of this method is to stop consuming from a source, based on input
> data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
