[
https://issues.apache.org/jira/browse/BEAM-8218?focusedWorklogId=668691&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-668691
]
ASF GitHub Bot logged work on BEAM-8218:
----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Oct/21 23:10
Start Date: 21/Oct/21 23:10
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #15572:
URL: https://github.com/apache/beam/pull/15572#issuecomment-949067175
> Hi @lukecwik , I am working on the watermark estimator, and taking a look
to KafkaIO, I see it has two types of watermark (MonotonicallyIncreasing and
Manual), for first version of PulsarIO, I was consider only using
MonotonicallyIncreasing, I want to know your input on this concern
The purpose of the watermark tracking is to be able to fire certain triggers
downstream of this transform instead of waiting till all the output of this
transform is produced before being able to advance the watermark. In this case
if the timestamp of the record is always equivalent to `currentOffset` then
using `MonotonicallyIncreasing` makes sense. You'll also want to use
`outputWithTimestamp(record, new Instant(currentOffset))` instead of the
existing `output(record)` method.
If the timestamp of the record is from some data associated within the
record then it is dependent on the ordering the user provides and you'll want
to provide some user some configuration options that allow you to convert a
record to a timestamp and also to state how they want the watermark estimation
to happen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 668691)
Time Spent: 2h 40m (was: 2.5h)
> Implement Apache PulsarIO
> -------------------------
>
> Key: BEAM-8218
> URL: https://issues.apache.org/jira/browse/BEAM-8218
> Project: Beam
> Issue Type: Task
> Components: io-ideas
> Reporter: Alex Van Boxel
> Assignee: Marco Robles
> Priority: P3
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Apache Pulsar is starting to gain popularity. Having a native Beam PulsarIO
> could be beneficial.
> [https://pulsar.apache.org/|https://pulsar.apache.org/en/]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
