This is an automated email from the ASF dual-hosted git repository.

shuwenwei pushed a commit to branch supports_subscription
in repository https://gitbox.apache.org/repos/asf/iotdb-client-go.git

commit 16bf05838385de1f8a6689f7876a451a1edd6104
Author: shuwenwei <[email protected]>
AuthorDate: Fri Nov 13 11:25:46 2026 +0800

    Implement subscription push consumer
---
 subscription/README.md        |  71 ++++++-
 subscription/config.go        |  19 +-
 subscription/consumer.go      |  34 +++-
 subscription/doc.go           |   7 +-
 subscription/pull_consumer.go |  10 +
 subscription/push_consumer.go | 449 ++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 568 insertions(+), 22 deletions(-)

diff --git a/subscription/README.md b/subscription/README.md
index f470148..aa49988 100644
--- a/subscription/README.md
+++ b/subscription/README.md
@@ -19,7 +19,7 @@ under the License.
 
 # Go Subscription API
 
-The module implements `SubscriptionSession` management and the configuration, 
construction, subscription management, record polling, commits, progress, seek, 
processor pipeline, and background connection coordination of 
`SubscriptionPullConsumer`. Poll response decoding, record decoding, and 
message release are also connected. Push consumers and built-in processors 
remain for subsequent steps. These implementations have not been compiled or 
exercised at runtime; this document describes t [...]
+The module implements `SubscriptionSession` management, Pull consumption with 
record polling, commits, progress, seek, processor pipelines, and background 
connection coordination, and Push consumption with sequential callbacks and 
acknowledgment strategies. Poll response decoding, record decoding, and message 
release are also connected. Built-in processors remain for a subsequent step. 
These implementations have not been compiled or exercised at runtime; this 
document describes the code  [...]
 
 The module path is `github.com/apache/iotdb-client-go/subscription`. It uses 
Go 1.25 and maintains dependencies and versions separately from the existing 
`github.com/apache/iotdb-client-go/v2` Session module. Its direct dependencies 
are `github.com/apache/iotdb-client-go/v2` at 
`v2.0.0-20260914074454-a25f68aa9681`, which provides existing types such as 
`TSDataType` and `ColumnCategory`, and `github.com/apache/thrift` at `v0.24.0`, 
which provides the RPC transport and protocol. It adds no [...]
 
@@ -43,7 +43,7 @@ Only the public method contract is shared. Management 
sessions pass the model wh
 | `SubscriptionPushConsumerConfig` | `AbstractSubscriptionPushConsumerBuilder` 
| Combines a Handler, ACK strategy, and polling interval and timeout |
 | `AckStrategy`, `AckAfterConsume`, `AckBeforeConsume` | 
`AckStrategy.AFTER_CONSUME`, `BEFORE_CONSUME` | The zero value acknowledges 
after consumption; callback failure does not cause redelivery when 
acknowledgment precedes consumption |
 
-`SubscriptionConsumerConfig.OwnerEpoch == nil` means that no epoch was 
supplied; a pointer to zero explicitly supplies epoch 0. 
`DefaultSubscriptionConsumerConfig`, `DefaultSubscriptionPullConsumerConfig`, 
and `DefaultSubscriptionPushConsumerConfig` are available. The default Push 
Handler remains nil, so callers must supply a real handler. Internal 
normalization copies endpoint slices and optional pointers. Zero durations 
select defaults, negative durations return errors, and positive va [...]
+`SubscriptionConsumerConfig.OwnerEpoch == nil` means that no epoch was 
supplied; a pointer to zero explicitly supplies epoch 0. 
`DefaultSubscriptionConsumerConfig`, `DefaultSubscriptionPullConsumerConfig`, 
and `DefaultSubscriptionPushConsumerConfig` are available. The default Push 
Handler remains nil, so callers must supply a real handler. Internal 
normalization copies endpoint slices and optional pointers. Zero durations 
select defaults, negative durations return errors, and positive va [...]
 
 The management session defaults are `localhost:6667`, the tree model, 
root/root authentication, a 64 MiB frame limit, and a ten-second connection and 
authentication timeout. Java management sessions do not limit connection time 
by default; the Go entry point uses a bounded timeout here. The constructor 
fills empty endpoints, username, and dialect, as well as zero timeout and frame 
size, validates addresses and mutually exclusive authentication, and copies the 
endpoint slice. Passwords ar [...]
 
@@ -65,7 +65,7 @@ Management connections own query pagination and closing while 
reusing the existi
 
 ## Internal Subscription Protocol and Providers
 
-The internal implementation covers handshakes, heartbeats, 
subscribe/unsubscribe, close, commits, seek, Poll and Tablet-continuation 
requests, and oversized-request slicing. These are connected to the public Pull 
consumer.
+The internal implementation covers handshakes, heartbeats, 
subscribe/unsubscribe, close, commits, seek, Poll and Tablet-continuation 
requests, and oversized-request slicing. Pull and Push share these internal 
capabilities.
 
 - Primitive encoding uses Java's big-endian layout. Commit contexts use 
version 2 and retain legacy commit fields plus optional Writer identity and 
progress. Topic/Region/Writer progress follows the original protocol structure.
 - Topic sets in subscribe and Poll requests use Java ObjectSet string tags, 
separately from the ordinary StringList used by heartbeat unsubscribe lists. 
Set encoding is sorted for stability. Null strings map to empty strings in 
existing Go string fields.
@@ -136,6 +136,61 @@ return consumer.Commit(ctx, messages...)
 - Heartbeats are scheduled independently per Provider. Endpoint 
synchronization and connection failure trigger recovery. Background recovery 
attempts are at least five seconds apart. Late old-connection responses cannot 
reactivate an old Provider or incorrectly fence the replacement connection's 
Consumer.
 - `Close` interrupts blocked polling and stops background work. In manual 
mode, buffered processor messages make closing return an error while preserving 
the drain path. Call `DrainBufferedMessages`, process and commit its records, 
then close again. Automatic mode makes a best-effort commit of delivered and 
drained records during closing. Commit failures return errors; a successful 
close does not imply that all historical messages were processed.
 
+## Push Consumption
+
+`NewSubscriptionPushConsumer` normalizes configuration once, requires a 
non-nil Handler, and does not connect. It reuses the manual Pull core for 
providers, progress, and polling, without installing processors or enabling 
automatic commits. `Open(ctx)` starts one fixed-delay polling loop; its 
background lifetime is independent of the opening context. Handlers run 
sequentially, and the next round starts only after the current round and 
`PollInterval` delay finish.
+
+```go
+cfg := subscription.DefaultSubscriptionPushConsumerConfig()
+cfg.Consumer.Dialect = subscription.TableDialect
+cfg.AckStrategy = subscription.AckAfterConsume
+cfg.Handler = func(callbackCtx context.Context, message 
subscription.SubscriptionMessage) error {
+    batches, err := message.Records()
+    if err != nil {
+        return err
+    }
+    for _, batch := range batches {
+        for row, err := range batch.Rows() {
+            if err != nil {
+                return err
+            }
+            if err := callbackCtx.Err(); err != nil {
+                return err
+            }
+            fmt.Println(row.Timestamp, row.Values)
+        }
+    }
+    return nil
+}
+consumer, err := subscription.NewSubscriptionPushConsumer(cfg)
+if err != nil {
+    return err
+}
+if err := consumer.Open(ctx); err != nil {
+    return err
+}
+defer consumer.Close()
+if err := consumer.Subscribe(ctx, "measurements"); err != nil {
+    return err
+}
+select {
+case err, ok := <-consumer.Errors():
+    if !ok {
+        return nil
+    }
+    return err
+case <-ctx.Done():
+    return ctx.Err()
+}
+```
+
+- `AckBeforeConsume` acknowledges the whole batch before invoking any Handler. 
A partial or failed ACK prevents callbacks for that batch. A later Handler 
error or panic does not trigger NACK under this strategy.
+- `AckAfterConsume` runs all callbacks in order, then acknowledges the 
successful group and nacks the failed group. An ACK error skips the subsequent 
NACK request. A nil Handler result means success; an error or panic means 
consumption failed, not that the background loop has terminated.
+- Ordinary network failures are retried in a later polling round. TsFile 
decoding, fencing, and critical background errors stop delivery. `Errors()` 
buffers one result, sends the first terminal error once, and is closed by the 
loop when it exits. Normal closing does not invent a terminal error. A TsFile 
failure never enters the Handler or acknowledgment path.
+- Handlers run without an internal consumer operation gate and may call 
Subscribe, Seek, or Close. Push seek invalidates pending delivery for the 
target topic before sending the request. That topic's active callback context 
is canceled after the seek request finishes, whether it succeeds or fails, so a 
Handler can use its own context to call Seek without canceling that request at 
entry. An already-started callback can run until it returns; later callbacks 
and ACKs for the old delivery ge [...]
+- `Close` cancels the active callback context and synchronously closes core 
connection resources, but does not wait for user Handler code to return. This 
permits calling Close from inside a Handler without waiting on itself. The 
active message is released only after its callback returns, so borrowed data 
remains valid during that callback. Handlers should return promptly after 
cancellation. If a callback has not returned, the loop and `Errors()` channel 
may finish after Close returns.
+- Push messages are owned by the delivery loop. Do not call `Release` in a 
Handler; copy any record data that must outlive the callback. Watermarks update 
consumer state but do not reach record Handlers. Push does not expose the Pull 
processor pipeline.
+
 ## Management Sessions
 
 `SubscriptionSession` corresponds to Java's `ISubscriptionTreeSession` and 
`ISubscriptionTableSession`. Methods take an explicit `context.Context`, while 
parameter structures express existence conditions.
@@ -159,6 +214,8 @@ return consumer.Commit(ctx, messages...)
 
 | Go method or type | Java counterpart | Consolidation |
 | --- | --- | --- |
+| `NewSubscriptionPullConsumer(config)` | 
`SubscriptionTreePullConsumerBuilder.build`, 
`SubscriptionTablePullConsumerBuilder.build` | One constructor with `Dialect`; 
validates configuration without connecting |
+| `NewSubscriptionPushConsumer(config)` | 
`SubscriptionTreePushConsumerBuilder.build`, 
`SubscriptionTablePushConsumerBuilder.build` | One constructor with `Dialect` 
and a required Go Handler; validates configuration without connecting |
 | `SubscriptionConsumer.Open(ctx)`, `Close()` | Each Consumer's `open/close` | 
Shared lifecycle interface |
 | `Subscribe(ctx, topics...)`, `Unsubscribe(ctx, topics...)` | Single-topic, 
array, and collection overloads | Variadic topic names |
 | `Info()`, `SubscriptionConsumerInfo` | 
`getConsumerId/getConsumerGroupId/getOwnerId/getOwnerEpoch` | Returns a 
credential-free identity snapshot, including server-assigned IDs |
@@ -182,7 +239,7 @@ return consumer.Commit(ctx, messages...)
 
 `CommitAsync` returns a channel with capacity 1. It sends one result, nil on 
success, then closes. An unread result cannot block cleanup. Canceling the 
context does not undo commits accepted by the server, and a synchronous or 
asynchronous commit error does not mean that the whole batch failed.
 
-The `SubscriptionPushConsumer.Errors()` channel buffers at least one error. A 
TsFile decoding error must stop automatic polling before notification; the 
failed round cannot enter the Handler or ACK path. The lifecycle owner closes 
the channel after coordinating background senders.
+The `SubscriptionPushConsumer.Errors()` channel buffers one error. A TsFile 
decoding error stops automatic polling before notification; the failed round 
cannot enter the Handler or ACK path. The polling loop owns channel closure. If 
Close cancels a still-running Handler, channel closure can occur after Close 
returns, when that Handler returns and the loop finishes.
 
 ## Messages, Records, and Processors
 
@@ -211,8 +268,8 @@ A `SubscriptionMessageProcessor` that retains messages must 
implement `Buffering
 - Batches, columns, and row data from `SubscriptionMessage.Records()` are 
borrowed, read-only views valid until the message's `Release()`. Callers must 
copy values and byte slices that need to outlive that point.
 - `Release()` is idempotent. Do not reuse borrowed batch, column, or row data 
after release. `Records()` returns `ErrDataReleased`, while kind, commit 
context, and watermark remain available.
 - Nil in `Row.Values` means SQL NULL. DATE uses `time.Time`; TIMESTAMP and 
record timestamps use `int64` in server precision; BLOB/OBJECT use `[]byte`.
-- A Push Handler must not release its message. Copy record data before 
returning if it is needed afterward. The callback context is canceled when 
delivery stops.
-- The `Open(ctx)` context bounds opening only, not the background lifetime. 
Background work stops with `Close()`. Other operation contexts bound their 
corresponding operations only.
+- A Push Handler must not release its message. Copy record data before 
returning if it is needed afterward. The callback context is canceled when 
delivery stops or after a seek request for its topic finishes, regardless of 
that request's outcome. Close does not wait for the Handler; its message 
remains valid until the Handler returns.
+- The `Open(ctx)` context bounds opening only, not the background lifetime. 
Close stops new work and cancels active work. Other operation contexts bound 
their corresponding operations only; Push user code can finish after Close 
returns.
 - A positive Pull timeout bounds foreground queue waiting and network polling, 
subject to an earlier context deadline. Zero has the single-round behavior 
described above; processor callbacks must return promptly. Operation timeouts 
use `time.Duration` without changing message timestamp units.
 - Repeated Consumer `Open` while open and repeated `Close` after closing are 
harmless. A successfully closed Consumer cannot reopen. A failed close 
requiring buffered data to be drained preserves the drain-and-commit recovery 
path.
 
@@ -228,4 +285,4 @@ A `SubscriptionMessageProcessor` that retains messages must 
implement `Buffering
 
 Operation errors can wrap these sentinels; callers identify them with 
`errors.Is`. Unsupported TsFile errors must not trigger connection retries or 
file-payload acknowledgments. Pull returns an error and no messages for that 
round, including records obtained by parallel branches. That round must not 
advance positions or enter processor or automatic-commit buffers. Previously 
delivered records and confirmed progress are not rolled back. Callers can 
unsubscribe the offending topic and poll again.
 
-Remaining work includes Push consumers, built-in Watermark/ColumnAlign 
processors, further consumption examples, and behavior tests. Management 
queries reuse the root module's decoder; this step does not change its handling 
of malformed data. Verification has been static only: no compilation, tests, or 
formatting tools have run, and no runtime validation result is claimed.
+The remaining implementation step is the built-in Watermark/ColumnAlign 
processors; further consumption examples and behavior tests are also pending. 
Management queries reuse the root module's decoder; this step does not change 
its handling of malformed data. Verification has been static only: no 
compilation, tests, or formatting tools have run, and no runtime validation 
result is claimed.
diff --git a/subscription/config.go b/subscription/config.go
index 9039c4c..d1fe2fa 100644
--- a/subscription/config.go
+++ b/subscription/config.go
@@ -116,10 +116,12 @@ type SubscriptionPullConsumerConfig struct {
 type AckStrategy uint8
 
 const (
-       // AckAfterConsume acknowledges successful callbacks and nacks failed 
ones.
+       // AckAfterConsume runs callbacks sequentially, acknowledges the 
successful
+       // group, then nacks the failed group. An ACK error skips the 
subsequent NACK.
        AckAfterConsume AckStrategy = iota
-       // AckBeforeConsume acknowledges before invoking callbacks; callback 
failure
-       // does not cause redelivery under this strategy.
+       // AckBeforeConsume acknowledges the entire batch before invoking 
callbacks.
+       // A partial or failed ACK prevents callbacks for that batch; callback 
failure
+       // does not cause a NACK or redelivery under this strategy.
        AckBeforeConsume
 )
 
@@ -131,11 +133,14 @@ type SubscriptionPushConsumerConfig struct {
        // AckStrategy selects acknowledgment before delivery or after 
successful
        // processing. Its zero value is AckAfterConsume.
        AckStrategy AckStrategy
-       // Handler receives record messages. It is required; nil returned by a 
call
-       // means successful processing, and a non-nil error means processing 
failed.
+       // Handler receives record messages sequentially. It is required; nil 
returned
+       // by a call means success, while an error or panic means consumption 
failed.
+       // It must not release the borrowed message and should return promptly 
when
+       // its context is canceled. Close does not wait for Handler to return.
        Handler ConsumeFunc
-       // PollInterval controls the interval between automatic polling work. 
Java
-       // defaults to 100 milliseconds and requires an interval greater than 
zero.
+       // PollInterval is the fixed delay after one polling and delivery round
+       // finishes before another starts. Java defaults to 100 milliseconds and
+       // requires an interval greater than zero.
        PollInterval time.Duration
        // PollTimeout limits one automatic poll. Java defaults to ten seconds 
and
        // enforces a one-second minimum. Message timestamps retain server 
precision.
diff --git a/subscription/consumer.go b/subscription/consumer.go
index 23b5b3e..8097f6a 100644
--- a/subscription/consumer.go
+++ b/subscription/consumer.go
@@ -35,7 +35,9 @@ type SubscriptionConsumer interface {
        // work. It returns an error if opening fails. The context limits this
        // operation only; canceling it after Open returns does not stop 
consumption.
        Open(ctx context.Context) error
-       // Close stops background work and releases connections. In 
manual-commit
+       // Close stops background work and releases connections. Push cancels 
the
+       // current callback context but does not wait for the callback to 
return;
+       // that callback's message remains valid until it returns. In 
manual-commit
        // mode, buffered processor output must first be drained, processed, and
        // committed before retrying a close that reports undelivered messages.
        // Closing does not guarantee that every outstanding commit succeeded.
@@ -125,21 +127,39 @@ type SubscriptionPullConsumer interface {
 // ConsumeFunc handles a delivered record message. A nil error means success;
 // a non-nil error means failure. Acknowledgment and redelivery depend on the
 // configured AckStrategy rather than on the callback alone.
-// The callback context is canceled when the consumer stops delivery. The 
message
-// remains valid for the callback; the callback must not release it or retain
+// Callbacks run sequentially and without holding an internal consumer 
operation
+// gate. They may call Subscribe, Seek, or Close. A panic is treated as a 
failed
+// consumption attempt, not as a terminal background error.
+// The callback context is canceled when delivery stops or after a seek request
+// for its topic finishes, whether that request succeeds or fails. Close does
+// not wait for the callback, so it should return promptly when
+// its context is canceled. The message remains valid for the callback; the
+// callback must not release it or retain
 // borrowed record data after returning without copying that data.
 type ConsumeFunc func(ctx context.Context, message SubscriptionMessage) error
 
-// SubscriptionPushConsumer polls in the background and delivers records to 
SubscriptionPushConsumerConfig.Handler.
+// SubscriptionPushConsumer polls in the background and delivers records to
+// SubscriptionPushConsumerConfig.Handler. One fixed-delay loop invokes 
handlers
+// sequentially and does not install record processors. Before-consumption ACK
+// confirms the whole batch before any callback; a failed ACK prevents 
delivery.
+// After-consumption ACK confirms successful callbacks before nacking failed 
ones.
+// A failed ACK skips that batch's subsequent NACK request.
 // A TsFile decoding error stops automatic polling before it is reported; no
 // record callback or acknowledgment is performed for that failed poll round.
+// Seeking invalidates pending delivery for the target topic before its request
+// starts, then cancels that topic's active callback context after the request
+// finishes. No new callbacks or ACKs are started for invalidated records,
+// even if seek fails, because server positions might have changed partially.
+// Already accepted acknowledgments are not undone.
 type SubscriptionPushConsumer interface {
        SubscriptionConsumer
 
-       // Errors reports the first terminal background error, then delivery 
stops.
+       // Errors reports the first terminal background error after new 
delivery stops.
        // The channel has capacity at least one; the error is sent once even 
if the
-       // caller is not receiving. The lifecycle owner closes it after all 
senders
-       // have stopped. An ordinary callback failure is not a terminal error.
+       // caller is not receiving. The polling loop closes it when that loop 
exits.
+       // A Close during an active callback may return before this
+       // channel closes. An ordinary callback error or panic is not terminal;
+       // TsFile decoding, fencing, and critical background errors stop 
delivery.
        Errors() <-chan error
 }
 
diff --git a/subscription/doc.go b/subscription/doc.go
index 6f68007..9cb40cd 100644
--- a/subscription/doc.go
+++ b/subscription/doc.go
@@ -26,7 +26,9 @@
 // record views, message release, and watermark messages are implemented 
internally.
 // NewSubscriptionPullConsumer provides pull consumption, record continuation,
 // commit and seek coordination, background heartbeats, and provider recovery.
-// Push consumers and built-in processors remain subsequent implementation 
steps.
+// NewSubscriptionPushConsumer adds sequential callback delivery with before- 
or
+// after-consumption acknowledgment and a terminal-error channel. Built-in
+// processors remain a subsequent implementation step.
 //
 // Record messages are the supported payload format. Topic management and
 // subscription do not reject TsFile topics; decoding a TsFile payload must 
return
@@ -36,4 +38,7 @@
 // Contexts passed to Open limit the opening operation. Background work belongs
 // to the consumer and stops on Close, rather than when that context is 
canceled.
 // Other operation contexts bound the corresponding operation only.
+// Push Close cancels the active callback context and closes connection 
resources
+// without waiting for user code to return. Its borrowed message is released 
only
+// after that callback returns; the error channel can therefore close later.
 package subscription
diff --git a/subscription/pull_consumer.go b/subscription/pull_consumer.go
index b8ecbbc..78d081b 100644
--- a/subscription/pull_consumer.go
+++ b/subscription/pull_consumer.go
@@ -42,6 +42,16 @@ func NewSubscriptionPullConsumer(config 
SubscriptionPullConsumerConfig) (Subscri
        if err != nil {
                return nil, err
        }
+       consumer, err := newSubscriptionPullConsumer(resolved)
+       if err != nil {
+               return nil, err
+       }
+       return consumer, nil
+}
+
+// newSubscriptionPullConsumer accepts normalized configuration so push 
consumers
+// can reuse the connection and progress engine without parsing identities 
twice.
+func newSubscriptionPullConsumer(resolved SubscriptionPullConsumerConfig) 
(*subscriptionPullConsumer, error) {
        var suffix [8]byte
        if _, err := rand.Read(suffix[:]); err != nil {
                return nil, fmt.Errorf("subscription: generate consumer 
instance identity: %w", err)
diff --git a/subscription/push_consumer.go b/subscription/push_consumer.go
new file mode 100644
index 0000000..89f40e6
--- /dev/null
+++ b/subscription/push_consumer.go
@@ -0,0 +1,449 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package subscription
+
+import (
+       "context"
+       "errors"
+       "fmt"
+       "sync"
+       "time"
+)
+
+// NewSubscriptionPushConsumer copies and validates config without connecting.
+// Handler is required. Open starts sequential background delivery with the
+// configured acknowledgment strategy; its context limits opening only.
+// Close cancels delivery and closes connections without waiting for an active
+// Handler. Its message remains valid until that Handler returns.
+func NewSubscriptionPushConsumer(config SubscriptionPushConsumerConfig) 
(SubscriptionPushConsumer, error) {
+       resolved, err := normalizeSubscriptionPushConsumerConfig(config)
+       if err != nil {
+               return nil, err
+       }
+       autoCommit := false
+       consumer, err := 
newSubscriptionPullConsumer(SubscriptionPullConsumerConfig{
+               Consumer: resolved.Consumer,
+               AutoCommit: &autoCommit,
+               AutoCommitInterval: 
DefaultSubscriptionPullConsumerConfig().AutoCommitInterval,
+       })
+       if err != nil {
+               return nil, err
+       }
+       return &subscriptionPushConsumer{
+               config: resolved,
+               consumer: consumer,
+               operations: make(chan struct{}, 1),
+               generations: make(map[string]uint64),
+               errors: make(chan error, 1),
+       }, nil
+}
+
+type subscriptionPushConsumer struct {
+       config SubscriptionPushConsumerConfig
+       consumer *subscriptionPullConsumer
+       operations chan struct{}
+       mu sync.Mutex
+       generations map[string]uint64
+       started bool
+       closed bool
+       terminal error
+       deliveryCancel context.CancelFunc
+       callbackTopic string
+       callbackCancel context.CancelFunc
+       errors chan error
+       errorsClosed sync.Once
+}
+
+type subscriptionPushDelivery struct {
+       message SubscriptionMessage
+       generation uint64
+}
+
+func (c *subscriptionPushConsumer) Open(ctx context.Context) error {
+       if ctx == nil {
+               return errors.New("subscription: context must not be nil")
+       }
+       if err := ctx.Err(); err != nil {
+               return err
+       }
+       c.mu.Lock()
+       err := c.deliveryStateLocked()
+       c.mu.Unlock()
+       if err != nil {
+               return err
+       }
+       if err := c.consumer.Open(ctx); err != nil {
+               return err
+       }
+       c.mu.Lock()
+       if err := c.deliveryStateLocked(); err != nil {
+               c.mu.Unlock()
+               return err
+       }
+       if c.started {
+               c.mu.Unlock()
+               return nil
+       }
+       c.consumer.mu.RLock()
+       lifeCtx := c.consumer.lifeCtx
+       c.consumer.mu.RUnlock()
+       deliveryCtx, cancel := context.WithCancel(lifeCtx)
+       c.deliveryCancel = cancel
+       c.started = true
+       c.mu.Unlock()
+       go c.runDelivery(deliveryCtx)
+       return nil
+}
+
+func (c *subscriptionPushConsumer) Close() error {
+       c.mu.Lock()
+       if c.closed {
+               c.mu.Unlock()
+               return nil
+       }
+       c.closed = true
+       if c.deliveryCancel != nil {
+               c.deliveryCancel()
+       }
+       started := c.started
+       c.mu.Unlock()
+       // The loop is not a core worker: a Handler may close its own consumer.
+       // It retains ownership of its messages until the Handler has returned.
+       err := c.consumer.Close()
+       if !started {
+               c.closeErrors()
+       }
+       return err
+}
+
+func (c *subscriptionPushConsumer) Errors() <-chan error {
+       return c.errors
+}
+
+func (c *subscriptionPushConsumer) Info() SubscriptionConsumerInfo {
+       return c.consumer.Info()
+}
+
+func (c *subscriptionPushConsumer) LatestWatermark() (int64, bool) {
+       return c.consumer.LatestWatermark()
+}
+
+func (c *subscriptionPushConsumer) AllConsumed() bool {
+       return c.consumer.AllConsumed()
+}
+
+func (c *subscriptionPushConsumer) Subscribe(ctx context.Context, topics 
...string) error {
+       return c.consumer.Subscribe(ctx, topics...)
+}
+
+func (c *subscriptionPushConsumer) Unsubscribe(ctx context.Context, topics 
...string) error {
+       return c.consumer.Unsubscribe(ctx, topics...)
+}
+
+func (c *subscriptionPushConsumer) Positions(topic string) (TopicProgress, 
error) {
+       return c.consumer.Positions(topic)
+}
+
+func (c *subscriptionPushConsumer) CommittedPositions(topic string) 
(TopicProgress, error) {
+       return c.consumer.CommittedPositions(topic)
+}
+
+func (c *subscriptionPushConsumer) SeekToBeginning(ctx context.Context, topic 
string) error {
+       return c.seek(ctx, topic, subscriptionSeekBeginning, nil)
+}
+
+func (c *subscriptionPushConsumer) SeekToEnd(ctx context.Context, topic 
string) error {
+       return c.seek(ctx, topic, subscriptionSeekEnd, nil)
+}
+
+func (c *subscriptionPushConsumer) Seek(ctx context.Context, topic string, 
progress TopicProgress) error {
+       return c.seek(ctx, topic, subscriptionSeekProgress, progress)
+}
+
+func (c *subscriptionPushConsumer) SeekAfter(ctx context.Context, topic 
string, progress TopicProgress) error {
+       return c.seek(ctx, topic, subscriptionSeekAfterProgress, progress)
+}
+
+func (c *subscriptionPushConsumer) seek(ctx context.Context, topic string, 
kind int16, progress TopicProgress) error {
+       name, err := parseSubscriptionIdentifier(topic)
+       if err != nil {
+               return err
+       }
+       if err := c.acquireOperations(ctx); err != nil {
+               return err
+       }
+       defer c.releaseOperations()
+       if err := c.consumer.checkOpen(); err != nil {
+               return err
+       }
+       if err := ctx.Err(); err != nil {
+               return err
+       }
+       c.mu.Lock()
+       if c.closed {
+               c.mu.Unlock()
+               return ErrClosed
+       }
+       // Even a failed multi-provider seek can change server state. Never 
confirm
+       // or start another callback for an older delivery of the affected 
topic.
+       c.generations[name]++
+       var cancelCallback context.CancelFunc
+       if c.callbackTopic == name {
+               cancelCallback = c.callbackCancel
+       }
+       c.mu.Unlock()
+       // Keep a callback's context usable for a seek invoked from that 
callback.
+       // Cancel the obsolete delivery after the request completes instead.
+       if cancelCallback != nil {
+               defer cancelCallback()
+       }
+       return c.consumer.seekProgress(ctx, topic, kind, progress)
+}
+
+func (c *subscriptionPushConsumer) acquireOperations(ctx context.Context) 
error {
+       if ctx == nil {
+               return errors.New("subscription: context must not be nil")
+       }
+       if err := ctx.Err(); err != nil {
+               return err
+       }
+       select {
+       case c.operations <- struct{}{}:
+               return nil
+       case <-ctx.Done():
+               return ctx.Err()
+       }
+}
+
+func (c *subscriptionPushConsumer) releaseOperations() {
+       <-c.operations
+}
+
+func (c *subscriptionPushConsumer) deliveryStateLocked() error {
+       if c.closed {
+               return ErrClosed
+       }
+       return c.terminal
+}
+
+func (c *subscriptionPushConsumer) pollDeliveries(ctx context.Context) 
([]subscriptionPushDelivery, error) {
+       if err := c.acquireOperations(ctx); err != nil {
+               return nil, err
+       }
+       defer c.releaseOperations()
+       c.mu.Lock()
+       err := c.deliveryStateLocked()
+       c.mu.Unlock()
+       if err != nil {
+               return nil, err
+       }
+       messages, err := c.consumer.Poll(ctx, c.config.PollTimeout)
+       if err != nil {
+               return nil, err
+       }
+       c.mu.Lock()
+       defer c.mu.Unlock()
+       if err := c.deliveryStateLocked(); err != nil {
+               for _, message := range messages {
+                       message.Release()
+               }
+               return nil, err
+       }
+       deliveries := make([]subscriptionPushDelivery, 0, len(messages))
+       for _, message := range messages {
+               deliveries = append(deliveries, subscriptionPushDelivery{
+                       message: message,
+                       generation: 
c.generations[message.CommitContext().Topic],
+               })
+       }
+       return deliveries, nil
+}
+
+func (c *subscriptionPushConsumer) runDelivery(ctx context.Context) {
+       var terminal error
+       defer func() { c.finishDelivery(terminal) }()
+       for {
+               if err := ctx.Err(); err != nil {
+                       terminal = c.terminalDeliveryError(err)
+                       return
+               }
+               deliveries, err := c.pollDeliveries(ctx)
+               if err == nil {
+                       err = c.deliver(ctx, deliveries)
+               }
+               if err != nil {
+                       terminal = c.terminalDeliveryError(err)
+                       if terminal != nil || ctx.Err() != nil {
+                               return
+                       }
+               }
+               // Wait after the entire round, including callbacks and 
acknowledgment.
+               // A slow round therefore never overlaps the next round.
+               timer := time.NewTimer(c.config.PollInterval)
+               select {
+               case <-ctx.Done():
+                       timer.Stop()
+                       terminal = c.terminalDeliveryError(ctx.Err())
+                       return
+               case <-timer.C:
+               }
+       }
+}
+
+func (c *subscriptionPushConsumer) deliver(ctx context.Context, deliveries 
[]subscriptionPushDelivery) error {
+       defer func() {
+               for _, delivery := range deliveries {
+                       delivery.message.Release()
+               }
+       }()
+       if c.config.AckStrategy == AckBeforeConsume {
+               if err := c.commitDeliveries(ctx, deliveries, false); err != 
nil {
+                       return err
+               }
+       }
+       var succeeded, failed []subscriptionPushDelivery
+       for _, delivery := range deliveries {
+               if err := ctx.Err(); err != nil {
+                       return err
+               }
+               callbackCtx, finish, current := c.beginCallback(ctx, delivery)
+               if !current {
+                       continue
+               }
+               err := c.invokeHandler(callbackCtx, delivery.message)
+               finish()
+               delivery.message.Release()
+               if err == nil {
+                       succeeded = append(succeeded, delivery)
+               } else {
+                       failed = append(failed, delivery)
+               }
+       }
+       if c.config.AckStrategy == AckBeforeConsume {
+               return nil
+       }
+       // Java confirms successful callbacks before nacking failures. An ACK 
error
+       // leaves the failed group unconfirmed for server-side recovery.
+       if err := c.commitDeliveries(ctx, succeeded, false); err != nil {
+               return err
+       }
+       return c.commitDeliveries(ctx, failed, true)
+}
+
+func (c *subscriptionPushConsumer) beginCallback(ctx context.Context, delivery 
subscriptionPushDelivery) (context.Context, func(), bool) {
+       topic := delivery.message.CommitContext().Topic
+       c.mu.Lock()
+       defer c.mu.Unlock()
+       if c.deliveryStateLocked() != nil || ctx.Err() != nil || 
c.generations[topic] != delivery.generation {
+               return nil, nil, false
+       }
+       callbackCtx, cancel := context.WithCancel(ctx)
+       c.callbackTopic = topic
+       c.callbackCancel = cancel
+       return callbackCtx, func() {
+               cancel()
+               c.mu.Lock()
+               c.callbackTopic = ""
+               c.callbackCancel = nil
+               c.mu.Unlock()
+       }, true
+}
+
+func (c *subscriptionPushConsumer) invokeHandler(ctx context.Context, message 
SubscriptionMessage) (err error) {
+       defer func() {
+               if failure := recover(); failure != nil {
+                       err = fmt.Errorf("subscription: push handler panicked: 
%v", failure)
+               }
+       }()
+       return c.config.Handler(ctx, message)
+}
+
+func (c *subscriptionPushConsumer) commitDeliveries(ctx context.Context, 
deliveries []subscriptionPushDelivery, nack bool) error {
+       if len(deliveries) == 0 {
+               return nil
+       }
+       if err := c.acquireOperations(ctx); err != nil {
+               return err
+       }
+       defer c.releaseOperations()
+       c.mu.Lock()
+       if err := c.deliveryStateLocked(); err != nil {
+               c.mu.Unlock()
+               return err
+       }
+       messages := make([]SubscriptionMessage, 0, len(deliveries))
+       for _, delivery := range deliveries {
+               if c.generations[delivery.message.CommitContext().Topic] == 
delivery.generation {
+                       messages = append(messages, delivery.message)
+               }
+       }
+       c.mu.Unlock()
+       if len(messages) == 0 {
+               return nil
+       }
+       commitCtx, cancel := context.WithTimeout(ctx, 
c.config.Consumer.Connection.ConnectTimeout)
+       defer cancel()
+       operationCtx, finish, err := c.consumer.linkContext(commitCtx)
+       if err != nil {
+               return err
+       }
+       defer finish()
+       contexts, err := c.consumer.messageCommitContexts(messages)
+       if err != nil {
+               return err
+       }
+       return c.consumer.commitContexts(operationCtx, contexts, nack)
+}
+
+func (c *subscriptionPushConsumer) terminalDeliveryError(failure error) error {
+       c.mu.Lock()
+       closed := c.closed
+       c.mu.Unlock()
+       if closed {
+               return nil
+       }
+       if err := c.consumer.checkOpen(); err != nil {
+               return err
+       }
+       if flowFatalPollError(failure) || flowCriticalPollError(failure) {
+               return failure
+       }
+       return nil
+}
+
+func (c *subscriptionPushConsumer) finishDelivery(failure error) {
+       c.mu.Lock()
+       notify := failure != nil && !c.closed
+       if notify {
+               c.terminal = failure
+       }
+       if c.deliveryCancel != nil {
+               c.deliveryCancel()
+       }
+       c.mu.Unlock()
+       if notify {
+               c.errors <- failure
+       }
+       c.closeErrors()
+}
+
+func (c *subscriptionPushConsumer) closeErrors() {
+       c.errorsClosed.Do(func() { close(c.errors) })
+}

Reply via email to