This is an automated email from the ASF dual-hosted git repository.

shuwenwei pushed a commit to branch supports_subscription
in repository https://gitbox.apache.org/repos/asf/iotdb-client-go.git

commit 9c11a320cc07add4dff80333c110096a0df6c813
Author: shuwenwei <[email protected]>
AuthorDate: Mon Nov 16 18:42:48 2026 +0800

    Optimize subscription data-plane hot paths
---
 subscription/README.md                           |   8 +-
 subscription/consumer/pull_consumer.go           |  61 +++-
 subscription/consumer/pull_flow.go               | 226 +++++++------
 subscription/consumer/pull_progress.go           | 155 +++++++--
 subscription/consumer/push_consumer.go           |   7 +-
 subscription/internal/protocol/tablet.go         |  53 +--
 subscription/internal/record/message.go          |  60 ++--
 subscription/internal/record/mutate.go           | 245 ++++++++++++++
 subscription/internal/record/tablet.go           | 348 +++++++++++++++++++
 subscription/processor/column_align_processor.go | 411 +++++++++++++++++------
 subscription/processor/watermark_processor.go    |  47 ++-
 11 files changed, 1301 insertions(+), 320 deletions(-)

diff --git a/subscription/README.md b/subscription/README.md
index c03c95d..c17f922 100644
--- a/subscription/README.md
+++ b/subscription/README.md
@@ -177,7 +177,7 @@ return consumer.Commit(ctx, messages...)
 
 - A positive Poll timeout starts at entry and covers waiting for the 
serialized foreground gate, polling, and termination unsubscribe cleanup. 
Processor callbacks must return promptly. Negative timeouts return argument 
errors. A zero timeout does not wait for a busy foreground gate or perform 
synchronous provider recovery; it sends one round of requests to available 
Providers with server timeout 0. Network I/O can still wait and is bounded by 
the caller's context. Caller context cancella [...]
 - Poll work is serial even when `MaxPollParallelism` is greater than one, 
matching the current Java client. TsFile or current-consumer fencing makes the 
entire round fail without passing records to processors, automatic commit 
tracking, or fetch positions. Custom processor code must still avoid blocking 
indefinitely.
-- Heartbeats renew records buffered by Pull processors. Automatic commits 
track only records delivered to the caller and wait at least one configured 
interval. Manual `Commit` can run concurrently with Poll; `CommitAsync` reports 
its result once through a buffered channel.
+- Heartbeats renew records buffered by Pull processors. Automatic commits 
track only records delivered to the caller and use an expiry heap so each tick 
processes due entries instead of scanning the full backlog. Manual `Commit` can 
run concurrently with Poll; `CommitAsync` reports its result once through a 
buffered channel.
 - Commit and seek server requests and local progress updates are mutually 
exclusive. Partial acceptance is applied first and accepted contexts leave the 
pending queue; only the remainder is retried. NACKs are not replayed. A writer 
ACK pending after node loss is redirected only after the same Topic/Region 
reappears through a healthy node.
 - Heartbeats are scheduled independently per Provider. Endpoint 
synchronization and connection failure trigger recovery. Background recovery 
attempts are at least five seconds apart. Late old-connection responses cannot 
reactivate an old Provider or incorrectly fence the replacement connection's 
Consumer.
 - `Close` interrupts blocked polling and stops background work. In manual 
mode, buffered processor messages make closing return an error while preserving 
the drain path. Call `DrainBufferedMessages`, process and commit its records, 
then close again. Automatic mode makes a best-effort commit of delivered and 
drained records during closing. Commit failures return errors; a successful 
close does not imply that all historical messages were processed.
@@ -347,11 +347,11 @@ if err := consumer.AddProcessor(alignment); err != nil {
 
 Messages are ordered by their maximum raw timestamp and emitted whole when 
that maximum is at most the watermark. This does not globally sort rows across 
messages or batches. Equal message maxima have no guaranteed order. Capacity 
overflow force-flushes all remaining messages after normal emission. A timeout 
force-flushes when no message was emitted and the interval since the previous 
emission has elapsed. When every source is stale, the previous watermark 
remains in effect. New sources  [...]
 
-`Buffered()` snapshots the message count and every committable context for 
heartbeat renewal. `Flush()` transfers buffered records in maximum-timestamp 
order without clearing source progress. `Reset()` releases all retained records 
and clears ordering state; `ResetTopic(topic)` releases only that topic and 
recomputes ordering from the remaining sources. Reset discards unconfirmed 
records without acknowledging them. `Watermark()` returns `ok == false` before 
progress is known or when rese [...]
+`Buffered()` snapshots the message count and every committable context for 
heartbeat renewal. `Flush()` transfers buffered records in maximum-timestamp 
order without clearing source progress. `Reset()` releases all retained records 
and clears ordering state; `ResetTopic(topic)` releases only that topic and 
recomputes ordering from the remaining sources. Reset discards unconfirmed 
records without acknowledging them. `Watermark()` returns `ok == false` before 
progress is known or when rese [...]
 
-`ColumnAlignProcessor` preserves raw row order, as Java does. It fills a NULL 
only from a cached non-null value with a strictly earlier timestamp; a non-null 
observation updates the cache at equal or greater timestamps. Columns are 
identified by name and data type, so reordered columns remain independent and 
changed types cannot reuse incompatible values. Tree state is isolated by topic 
and device path. Table state is isolated by topic, database, table name, and 
ordered TAG values per ro [...]
+`ColumnAlignProcessor` preserves raw row order, as Java does. It fills a NULL 
only from a cached non-null value with a strictly earlier timestamp; a non-null 
observation updates the cache at equal or greater timestamps. Columns are 
identified by name and data type, so reordered columns remain independent and 
changed types cannot reuse incompatible values. Tree state is isolated by topic 
and device path. Table state is isolated by topic, database, table name, and 
ordered TAG values per ro [...]
 
-Alignment reads inputs through the existing public RecordBatch API, copies 
record buffers, and returns replacement messages with the same commit contexts 
and projection. It releases consumed originals only after all input access 
succeeds. Input errors and dialect mismatches leave inputs and cache unchanged. 
Cache byte values and filled byte values are independently copied so earlier 
Release calls cannot invalidate later records. Watermarks pass through 
unchanged. It buffers no messages,  [...]
+Decoded subscription records use typed column storage and are aligned in place 
under one batch lock; their buffer ownership is transferred to the returned 
messages. User-provided RecordBatch implementations use a copying compatibility 
path. Input errors and dialect mismatches leave cache and ownership unchanged. 
Cache byte values and filled byte values are independently copied so Release 
calls cannot invalidate cached data. Watermarks pass through unchanged. It 
buffers no messages, so Fl [...]
 
 ## Data Ownership and Context Contracts
 
diff --git a/subscription/consumer/pull_consumer.go 
b/subscription/consumer/pull_consumer.go
index 4b16ee1..0a59eaf 100644
--- a/subscription/consumer/pull_consumer.go
+++ b/subscription/consumer/pull_consumer.go
@@ -84,6 +84,7 @@ func newSubscriptionPullConsumer(resolved 
SubscriptionPullConsumerConfig) (*subs
                positions: make(map[string]payload.TopicProgress),
                committed: make(map[string]payload.TopicProgress),
                uncommitted: 
make(map[subscriptionCommitKey]subscriptionPendingCommit),
+               uncommittedDeadlines: make(subscriptionCommitDeadlineHeap, 0),
                redirectAcks: 
make(map[subscriptionCommitKey]payload.CommitContext),
                identity: SubscriptionConsumerInfo{
                        ConsumerID: resolved.Consumer.ConsumerID,
@@ -125,6 +126,7 @@ type subscriptionPullConsumer struct {
        positions map[string]payload.TopicProgress
        committed map[string]payload.TopicProgress
        uncommitted map[subscriptionCommitKey]subscriptionPendingCommit
+       uncommittedDeadlines subscriptionCommitDeadlineHeap
        redirectAcks map[subscriptionCommitKey]payload.CommitContext
        watermark int64
        hasWatermark bool
@@ -357,7 +359,7 @@ func (c *subscriptionPullConsumer) Close() error {
        defer cancel()
        var failures []error
        if opened && fenced == nil && *c.config.AutoCommit {
-               if err := c.commitContexts(ctx, c.pendingCommitContexts(true, 
time.Now()), false); err != nil {
+               if err := c.commitUniqueContexts(ctx, 
c.pendingCommitContexts(true, time.Now()), false); err != nil {
                        failures = append(failures, err)
                }
        }
@@ -501,6 +503,59 @@ func (c *subscriptionPullConsumer) changeSubscriptions(ctx 
context.Context, topi
        return errors.Join(failures...)
 }
 
+type subscriptionProviderSet struct {
+       single   *subscriptionProvider
+       multiple []*subscriptionProvider
+}
+
+func (p subscriptionProviderSet) Len() int {
+       if p.single != nil {
+               return 1
+       }
+       return len(p.multiple)
+}
+
+func (p subscriptionProviderSet) Provider(index int) *subscriptionProvider {
+       if p.single != nil {
+               if index == 0 {
+                       return p.single
+               }
+               return nil
+       }
+       if index < 0 || index >= len(p.multiple) {
+               return nil
+       }
+       return p.multiple[index]
+}
+
+func (c *subscriptionPullConsumer) pollProviderSet() subscriptionProviderSet {
+       c.mu.RLock()
+       defer c.mu.RUnlock()
+       var single *subscriptionProvider
+       count := 0
+       for _, provider := range c.providers {
+               if provider.connection.Closed() {
+                       continue
+               }
+               single = provider
+               count++
+       }
+       if count == 1 {
+               return subscriptionProviderSet{single: single}
+       }
+       if count == 0 {
+               return subscriptionProviderSet{}
+       }
+       providers := make([]*subscriptionProvider, 0, count)
+       for _, provider := range c.providers {
+               if !provider.connection.Closed() {
+                       providers = append(providers, provider)
+               }
+       }
+       sort.Slice(providers, func(i, j int) bool { return 
providers[i].dataNodeID < providers[j].dataNodeID })
+       return subscriptionProviderSet{multiple: providers}
+}
+
 func (c *subscriptionPullConsumer) providerSnapshot() []*subscriptionProvider {
        c.mu.RLock()
        defer c.mu.RUnlock()
@@ -508,7 +563,9 @@ func (c *subscriptionPullConsumer) providerSnapshot() 
[]*subscriptionProvider {
        for _, provider := range c.providers {
                providers = append(providers, provider)
        }
-       sort.Slice(providers, func(i, j int) bool { return 
providers[i].dataNodeID < providers[j].dataNodeID })
+       if len(providers) > 1 {
+               sort.Slice(providers, func(i, j int) bool { return 
providers[i].dataNodeID < providers[j].dataNodeID })
+       }
        return providers
 }
 
diff --git a/subscription/consumer/pull_flow.go 
b/subscription/consumer/pull_flow.go
index e0a832d..07e25ae 100644
--- a/subscription/consumer/pull_flow.go
+++ b/subscription/consumer/pull_flow.go
@@ -87,38 +87,53 @@ func (c *subscriptionPullConsumer) beginFlowPoll(ctx 
context.Context, timeout ti
                operationCtx, finish, acquired, err = 
c.beginNonblockingOperation(ctx)
                return operationCtx, operationCtx, finish, acquired, err
        }
-       queueCtx, cancelQueue := context.WithTimeout(ctx, timeout)
-       operationCtx, release, err := c.beginQueuedOperation(ctx, queueCtx)
-       if err != nil {
-               cancelQueue()
-               if stateErr := c.checkOpen(); stateErr != nil {
-                       return nil, nil, nil, false, stateErr
-               }
-               if callerErr := ctx.Err(); callerErr != nil {
-                       return nil, nil, nil, false, callerErr
-               }
-               if errors.Is(err, context.DeadlineExceeded) {
-                       return nil, nil, nil, false, nil
-               }
+       if err := ctx.Err(); err != nil {
                return nil, nil, nil, false, err
        }
-       if queueCtx.Err() != nil {
-               release()
-               cancelQueue()
+       c.mu.RLock()
+       lifeCtx := c.lifeCtx
+       stateErr := c.stateErrorLocked(false)
+       c.mu.RUnlock()
+       if stateErr != nil {
+               return nil, nil, nil, false, stateErr
+       }
+       operationCtx, cancelOperation := context.WithCancel(ctx)
+       stop := context.AfterFunc(lifeCtx, cancelOperation)
+       pollCtx, cancelPoll := context.WithTimeout(operationCtx, timeout)
+       if err := c.acquireFlow(pollCtx); err != nil {
+               stop()
+               cancelPoll()
+               cancelOperation()
                if stateErr := c.checkOpen(); stateErr != nil {
                        return nil, nil, nil, false, stateErr
                }
                if callerErr := ctx.Err(); callerErr != nil {
                        return nil, nil, nil, false, callerErr
                }
+               if operationCtx.Err() != nil {
+                       return nil, nil, nil, false, operationCtx.Err()
+               }
                return nil, nil, nil, false, nil
        }
-       deadline, _ := queueCtx.Deadline()
-       pollCtx, cancelPoll := context.WithDeadline(operationCtx, deadline)
+       c.mu.Lock()
+       if stateErr := c.stateErrorLocked(false); stateErr != nil {
+               c.mu.Unlock()
+               c.releaseFlow()
+               stop()
+               cancelPoll()
+               cancelOperation()
+               return nil, nil, nil, false, stateErr
+       }
+       c.flowCancel = cancelOperation
+       c.mu.Unlock()
        return operationCtx, pollCtx, func() {
+               stop()
                cancelPoll()
-               cancelQueue()
-               release()
+               cancelOperation()
+               c.mu.Lock()
+               c.flowCancel = nil
+               c.mu.Unlock()
+               c.releaseFlow()
        }, true, nil
 }
 
@@ -137,13 +152,13 @@ func (c *subscriptionPullConsumer) 
pollLocked(operationCtx, pollCtx context.Cont
        if len(selected) == 0 {
                return nil, nil
        }
-       networkReady := true
-       if !hasAvailableSubscriptionProvider(c.providerSnapshot()) {
+       providers := c.pollProviderSet()
+       networkReady := providers.Len() != 0
+       if !networkReady {
                if immediate {
                        // A zero-wait poll can drive existing processor state, 
but must
                        // not turn into a synchronous connection-recovery 
attempt.
                        c.requestRecovery()
-                       networkReady = false
                } else if err := c.recoverProviders(pollCtx); err != nil {
                        if stateErr := c.checkOpen(); stateErr != nil {
                                return nil, stateErr
@@ -155,37 +170,36 @@ func (c *subscriptionPullConsumer) 
pollLocked(operationCtx, pollCtx context.Cont
                        } else {
                                return nil, err
                        }
+               } else {
+                       providers = c.pollProviderSet()
+                       networkReady = providers.Len() != 0
                }
        }
-       var results []flowPollResult
+       result := flowPollResult{}
        if networkReady {
-               results = c.pollFlowPartitions(pollCtx, selected, immediate)
-       }
-       var messages []payload.SubscriptionMessage
-       var emptyContexts, failedContexts, allContexts []payload.CommitContext
-       var terminations []string
+               cursor := c.pollCursor
+               c.pollCursor++
+               result = c.pollFlowPartition(pollCtx, selected, cursor, 
immediate, providers)
+       }
+       allContexts := make([]payload.CommitContext, 0, 
len(result.rollbackContexts)+len(result.nackContexts))
+       allContexts = append(allContexts, result.rollbackContexts...)
+       allContexts = append(allContexts, result.nackContexts...)
+       failedContexts := make([]payload.CommitContext, 
len(result.nackContexts))
+       copy(failedContexts, result.nackContexts)
        var fatal, critical, ordinary error
-       for _, result := range results {
-               allContexts = append(allContexts, result.rollbackContexts...)
-               allContexts = append(allContexts, result.nackContexts...)
-               failedContexts = append(failedContexts, result.nackContexts...)
-               if result.err != nil {
-                       failedContexts = append(failedContexts, 
result.rollbackContexts...)
-                       if flowFatalPollError(result.err) {
-                               if fatal == nil {
-                                       fatal = result.err
-                               }
-                       } else if flowCriticalPollError(result.err) {
-                               critical = result.err
-                       } else if !((errors.Is(result.err, 
context.DeadlineExceeded) || errors.Is(result.err, context.Canceled)) && 
pollCtx.Err() != nil) {
-                               ordinary = result.err
-                       }
-                       continue
-               }
-               messages = append(messages, result.messages...)
-               emptyContexts = append(emptyContexts, result.emptyContexts...)
-               terminations = append(terminations, result.terminated...)
-       }
+       if result.err != nil {
+               failedContexts = append(failedContexts, 
result.rollbackContexts...)
+               if flowFatalPollError(result.err) {
+                       fatal = result.err
+               } else if flowCriticalPollError(result.err) {
+                       critical = result.err
+               } else if !((errors.Is(result.err, context.DeadlineExceeded) || 
errors.Is(result.err, context.Canceled)) && pollCtx.Err() != nil) {
+                       ordinary = result.err
+               }
+       }
+       messages := result.messages
+       emptyContexts := result.emptyContexts
+       terminations := result.terminated
        if err := c.checkOpen(); err != nil {
                c.rollbackFlow(pollCtx, allContexts)
                return nil, err
@@ -214,9 +228,8 @@ func (c *subscriptionPullConsumer) pollLocked(operationCtx, 
pollCtx context.Cont
                c.mu.Unlock()
        }
        // The caller/lifetime context aborts the entire round. Exhausting only 
the
-       // polling budget preserves complete healthy partitions, as Java does 
for
-       // completed sibling futures. Do not start cleanup RPCs after that 
budget has
-       // expired; undelivered events remain unconfirmed.
+       // polling budget leaves undelivered events unconfirmed; no cleanup RPC 
is
+       // started after that budget has expired.
        c.rollbackFlow(pollCtx, failedContexts)
        if pollCtx.Err() == nil && len(emptyContexts) != 0 {
                if err := c.commitContexts(pollCtx, emptyContexts, false); err 
!= nil {
@@ -252,7 +265,9 @@ func (c *subscriptionPullConsumer) pollLocked(operationCtx, 
pollCtx context.Cont
                        return nil, err
                }
        }
-       c.refreshBufferedContexts()
+       if len(c.processors) != 0 || len(c.drained) != 0 {
+               c.refreshBufferedContexts()
+       }
        if err := operationCtx.Err(); err != nil {
                c.rollbackFlow(pollCtx, allContexts)
                return nil, err
@@ -273,8 +288,11 @@ func (c *subscriptionPullConsumer) 
pollLocked(operationCtx, pollCtx context.Cont
        // Fetch progress can include records retained by a successful 
processor, but
        // automatic commit only tracks the records actually handed to the 
caller.
        c.advancePositions(messages)
-       // Observe raw watermark signals even if a processor consumed them.
-       _ = c.visibleFlowMessages(messages)
+       // Observe raw watermark signals even if a processor consumed them. 
Without
+       // processors, one pass over the same batch is sufficient.
+       if len(c.processors) != 0 {
+               _ = c.visibleFlowMessages(messages)
+       }
        processed = c.visibleFlowMessages(processed)
        c.trackDelivered(processed)
        return processed, nil
@@ -320,39 +338,9 @@ func uniqueFlowTopics(topics []string) []string {
        return names
 }
 
-func (c *subscriptionPullConsumer) pollFlowPartitions(ctx context.Context, 
topics []string, immediate bool) []flowPollResult {
-       parallelism := 1
-       partitions := make([][]string, parallelism)
-       for index, topic := range topics {
-               partitions[index%parallelism] = 
append(partitions[index%parallelism], topic)
-       }
-       cursor := c.pollCursor
-       c.pollCursor++
-       partitionCtx, cancel := context.WithCancel(ctx)
-       defer cancel()
-       type partitionResult struct {
-               index int
-               result flowPollResult
-       }
-       completed := make(chan partitionResult, parallelism)
-       for index, partition := range partitions {
-               go func(index int, topics []string) {
-                       completed <- partitionResult{index: index, result: 
c.pollFlowPartition(partitionCtx, topics, cursor+uint64(index), immediate)}
-               }(index, partition)
-       }
-       results := make([]flowPollResult, parallelism)
-       for range partitions {
-               result := <-completed
-               results[result.index] = result.result
-               if flowFatalPollError(result.result.err) {
-                       cancel()
-               }
-       }
-       return results
-}
-
-func (c *subscriptionPullConsumer) pollFlowPartition(ctx context.Context, 
topics []string, cursor uint64, immediate bool) (result flowPollResult) {
+func (c *subscriptionPullConsumer) pollFlowPartition(ctx context.Context, 
topics []string, cursor uint64, immediate bool, providers 
subscriptionProviderSet) (result flowPollResult) {
        var lastFailure error
+       refreshProviders := false
        for {
                if err := ctx.Err(); err != nil {
                        result.err = err
@@ -362,14 +350,11 @@ func (c *subscriptionPullConsumer) pollFlowPartition(ctx 
context.Context, topics
                        result.err = err
                        return result
                }
-               providers := c.providerSnapshot()
-               available := providers[:0]
-               for _, provider := range providers {
-                       if !provider.connection.Closed() {
-                               available = append(available, provider)
-                       }
+               if refreshProviders {
+                       providers = c.pollProviderSet()
+                       refreshProviders = false
                }
-               if len(available) == 0 {
+               if providers.Len() == 0 {
                        c.requestRecovery()
                        if lastFailure == nil {
                                lastFailure = errors.New("subscription: no 
available provider for poll")
@@ -377,18 +362,27 @@ func (c *subscriptionPullConsumer) pollFlowPartition(ctx 
context.Context, topics
                        result.err = lastFailure
                        return result
                }
-               progress := make(map[string]payload.TopicProgress, len(topics))
+               var progress map[string]payload.TopicProgress
                c.mu.RLock()
                for _, topic := range topics {
                        if position := c.positions[topic]; len(position) != 0 {
+                               if progress == nil {
+                                       progress = 
make(map[string]payload.TopicProgress, len(topics))
+                               }
                                progress[topic] = cloneTopicProgress(position)
                        }
                }
                c.mu.RUnlock()
                healthy := false
-               for attempt := range available {
-                       provider := 
available[(int(cursor%uint64(len(available)))+attempt)%len(available)]
-                       responses, err := provider.poll(ctx, topics, 
flowRemainingMilliseconds(ctx, len(available)-attempt, immediate), 
int64(provider.bodyLimit), progress)
+               attempted := 0
+               for attempt := 0; attempt < providers.Len(); attempt++ {
+                       provider := 
providers.Provider((int(cursor%uint64(providers.Len())) + attempt) % 
providers.Len())
+                       if provider == nil || provider.connection.Closed() {
+                               refreshProviders = true
+                               continue
+                       }
+                       attempted++
+                       responses, err := provider.poll(ctx, topics, 
flowRemainingMilliseconds(ctx, providers.Len()-attempt, immediate), 
int64(provider.bodyLimit), progress)
                        if err != nil {
                                err = c.handleProviderError(provider, err)
                                if flowFatalPollError(err) || 
flowCriticalPollError(err) || ctx.Err() != nil {
@@ -396,6 +390,7 @@ func (c *subscriptionPullConsumer) pollFlowPartition(ctx 
context.Context, topics
                                        return result
                                }
                                lastFailure = err
+                               refreshProviders = true
                                continue
                        }
                        healthy = true
@@ -454,6 +449,14 @@ func (c *subscriptionPullConsumer) pollFlowPartition(ctx 
context.Context, topics
                                return result
                        }
                }
+               if attempted == 0 {
+                       c.requestRecovery()
+                       if lastFailure == nil {
+                               lastFailure = errors.New("subscription: no 
available provider for poll")
+                       }
+                       result.err = lastFailure
+                       return result
+               }
                if !healthy && lastFailure != nil {
                        result.err = lastFailure
                        return result
@@ -562,10 +565,16 @@ func (c *subscriptionPullConsumer) drainForClose() 
([]payload.SubscriptionMessag
 }
 
 func (c *subscriptionPullConsumer) refreshBufferedContexts() {
+       if len(c.drained) == 0 && len(c.processors) == 0 {
+               c.mu.Lock()
+               c.bufferedContexts = nil
+               c.mu.Unlock()
+               return
+       }
        var contexts []payload.CommitContext
        for _, message := range c.drained {
                if message != nil {
-                       commitContext := message.CommitContext()
+                       commitContext := record.ReadCommitContext(message)
                        if record.Committable(commitContext) {
                                contexts = append(contexts, 
record.CloneCommitContext(commitContext))
                        }
@@ -620,7 +629,7 @@ func (c *subscriptionPullConsumer) resetProcessors(topic 
string) {
        }
        retained := c.drained[:0]
        for _, message := range c.drained {
-               if message != nil && message.CommitContext().Topic == topic {
+               if message != nil && record.ReadCommitContext(message).Topic == 
topic {
                        message.Release()
                        continue
                }
@@ -634,9 +643,10 @@ func (c *subscriptionPullConsumer) resetProcessors(topic 
string) {
 // visibleFlowMessages also observes watermarks consumed by a processor. 
Callers
 // pass the original successful poll messages separately when a stage removes 
them.
 func (c *subscriptionPullConsumer) visibleFlowMessages(messages 
[]payload.SubscriptionMessage) []payload.SubscriptionMessage {
-       visible := make([]payload.SubscriptionMessage, 0, len(messages))
+       filter := false
        for _, message := range messages {
                if message == nil {
+                       filter = true
                        continue
                }
                if timestamp, ok := message.Watermark(); ok {
@@ -645,9 +655,19 @@ func (c *subscriptionPullConsumer) 
visibleFlowMessages(messages []payload.Subscr
                                c.watermark, c.hasWatermark = timestamp, true
                        }
                        c.mu.Unlock()
+                       filter = true
                        continue
                }
-               if message.Kind() == payload.RecordMessage {
+               if message.Kind() != payload.RecordMessage {
+                       filter = true
+               }
+       }
+       if !filter {
+               return messages
+       }
+       visible := make([]payload.SubscriptionMessage, 0, len(messages))
+       for _, message := range messages {
+               if message != nil && message.Kind() == payload.RecordMessage {
                        visible = append(visible, message)
                }
        }
diff --git a/subscription/consumer/pull_progress.go 
b/subscription/consumer/pull_progress.go
index 1628c52..cd54a51 100644
--- a/subscription/consumer/pull_progress.go
+++ b/subscription/consumer/pull_progress.go
@@ -26,6 +26,7 @@ import (
        "github.com/apache/iotdb-client-go/subscription/internal/record"
        "github.com/apache/iotdb-client-go/subscription/internal/identifier"
 
+       "container/heap"
        "context"
        "errors"
        "fmt"
@@ -50,7 +51,34 @@ type subscriptionCommitKey struct {
 
 type subscriptionPendingCommit struct {
        Context payload.CommitContext
-       DeliveredAt time.Time
+       ExpiresAt time.Time
+}
+
+type subscriptionCommitDeadline struct {
+       key subscriptionCommitKey
+       ExpiresAt time.Time
+}
+
+type subscriptionCommitDeadlineHeap []subscriptionCommitDeadline
+
+func (h subscriptionCommitDeadlineHeap) Len() int { return len(h) }
+func (h subscriptionCommitDeadlineHeap) Less(i, j int) bool {
+       if h[i].ExpiresAt.Equal(h[j].ExpiresAt) {
+               return h[i].key.CommitID < h[j].key.CommitID
+       }
+       return h[i].ExpiresAt.Before(h[j].ExpiresAt)
+}
+func (h subscriptionCommitDeadlineHeap) Swap(i, j int) { h[i], h[j] = h[j], 
h[i] }
+func (h *subscriptionCommitDeadlineHeap) Push(value any) {
+       *h = append(*h, value.(subscriptionCommitDeadline))
+}
+func (h *subscriptionCommitDeadlineHeap) Pop() any {
+       old := *h
+       last := len(old) - 1
+       value := old[last]
+       old[last] = subscriptionCommitDeadline{}
+       *h = old[:last]
+       return value
 }
 
 func subscriptionContextKey(value payload.CommitContext) subscriptionCommitKey 
{
@@ -71,14 +99,6 @@ func subscriptionContextKey(value payload.CommitContext) 
subscriptionCommitKey {
        return key
 }
 
-func cloneCommitContexts(contexts []payload.CommitContext) 
[]payload.CommitContext {
-       result := make([]payload.CommitContext, len(contexts))
-       for index, commitContext := range contexts {
-               result[index] = record.CloneCommitContext(commitContext)
-       }
-       return result
-}
-
 func cloneTopicProgress(progress payload.TopicProgress) payload.TopicProgress {
        result := make(payload.TopicProgress, len(progress))
        for region, writers := range progress {
@@ -101,7 +121,7 @@ func (c *subscriptionPullConsumer) Commit(ctx 
context.Context, messages ...paylo
        if err != nil {
                return err
        }
-       return c.commitContexts(operationCtx, contexts, false)
+       return c.commitUniqueContexts(operationCtx, contexts, false)
 }
 
 func (c *subscriptionPullConsumer) CommitAsync(ctx context.Context, messages 
...payload.SubscriptionMessage) <-chan error {
@@ -122,7 +142,7 @@ func (c *subscriptionPullConsumer) CommitAsync(ctx 
context.Context, messages ...
        go func() {
                defer finish()
                defer close(result)
-               result <- c.commitContexts(operationCtx, contexts, false)
+               result <- c.commitUniqueContexts(operationCtx, contexts, false)
        }()
        return result
 }
@@ -131,8 +151,11 @@ func (c *subscriptionPullConsumer) 
messageCommitContexts(messages []payload.Subs
        c.mu.RLock()
        groupID := c.identity.GroupID
        c.mu.RUnlock()
+       var seen map[subscriptionCommitKey]struct{}
+       if len(messages) > 1 {
+               seen = make(map[subscriptionCommitKey]struct{}, len(messages))
+       }
        contexts := make([]payload.CommitContext, 0, len(messages))
-       seen := make(map[subscriptionCommitKey]struct{}, len(messages))
        for _, message := range messages {
                if record.IsNilMessage(message) {
                        return nil, errors.New("subscription: cannot commit a 
nil message")
@@ -140,17 +163,21 @@ func (c *subscriptionPullConsumer) 
messageCommitContexts(messages []payload.Subs
                if message.Kind() != payload.RecordMessage {
                        return nil, subscription.ErrIncompatibleMessage
                }
-               commitContext := message.CommitContext()
+               commitContext := record.ReadCommitContext(message)
                if commitContext.Topic == "" || commitContext.GroupID != 
groupID {
                        return nil, errors.New("subscription: message does not 
belong to this consumer group")
                }
                if !record.Committable(commitContext) {
                        return nil, errors.New("subscription: message has no 
committable context")
                }
+               if seen == nil {
+                       contexts = append(contexts, commitContext)
+                       continue
+               }
                key := subscriptionContextKey(commitContext)
                if _, exists := seen[key]; !exists {
                        seen[key] = struct{}{}
-                       contexts = append(contexts, 
record.CloneCommitContext(commitContext))
+                       contexts = append(contexts, commitContext)
                }
        }
        return contexts, nil
@@ -175,11 +202,28 @@ func (c *subscriptionPullConsumer) commitContexts(ctx 
context.Context, contexts
                return err
        }
        defer func() { <-c.progressOperations }()
-       return c.commitContextsLocked(ctx, contexts, nack)
+       return c.commitUniqueContextsLocked(ctx, 
normalizeCommitContexts(contexts), nack)
 }
 
-func (c *subscriptionPullConsumer) commitContextsLocked(ctx context.Context, 
contexts []payload.CommitContext, nack bool) error {
-       groups := make(map[int32][]payload.CommitContext)
+func (c *subscriptionPullConsumer) commitUniqueContexts(ctx context.Context, 
contexts []payload.CommitContext, nack bool) error {
+       if err := c.acquireProgress(ctx); err != nil {
+               return err
+       }
+       defer func() { <-c.progressOperations }()
+       return c.commitUniqueContextsLocked(ctx, contexts, nack)
+}
+
+func normalizeCommitContexts(contexts []payload.CommitContext) 
[]payload.CommitContext {
+       if len(contexts) == 0 {
+               return nil
+       }
+       if len(contexts) == 1 {
+               if !record.Committable(contexts[0]) {
+                       return nil
+               }
+               return 
[]payload.CommitContext{record.CloneCommitContext(contexts[0])}
+       }
+       unique := make([]payload.CommitContext, 0, len(contexts))
        seen := make(map[subscriptionCommitKey]struct{}, len(contexts))
        for _, commitContext := range contexts {
                if !record.Committable(commitContext) {
@@ -190,13 +234,28 @@ func (c *subscriptionPullConsumer) 
commitContextsLocked(ctx context.Context, con
                        continue
                }
                seen[key] = struct{}{}
-               groups[commitContext.DataNodeID] = 
append(groups[commitContext.DataNodeID], 
record.CloneCommitContext(commitContext))
+               unique = append(unique, 
record.CloneCommitContext(commitContext))
        }
-       nodes := make([]int, 0, len(groups))
-       for node := range groups {
-               nodes = append(nodes, int(node))
+       return unique
+}
+
+func (c *subscriptionPullConsumer) commitUniqueContextsLocked(ctx 
context.Context, contexts []payload.CommitContext, nack bool) error {
+       groups := make(map[int32][]payload.CommitContext)
+       for _, commitContext := range contexts {
+               groups[commitContext.DataNodeID] = 
append(groups[commitContext.DataNodeID], commitContext)
+       }
+       var nodes []int
+       if len(groups) > 1 {
+               nodes = make([]int, 0, len(groups))
+               for node := range groups {
+                       nodes = append(nodes, int(node))
+               }
+               sort.Ints(nodes)
+       } else {
+               for node := range groups {
+                       nodes = []int{int(node)}
+               }
        }
-       sort.Ints(nodes)
        var failures []error
        for _, node := range nodes {
                if err := c.commitNode(ctx, int32(node), groups[int32(node)], 
nack); err != nil {
@@ -211,7 +270,7 @@ func (c *subscriptionPullConsumer) commitContextsLocked(ctx 
context.Context, con
 }
 
 func (c *subscriptionPullConsumer) commitNode(ctx context.Context, node int32, 
contexts []payload.CommitContext, nack bool) error {
-       remaining := cloneCommitContexts(contexts)
+       remaining := contexts
        var lastFailure error
        for attempt := 0; len(remaining) != 0; attempt++ {
                if err := ctx.Err(); err != nil {
@@ -341,7 +400,7 @@ func (c *subscriptionPullConsumer) flushRedirectAcks(ctx 
context.Context, messag
        type topicRegion struct { topic, region string }
        targets := make(map[topicRegion]int32)
        for _, message := range messages {
-               commitContext := message.CommitContext()
+               commitContext := record.ReadCommitContext(message)
                if commitContext.Writer != nil && commitContext.RegionID != "" {
                        targets[topicRegion{commitContext.Topic, 
commitContext.RegionID}] = commitContext.DataNodeID
                }
@@ -384,29 +443,58 @@ func (c *subscriptionPullConsumer) 
trackDelivered(messages []payload.Subscriptio
                if message.Kind() != payload.RecordMessage {
                        continue
                }
-               commitContext := message.CommitContext()
+               commitContext := record.ReadCommitContext(message)
                if !record.Committable(commitContext) {
                        continue
                }
                key := subscriptionContextKey(commitContext)
                if _, exists := c.uncommitted[key]; !exists {
-                       c.uncommitted[key] = subscriptionPendingCommit{Context: 
record.CloneCommitContext(commitContext), DeliveredAt: now}
+                       expiresAt := now.Add(c.config.AutoCommitInterval)
+                       c.uncommitted[key] = subscriptionPendingCommit{Context: 
commitContext, ExpiresAt: expiresAt}
+                       heap.Push(&c.uncommittedDeadlines, 
subscriptionCommitDeadline{key: key, ExpiresAt: expiresAt})
                }
        }
 }
 
 func (c *subscriptionPullConsumer) pendingCommitContexts(all bool, now 
time.Time) []payload.CommitContext {
-       c.mu.RLock()
-       defer c.mu.RUnlock()
+       c.mu.Lock()
+       defer c.mu.Unlock()
+       if all {
+               contexts := make([]payload.CommitContext, 0, len(c.uncommitted))
+               for _, pending := range c.uncommitted {
+                       contexts = append(contexts, pending.Context)
+               }
+               return contexts
+       }
        var contexts []payload.CommitContext
-       for _, pending := range c.uncommitted {
-               if all || 
!now.Before(pending.DeliveredAt.Add(c.config.AutoCommitInterval)) {
-                       contexts = append(contexts, 
record.CloneCommitContext(pending.Context))
+       for c.uncommittedDeadlines.Len() != 0 {
+               deadline := c.uncommittedDeadlines[0]
+               if now.Before(deadline.ExpiresAt) {
+                       break
+               }
+               heap.Pop(&c.uncommittedDeadlines)
+               pending, exists := c.uncommitted[deadline.key]
+               if !exists || !pending.ExpiresAt.Equal(deadline.ExpiresAt) {
+                       continue
                }
+               contexts = append(contexts, pending.Context)
        }
        return contexts
 }
 
+func (c *subscriptionPullConsumer) requeuePendingCommitContexts(contexts 
[]payload.CommitContext, now time.Time) {
+       c.mu.Lock()
+       defer c.mu.Unlock()
+       for _, commitContext := range contexts {
+               key := subscriptionContextKey(commitContext)
+               pending, exists := c.uncommitted[key]
+               if !exists || pending.ExpiresAt.After(now) {
+                       continue
+               }
+               heap.Push(&c.uncommittedDeadlines, 
subscriptionCommitDeadline{key: key, ExpiresAt: pending.ExpiresAt})
+       }
+}
+
 func (c *subscriptionPullConsumer) autoCommit(ctx context.Context, now 
time.Time) {
        if c.config.AutoCommit == nil || !*c.config.AutoCommit {
                return
@@ -417,7 +505,8 @@ func (c *subscriptionPullConsumer) autoCommit(ctx 
context.Context, now time.Time
        defer func() { <-c.progressOperations }()
        contexts := c.pendingCommitContexts(false, now)
        if len(contexts) != 0 {
-               _ = c.commitContextsLocked(ctx, contexts, false)
+               _ = c.commitUniqueContextsLocked(ctx, contexts, false)
+               c.requeuePendingCommitContexts(contexts, now)
        }
 }
 
@@ -425,7 +514,7 @@ func (c *subscriptionPullConsumer) 
advancePositions(messages []payload.Subscript
        c.mu.Lock()
        defer c.mu.Unlock()
        for _, message := range messages {
-               commitContext := message.CommitContext()
+               commitContext := record.ReadCommitContext(message)
                writer, progress, ok := 
subscriptionContextProgress(commitContext)
                if ok {
                        mergeSubscriptionProgress(c.positions, 
commitContext.Topic, payload.TopicProgress{writer.RegionID: 
payload.RegionProgress{writer: progress}})
diff --git a/subscription/consumer/push_consumer.go 
b/subscription/consumer/push_consumer.go
index 83f5207..3725dab 100644
--- a/subscription/consumer/push_consumer.go
+++ b/subscription/consumer/push_consumer.go
@@ -23,6 +23,7 @@ import (
        "github.com/apache/iotdb-client-go/subscription"
        "github.com/apache/iotdb-client-go/subscription/payload"
        "github.com/apache/iotdb-client-go/subscription/internal/protocol"
+       "github.com/apache/iotdb-client-go/subscription/internal/record"
        "github.com/apache/iotdb-client-go/subscription/internal/identifier"
 
        "context"
@@ -275,7 +276,7 @@ func (c *subscriptionPushConsumer) pollDeliveries(ctx 
context.Context) ([]subscr
        for _, message := range messages {
                deliveries = append(deliveries, subscriptionPushDelivery{
                        message: message,
-                       generation: 
c.generations[message.CommitContext().Topic],
+                       generation: 
c.generations[record.ReadCommitContext(message).Topic],
                })
        }
        return deliveries, nil
@@ -353,7 +354,7 @@ func (c *subscriptionPushConsumer) deliver(ctx 
context.Context, deliveries []sub
 }
 
 func (c *subscriptionPushConsumer) beginCallback(ctx context.Context, delivery 
subscriptionPushDelivery) (context.Context, func(), bool) {
-       topic := delivery.message.CommitContext().Topic
+       topic := record.ReadCommitContext(delivery.message).Topic
        c.mu.Lock()
        defer c.mu.Unlock()
        if c.deliveryStateLocked() != nil || ctx.Err() != nil || 
c.generations[topic] != delivery.generation {
@@ -395,7 +396,7 @@ func (c *subscriptionPushConsumer) commitDeliveries(ctx 
context.Context, deliver
        }
        messages := make([]payload.SubscriptionMessage, 0, len(deliveries))
        for _, delivery := range deliveries {
-               if c.generations[delivery.message.CommitContext().Topic] == 
delivery.generation {
+               if 
c.generations[record.ReadCommitContext(delivery.message).Topic] == 
delivery.generation {
                        messages = append(messages, delivery.message)
                }
        }
diff --git a/subscription/internal/protocol/tablet.go 
b/subscription/internal/protocol/tablet.go
index a2a818a..2033032 100644
--- a/subscription/internal/protocol/tablet.go
+++ b/subscription/internal/protocol/tablet.go
@@ -112,7 +112,7 @@ func decodeSubscriptionTablet(reader 
*subscriptionWireReader) (record.Tablet, er
        if reader.Err() != nil {
                return record.Tablet{}, reader.Err()
        }
-       tablet.Values = make([][]any, len(tablet.Columns))
+       tablet.Values = make([]record.TabletColumn, len(tablet.Columns))
        if reader.ReadBool() {
                for column, schema := range tablet.Columns {
                        if !reader.ReadBool() {
@@ -122,9 +122,9 @@ func decodeSubscriptionTablet(reader 
*subscriptionWireReader) (record.Tablet, er
                        if err != nil {
                                return record.Tablet{}, err
                        }
-                       for row := range values {
+                       for row := 0; row < tablet.RowCount; row++ {
                                if bitmap := bitmaps[column]; bitmap != nil && 
bitmap[row/8]&(1<<uint(row%8)) != 0 {
-                                       values[row] = nil
+                                       values.SetNull(row)
                                }
                        }
                        tablet.Values[column] = values
@@ -162,54 +162,59 @@ func requireSubscriptionTabletValues(reader 
*subscriptionWireReader, count, size
        return nil
 }
 
-func decodeSubscriptionTabletColumn(reader *subscriptionWireReader, dataType 
client.TSDataType, rows int) ([]any, error) {
+func decodeSubscriptionTabletColumn(reader *subscriptionWireReader, dataType 
client.TSDataType, rows int) (record.TabletColumn, error) {
        size, err := subscriptionTabletValueSize(dataType)
        if err != nil {
-               return nil, err
+               return record.TabletColumn{}, err
        }
        if err := requireSubscriptionTabletValues(reader, rows, size); err != 
nil {
-               return nil, err
+               return record.TabletColumn{}, err
        }
-       values := make([]any, rows)
-       for row := range values {
+       values := record.NewTabletColumn(dataType, rows)
+       for row := 0; row < rows; row++ {
                switch dataType {
                case client.BOOLEAN:
-                       values[row] = reader.ReadBool()
+                       values.Booleans[row] = reader.ReadBool()
                case client.INT32:
-                       values[row] = reader.ReadInt32()
+                       values.Int32s[row] = reader.ReadInt32()
                case client.INT64, client.TIMESTAMP:
-                       values[row] = reader.ReadInt64()
+                       values.Int64s[row] = reader.ReadInt64()
                case client.FLOAT:
-                       values[row] = 
math.Float32frombits(uint32(reader.ReadInt32()))
+                       values.Float32s[row] = 
math.Float32frombits(uint32(reader.ReadInt32()))
                case client.DOUBLE:
-                       values[row] = 
math.Float64frombits(uint64(reader.ReadInt64()))
+                       values.Float64s[row] = 
math.Float64frombits(uint64(reader.ReadInt64()))
                case client.DATE:
                        date := reader.ReadInt32()
                        if reader.Err() != nil {
-                               return nil, reader.Err()
+                               return record.TabletColumn{}, reader.Err()
                        }
                        year, month, day := int(date/10000), 
time.Month((date/100)%100), int(date%100)
                        value := time.Date(year, month, day, 0, 0, 0, 0, 
time.UTC)
                        if value.Year() != year || value.Month() != month || 
value.Day() != day {
-                               return nil, fmt.Errorf("subscription: invalid 
Tablet DATE value %d", date)
+                               return record.TabletColumn{}, 
fmt.Errorf("subscription: invalid Tablet DATE value %d", date)
                        }
-                       values[row] = value
-               case client.TEXT, client.STRING, client.BLOB, client.OBJECT:
+                       values.Dates[row] = date
+               case client.TEXT, client.STRING:
                        var binary []byte
                        if reader.ReadBool() {
                                length := reader.ReadCount(1, false)
                                binary = reader.Take(int(length))
                        }
-                       if dataType == client.TEXT || dataType == client.STRING 
{
-                               values[row] = string(binary)
-                       } else {
-                               // Type.deserializeArray uses 
Binary.EMPTY_VALUE for a missing
-                               // Binary. SQL NULL is supplied separately by 
the Tablet bitmap.
-                               values[row] = append([]byte{}, binary...)
+                       values.Strings[row] = string(binary)
+                       values.VariableBytes += int64(len(binary))
+               case client.BLOB, client.OBJECT:
+                       var binary []byte
+                       if reader.ReadBool() {
+                               length := reader.ReadCount(1, false)
+                               binary = reader.Take(int(length))
                        }
+                       // Type.deserializeArray uses Binary.EMPTY_VALUE for a 
missing
+                       // Binary. SQL NULL is supplied separately by the 
Tablet bitmap.
+                       values.Binaries[row] = append([]byte{}, binary...)
+                       values.VariableBytes += int64(len(binary))
                }
                if reader.Err() != nil {
-                       return nil, reader.Err()
+                       return record.TabletColumn{}, reader.Err()
                }
        }
        return values, nil
diff --git a/subscription/internal/record/message.go 
b/subscription/internal/record/message.go
index 42ff71c..5a5fccc 100644
--- a/subscription/internal/record/message.go
+++ b/subscription/internal/record/message.go
@@ -24,12 +24,13 @@ import (
        "iter"
        "sort"
        "sync"
-       "time"
 
        "github.com/apache/iotdb-client-go/subscription"
        "github.com/apache/iotdb-client-go/subscription/payload"
 )
 
+const tabletColumnStorageEstimate = int64(240)
+
 // subscriptionMessage keeps immutable delivery metadata separate from 
releasable
 // record buffers. Its factories take ownership of decoded batches.
 type subscriptionMessage struct {
@@ -76,6 +77,15 @@ func (m *subscriptionMessage) CommitContext() 
payload.CommitContext {
        return CloneCommitContext(m.context)
 }
 
+// ReadCommitContext avoids copying immutable metadata of an internally owned
+// message. External message implementations keep their public snapshot 
contract.
+func ReadCommitContext(message payload.SubscriptionMessage) 
payload.CommitContext {
+       if internal, ok := message.(*subscriptionMessage); ok {
+               return internal.context
+       }
+       return message.CommitContext()
+}
+
 func (m *subscriptionMessage) Records() ([]payload.RecordBatch, error) {
        if m.kind != payload.RecordMessage {
                return nil, subscription.ErrIncompatibleMessage
@@ -138,7 +148,7 @@ type subscriptionRecordBatch struct {
        rowCount int
        columns []payload.Column
        timestamps []int64
-       values [][]any
+       values []TabletColumn
        rowOrder []int
        released bool
 }
@@ -238,13 +248,11 @@ func (b *subscriptionRecordBatch) valueLocked(row, column 
int) (any, error) {
        if column >= len(b.values) {
                return nil, fmt.Errorf("subscription: record column %d has no 
value storage", column)
        }
-       if b.values[column] == nil {
-               return nil, nil
-       }
-       if row >= len(b.values[column]) {
+       value, ok := b.values[column].Value(row)
+       if !ok {
                return nil, fmt.Errorf("subscription: record column %d has no 
value for row %d", column, row)
        }
-       return b.values[column][row], nil
+       return value, nil
 }
 
 func (b *subscriptionRecordBatch) Rows() iter.Seq2[payload.Row, error] {
@@ -310,46 +318,22 @@ func (b *subscriptionRecordBatch) release() {
 func (b *subscriptionRecordBatch) estimatedSize() int64 {
        b.mu.RLock()
        defer b.mu.RUnlock()
+       return b.estimatedSizeLocked()
+}
+
+func (b *subscriptionRecordBatch) estimatedSizeLocked() int64 {
        size := int64(128) + int64(len(b.dialect)) + int64(len(b.database)) +
                int64(len(b.target)) + int64(len(b.timestamps))*8 + 
int64(len(b.rowOrder))*8
-       size += int64(len(b.columns))*32 + int64(len(b.values))*24
+       size += int64(len(b.columns))*32 + 
int64(len(b.values))*tabletColumnStorageEstimate
        for _, column := range b.columns {
                size += int64(len(column.Name))
        }
-       for _, values := range b.values {
-               size += int64(len(values))*16
-               for _, value := range values {
-                       switch value := value.(type) {
-                       case bool:
-                               size++
-                       case int32, float32:
-                               size += 4
-                       case int64, float64:
-                               size += 8
-                       case string:
-                               size += 16 + int64(len(value))
-                       case []byte:
-                               size += 24 + int64(len(value))
-                       case time.Time:
-                               size += 24
-                       }
-               }
+       for column := range b.values {
+               size += b.values[column].estimatedSize()
        }
        return size
 }
 
-// Tablet follows TsFile 2.5.1-261023-SNAPSHOT Tablet's stream
-// format, independently of the Session insert protocol and TsBlock query 
format.
-// Nil Timestamps represents RowCount zero timestamps. Each nil Values column
-// represents RowCount null values without allocating storage for absent data.
-type Tablet struct {
-       Target string
-       Columns []payload.Column
-       RowCount int
-       Timestamps []int64
-       Values [][]any
-}
-
 // Committable reports whether a delivery has writer progress or a legacy ID.
 func Committable(context payload.CommitContext) bool {
        return context.Writer != nil || context.CommitID != 
payload.InvalidCommitID
diff --git a/subscription/internal/record/mutate.go 
b/subscription/internal/record/mutate.go
new file mode 100644
index 0000000..5707c52
--- /dev/null
+++ b/subscription/internal/record/mutate.go
@@ -0,0 +1,245 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package record
+
+import (
+       "errors"
+
+       "github.com/apache/iotdb-client-go/subscription"
+       "github.com/apache/iotdb-client-go/subscription/payload"
+)
+
+// TabletMutator provides one exclusive lock across an in-place Tablet 
mutation.
+// It is intentionally internal to the subscription module and never changes 
the
+// public RecordBatch contract.
+type TabletMutator struct {
+       batch *subscriptionRecordBatch
+}
+
+func (m TabletMutator) Dialect() subscription.Dialect { return m.batch.dialect 
}
+func (m TabletMutator) Database() string { return m.batch.database }
+func (m TabletMutator) Target() string { return m.batch.target }
+func (m TabletMutator) TimeSelected() bool { return m.batch.timeSelected }
+func (m TabletMutator) RowCount() int { return m.batch.rowCount }
+func (m TabletMutator) Columns() []payload.Column { return m.batch.columns }
+
+func (m TabletMutator) Timestamp(row int) int64 {
+       if m.batch.timestamps == nil {
+               return 0
+       }
+       return m.batch.timestamps[row]
+}
+
+func (m TabletMutator) Value(row, column int) (any, bool) {
+       if column < 0 || column >= len(m.batch.columns) {
+               return nil, false
+       }
+       if column >= len(m.batch.values) {
+               return nil, true
+       }
+       return m.batch.values[column].Value(row)
+}
+
+func (m TabletMutator) SetValue(row, column int, value any) bool {
+       if column < 0 || column >= len(m.batch.columns) {
+               return false
+       }
+       if column >= len(m.batch.values) {
+               values := make([]TabletColumn, len(m.batch.columns))
+               copy(values, m.batch.values)
+               m.batch.values = values
+       }
+       if !m.batch.values[column].present() {
+               m.batch.values[column] = 
NewTabletColumn(m.batch.columns[column].DataType, m.batch.rowCount)
+       }
+       return m.batch.values[column].Set(row, value)
+}
+
+// TryMutateRecordMessages mutates decoded record buffers in place and 
transfers
+// their ownership to replacement messages. The bool result is false without
+// mutation when an input is not an internally owned record message; callers 
can
+// then use a copying fallback for user-provided implementations.
+func TryMutateRecordMessages(messages []payload.SubscriptionMessage, mutate 
func(payload.CommitContext, TabletMutator) error) 
([]payload.SubscriptionMessage, bool, error) {
+       seen := make(map[*subscriptionMessage]struct{}, len(messages))
+       seenBatches := make(map[*subscriptionRecordBatch]struct{})
+       owned := make([]*subscriptionMessage, len(messages))
+       type ownedBatch struct {
+               context payload.CommitContext
+               batch *subscriptionRecordBatch
+       }
+       var batches []ownedBatch
+       for index, message := range messages {
+               if message == nil {
+                       return nil, true, errors.New("subscription: mutation 
received a nil message")
+               }
+               if message.Kind() == payload.WatermarkMessage {
+                       continue
+               }
+               internal, ok := message.(*subscriptionMessage)
+               if !ok {
+                       return nil, false, nil
+               }
+               if _, duplicate := seen[internal]; duplicate {
+                       return nil, false, nil
+               }
+               seen[internal] = struct{}{}
+               internal.mu.RLock()
+               if internal.released {
+                       internal.mu.RUnlock()
+                       return nil, true, subscription.ErrDataReleased
+               }
+               internalBatches := append([]payload.RecordBatch(nil), 
internal.batches...)
+               internal.mu.RUnlock()
+               owned[index] = internal
+               for _, batch := range internalBatches {
+                       internalBatch, ok := batch.(*subscriptionRecordBatch)
+                       if !ok {
+                               return nil, false, nil
+                       }
+                       if _, duplicate := seenBatches[internalBatch]; 
duplicate {
+                               return nil, false, nil
+                       }
+                       seenBatches[internalBatch] = struct{}{}
+                       batches = append(batches, ownedBatch{context: 
internal.context, batch: internalBatch})
+               }
+       }
+       for _, owned := range batches {
+               batch := owned.batch
+               batch.mu.RLock()
+               if batch.released {
+                       batch.mu.RUnlock()
+                       return nil, true, subscription.ErrDataReleased
+               }
+               if batch.timestamps != nil && len(batch.timestamps) != 
batch.rowCount {
+                       batch.mu.RUnlock()
+                       return nil, true, errors.New("subscription: Tablet 
timestamp count does not match row count")
+               }
+               for column := 0; column < len(batch.columns) && column < 
len(batch.values); column++ {
+                       values := &batch.values[column]
+                       if values.present() && values.DataType != 
batch.columns[column].DataType {
+                               batch.mu.RUnlock()
+                               return nil, true, errors.New("subscription: 
Tablet column type does not match schema")
+                       }
+                       if values.present() && values.Rows() != batch.rowCount {
+                               batch.mu.RUnlock()
+                               return nil, true, errors.New("subscription: 
Tablet column row count does not match Tablet")
+                       }
+               }
+               batch.mu.RUnlock()
+       }
+       for _, owned := range batches {
+               batch := owned.batch
+               batch.mu.Lock()
+               if batch.released {
+                       batch.mu.Unlock()
+                       return nil, true, subscription.ErrDataReleased
+               }
+               err := mutate(owned.context, TabletMutator{batch: batch})
+               batch.mu.Unlock()
+               if err != nil {
+                       return nil, true, err
+               }
+       }
+       result := make([]payload.SubscriptionMessage, len(messages))
+       for index, message := range messages {
+               if message.Kind() == payload.WatermarkMessage {
+                       result[index] = message
+                       continue
+               }
+               internal := owned[index]
+               result[index] = &subscriptionMessage{
+                       kind: payload.RecordMessage,
+                       context: CloneCommitContext(internal.context),
+                       batches: append([]payload.RecordBatch(nil), 
internal.batches...),
+               }
+               internal.mu.Lock()
+               internal.released = true
+               internal.batches = nil
+               internal.mu.Unlock()
+       }
+       return result, true, nil
+}
+
+// TabletReader exposes decoded timestamps and typed columns while the batch is
+// locked. Callers must not retain the reader after the callback returns.
+type TabletReader struct {
+       batch *subscriptionRecordBatch
+}
+
+func (r TabletReader) Database() string { return r.batch.database }
+func (r TabletReader) Target() string { return r.batch.target }
+func (r TabletReader) RowCount() int { return r.batch.rowCount }
+func (r TabletReader) Columns() []payload.Column { return r.batch.columns }
+func (r TabletReader) Timestamp(row int) int64 {
+       if r.batch.timestamps == nil {
+               return 0
+       }
+       return r.batch.timestamps[row]
+}
+func (r TabletReader) Value(row, column int) (any, bool) {
+       if column < 0 || column >= len(r.batch.columns) {
+               return nil, false
+       }
+       if column >= len(r.batch.values) {
+               return nil, true
+       }
+       return r.batch.values[column].Value(row)
+}
+func (r TabletReader) EstimatedSize() int64 { return 
r.batch.estimatedSizeLocked() }
+
+// TryReadRecordMessage visits decoded batches with one read lock per batch. It
+// returns false without invoking visit when message is not internally owned.
+func TryReadRecordMessage(message payload.SubscriptionMessage, visit 
func(payload.CommitContext, TabletReader) error) (bool, error) {
+       if message == nil {
+               return true, errors.New("subscription: reader received a nil 
message")
+       }
+       if message.Kind() != payload.RecordMessage {
+               return true, subscription.ErrIncompatibleMessage
+       }
+       internal, ok := message.(*subscriptionMessage)
+       if !ok {
+               return false, nil
+       }
+       internal.mu.RLock()
+       released := internal.released
+       context := internal.context
+       batches := append([]payload.RecordBatch(nil), internal.batches...)
+       internal.mu.RUnlock()
+       if released {
+               return true, subscription.ErrDataReleased
+       }
+       for _, batch := range batches {
+               internalBatch, ok := batch.(*subscriptionRecordBatch)
+               if !ok {
+                       return false, nil
+               }
+               internalBatch.mu.RLock()
+               if internalBatch.released {
+                       internalBatch.mu.RUnlock()
+                       return true, subscription.ErrDataReleased
+               }
+               err := visit(context, TabletReader{batch: internalBatch})
+               internalBatch.mu.RUnlock()
+               if err != nil {
+                       return true, err
+               }
+       }
+       return true, nil
+}
diff --git a/subscription/internal/record/tablet.go 
b/subscription/internal/record/tablet.go
new file mode 100644
index 0000000..2421ce6
--- /dev/null
+++ b/subscription/internal/record/tablet.go
@@ -0,0 +1,348 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package record
+
+import (
+       "time"
+
+       "github.com/apache/iotdb-client-go/subscription/payload"
+       "github.com/apache/iotdb-client-go/v2/client"
+)
+
+// Tablet follows TsFile's columnar stream representation. Values are stored in
+// typed backing arrays and materialized as any only at the public accessor.
+type Tablet struct {
+       Target     string
+       Columns    []payload.Column
+       RowCount   int
+       Timestamps []int64
+       Values     []TabletColumn
+}
+
+// TabletColumn is one typed Tablet column. Exactly one value slice is 
populated
+// according to DataType. A nil zero-value column represents an absent column.
+type TabletColumn struct {
+       DataType     client.TSDataType
+       Booleans     []bool
+       Int32s       []int32
+       Int64s       []int64
+       Float32s     []float32
+       Float64s     []float64
+       Dates        []int32
+       Strings      []string
+       Binaries     [][]byte
+       Nulls        []uint64
+       VariableBytes int64
+}
+
+// NewTabletColumn allocates the backing array required by dataType.
+func NewTabletColumn(dataType client.TSDataType, rows int) TabletColumn {
+       column := TabletColumn{DataType: dataType}
+       switch dataType {
+       case client.BOOLEAN:
+               column.Booleans = make([]bool, rows)
+       case client.INT32:
+               column.Int32s = make([]int32, rows)
+       case client.INT64, client.TIMESTAMP:
+               column.Int64s = make([]int64, rows)
+       case client.FLOAT:
+               column.Float32s = make([]float32, rows)
+       case client.DOUBLE:
+               column.Float64s = make([]float64, rows)
+       case client.DATE:
+               column.Dates = make([]int32, rows)
+       case client.TEXT, client.STRING:
+               column.Strings = make([]string, rows)
+       case client.BLOB, client.OBJECT:
+               column.Binaries = make([][]byte, rows)
+       }
+       return column
+}
+
+// Set stores a non-null value without introducing a per-cell interface slot.
+// It returns false when the value or storage type does not match the column.
+func (c *TabletColumn) Set(row int, value any) bool {
+       if !c.hasRow(row) {
+               return false
+       }
+       if value == nil {
+               c.SetNull(row)
+               return true
+       }
+       switch c.DataType {
+       case client.BOOLEAN:
+               typed, ok := value.(bool)
+               if !ok || row >= len(c.Booleans) {
+                       return false
+               }
+               c.Booleans[row] = typed
+       case client.INT32:
+               typed, ok := value.(int32)
+               if !ok || row >= len(c.Int32s) {
+                       return false
+               }
+               c.Int32s[row] = typed
+       case client.INT64, client.TIMESTAMP:
+               typed, ok := value.(int64)
+               if !ok || row >= len(c.Int64s) {
+                       return false
+               }
+               c.Int64s[row] = typed
+       case client.FLOAT:
+               typed, ok := value.(float32)
+               if !ok || row >= len(c.Float32s) {
+                       return false
+               }
+               c.Float32s[row] = typed
+       case client.DOUBLE:
+               typed, ok := value.(float64)
+               if !ok || row >= len(c.Float64s) {
+                       return false
+               }
+               c.Float64s[row] = typed
+       case client.DATE:
+               typed, ok := value.(time.Time)
+               if !ok || row >= len(c.Dates) {
+                       return false
+               }
+               c.Dates[row] = int32(typed.Year()*10000 + 
int(typed.Month())*100 + typed.Day())
+       case client.TEXT, client.STRING:
+               typed, ok := value.(string)
+               if !ok || row >= len(c.Strings) {
+                       return false
+               }
+               c.VariableBytes += int64(len(typed) - len(c.Strings[row]))
+               c.Strings[row] = typed
+       case client.BLOB, client.OBJECT:
+               typed, ok := value.([]byte)
+               if !ok || row >= len(c.Binaries) {
+                       return false
+               }
+               c.VariableBytes += int64(len(typed) - len(c.Binaries[row]))
+               c.Binaries[row] = append([]byte{}, typed...)
+       default:
+               return false
+       }
+       c.setNull(row, false)
+       return true
+}
+
+// Value materializes one typed value for the public RecordBatch accessor. The
+// second result is false when the column has no storage for row.
+func (c *TabletColumn) Value(row int) (any, bool) {
+       if row < 0 {
+               return nil, false
+       }
+       if !c.present() {
+               return nil, true
+       }
+       switch c.DataType {
+       case client.BOOLEAN:
+               if row >= len(c.Booleans) {
+                       return nil, false
+               }
+               if c.IsNull(row) {
+                       return nil, true
+               }
+               return c.Booleans[row], true
+       case client.INT32:
+               if row >= len(c.Int32s) {
+                       return nil, false
+               }
+               if c.IsNull(row) {
+                       return nil, true
+               }
+               return c.Int32s[row], true
+       case client.INT64, client.TIMESTAMP:
+               if row >= len(c.Int64s) {
+                       return nil, false
+               }
+               if c.IsNull(row) {
+                       return nil, true
+               }
+               return c.Int64s[row], true
+       case client.FLOAT:
+               if row >= len(c.Float32s) {
+                       return nil, false
+               }
+               if c.IsNull(row) {
+                       return nil, true
+               }
+               return c.Float32s[row], true
+       case client.DOUBLE:
+               if row >= len(c.Float64s) {
+                       return nil, false
+               }
+               if c.IsNull(row) {
+                       return nil, true
+               }
+               return c.Float64s[row], true
+       case client.DATE:
+               if row >= len(c.Dates) {
+                       return nil, false
+               }
+               if c.IsNull(row) {
+                       return nil, true
+               }
+               date := c.Dates[row]
+               year, month, day := int(date/10000), 
time.Month((date/100)%100), int(date%100)
+               value := time.Date(year, month, day, 0, 0, 0, 0, time.UTC)
+               if value.Year() != year || value.Month() != month || 
value.Day() != day {
+                       return nil, false
+               }
+               return value, true
+       case client.TEXT, client.STRING:
+               if row >= len(c.Strings) {
+                       return nil, false
+               }
+               if c.IsNull(row) {
+                       return nil, true
+               }
+               return c.Strings[row], true
+       case client.BLOB, client.OBJECT:
+               if row >= len(c.Binaries) {
+                       return nil, false
+               }
+               if c.IsNull(row) {
+                       return nil, true
+               }
+               return c.Binaries[row], true
+       case client.UNKNOWN:
+               return nil, true
+       default:
+               return nil, false
+       }
+}
+
+// SetNull marks an SQL NULL cell without changing the backing array length.
+func (c *TabletColumn) SetNull(row int) {
+       if !c.hasRow(row) {
+               return
+       }
+       c.setNull(row, true)
+}
+
+// IsNull reports whether row is SQL NULL. An absent column is all NULL.
+func (c *TabletColumn) IsNull(row int) bool {
+       if !c.hasRow(row) {
+               return true
+       }
+       if len(c.Nulls) == 0 {
+               return false
+       }
+       word := row / 64
+       return word < len(c.Nulls) && c.Nulls[word]&(uint64(1)<<uint(row%64)) 
!= 0
+}
+
+func (c *TabletColumn) hasRow(row int) bool {
+       return row >= 0 && ((row < len(c.Booleans)) || (row < len(c.Int32s)) ||
+               (row < len(c.Int64s)) || (row < len(c.Float32s)) ||
+               (row < len(c.Float64s)) || (row < len(c.Dates)) ||
+               (row < len(c.Strings)) || (row < len(c.Binaries)))
+}
+
+func (c *TabletColumn) setNull(row int, null bool) {
+       if !c.hasRow(row) {
+               return
+       }
+       rows := c.Rows()
+       words := (rows + 63) / 64
+       if len(c.Nulls) < words {
+               nulls := make([]uint64, words)
+               copy(nulls, c.Nulls)
+               c.Nulls = nulls
+       }
+       word, mask := row/64, uint64(1)<<uint(row%64)
+       if null {
+               c.Nulls[word] |= mask
+       } else {
+               c.Nulls[word] &^= mask
+       }
+}
+
+// Rows returns the backing row count for a populated column.
+func (c *TabletColumn) Rows() int {
+       switch c.DataType {
+       case client.BOOLEAN:
+               return len(c.Booleans)
+       case client.INT32:
+               return len(c.Int32s)
+       case client.INT64, client.TIMESTAMP:
+               return len(c.Int64s)
+       case client.FLOAT:
+               return len(c.Float32s)
+       case client.DOUBLE:
+               return len(c.Float64s)
+       case client.DATE:
+               return len(c.Dates)
+       case client.TEXT, client.STRING:
+               return len(c.Strings)
+       case client.BLOB, client.OBJECT:
+               return len(c.Binaries)
+       default:
+               return 0
+       }
+}
+
+
+func (c *TabletColumn) present() bool {
+       switch c.DataType {
+       case client.BOOLEAN:
+               return c.Booleans != nil
+       case client.INT32:
+               return c.Int32s != nil
+       case client.INT64, client.TIMESTAMP:
+               return c.Int64s != nil
+       case client.FLOAT:
+               return c.Float32s != nil
+       case client.DOUBLE:
+               return c.Float64s != nil
+       case client.DATE:
+               return c.Dates != nil
+       case client.TEXT, client.STRING:
+               return c.Strings != nil
+       case client.BLOB, client.OBJECT:
+               return c.Binaries != nil
+       default:
+               return false
+       }
+}
+
+func (c *TabletColumn) estimatedSize() int64 {
+       if !c.present() {
+               return 0
+       }
+       size := int64(64) + int64(len(c.Nulls))*8 + c.VariableBytes
+       switch c.DataType {
+       case client.BOOLEAN:
+               size += int64(len(c.Booleans))
+       case client.INT32, client.FLOAT, client.DATE:
+               size += int64(len(c.Int32s))*4 + int64(len(c.Float32s))*4 + 
int64(len(c.Dates))*4
+       case client.INT64, client.TIMESTAMP, client.DOUBLE:
+               size += int64(len(c.Int64s))*8 + int64(len(c.Float64s))*8
+       case client.TEXT, client.STRING:
+               size += int64(len(c.Strings))*16
+       case client.BLOB, client.OBJECT:
+               size += int64(len(c.Binaries))*24
+       default:
+               return 0
+       }
+       return size
+}
diff --git a/subscription/processor/column_align_processor.go 
b/subscription/processor/column_align_processor.go
index 9883578..6bbf46e 100644
--- a/subscription/processor/column_align_processor.go
+++ b/subscription/processor/column_align_processor.go
@@ -20,63 +20,96 @@
 package processor
 
 import (
-       "github.com/apache/iotdb-client-go/subscription"
-       "github.com/apache/iotdb-client-go/subscription/payload"
-       "github.com/apache/iotdb-client-go/subscription/internal/record"
-
+       "bytes"
        "errors"
        "fmt"
-       "strconv"
-       "strings"
+       "math"
        "sync"
+       "time"
 
+       "github.com/apache/iotdb-client-go/subscription"
+       "github.com/apache/iotdb-client-go/subscription/internal/record"
+       "github.com/apache/iotdb-client-go/subscription/payload"
        "github.com/apache/iotdb-client-go/v2/client"
 )
 
 // ColumnAlignProcessor forward-fills NULL columns with the latest known 
non-null
-// value for the same topic, device, and column. It does not retain messages.
-// A cached value fills a NULL only when its timestamp is strictly earlier;
-// an observed non-null value updates the cache at equal or greater timestamps.
-// Processing follows raw Tablet row order, matching Java, rather than Rows 
order.
-// Tree devices use the device path; table devices use database, table, and TAG
-// values for each row, including tables with no TAG columns. Use one instance
-// per consumer. Cache size grows with observed devices and columns until 
reset.
+// value for the same topic, device, and column. Decoded subscription messages
+// are changed in place under one batch lock; user-provided RecordBatch
+// implementations use a copying fallback. It does not retain messages.
 type ColumnAlignProcessor struct {
-       mu sync.Mutex
-       dialect subscription.Dialect
-       lastValues 
map[string]map[columnAlignDevice]map[columnAlignColumn]columnAlignValue
+       mu         sync.Mutex
+       dialect    subscription.Dialect
+       lastValues map[string]map[columnAlignDeviceKey][]columnAlignDeviceEntry
 }
 
-type columnAlignDevice struct {
+type columnAlignDeviceKey struct {
        database string
-       target string
-       tags string
+       target   string
+       tagHash  uint64
+}
+
+type columnAlignDeviceEntry struct {
+       tags   []any
+       values map[columnAlignColumn]columnAlignValue
 }
 
 type columnAlignColumn struct {
-       name string
+       name     string
        dataType client.TSDataType
 }
 
 type columnAlignValue struct {
        timestamp int64
-       value any
+       value     any
 }
 
-type columnAlignMessage struct {
-       message payload.SubscriptionMessage
-       batches []columnAlignBatch
+type columnAlignTablet interface {
+       target() string
+       rowCount() int
+       columns() []payload.Column
+       timestampAt(int) int64
+       valueAt(int, int) (any, bool)
+       setValueAt(int, int, any) bool
 }
 
-type columnAlignBatch struct {
-       database string
-       timeSelected bool
-       tablet record.Tablet
+type tabletReader struct{ tablet *record.Tablet }
+
+func (r tabletReader) target() string { return r.tablet.Target }
+func (r tabletReader) rowCount() int { return r.tablet.RowCount }
+func (r tabletReader) columns() []payload.Column { return r.tablet.Columns }
+func (r tabletReader) timestampAt(row int) int64 {
+       if r.tablet.Timestamps == nil {
+               return 0
+       }
+       return r.tablet.Timestamps[row]
+}
+func (r tabletReader) valueAt(row, column int) (any, bool) {
+       if column < 0 || column >= len(r.tablet.Values) {
+               return nil, false
+       }
+       return r.tablet.Values[column].Value(row)
+}
+func (r tabletReader) setValueAt(row, column int, value any) bool {
+       if column < 0 || column >= len(r.tablet.Values) {
+               return false
+       }
+       return r.tablet.Values[column].Set(row, value)
+}
+
+type tabletMutator struct{ tablet record.TabletMutator }
+
+func (m tabletMutator) target() string { return m.tablet.Target() }
+func (m tabletMutator) rowCount() int { return m.tablet.RowCount() }
+func (m tabletMutator) columns() []payload.Column { return m.tablet.Columns() }
+func (m tabletMutator) timestampAt(row int) int64 { return 
m.tablet.Timestamp(row) }
+func (m tabletMutator) valueAt(row, column int) (any, bool) { return 
m.tablet.Value(row, column) }
+func (m tabletMutator) setValueAt(row, column int, value any) bool {
+       return m.tablet.SetValue(row, column, value)
 }
 
 // NewColumnAlignProcessor creates a processor for one model. An empty dialect
-// selects subscription.TreeDialect, matching Java's no-argument constructor. 
Table consumers
-// must select subscription.TableDialect. Different databases never share 
cached table values.
+// selects subscription.TreeDialect, matching Java's no-argument constructor.
 func NewColumnAlignProcessor(dialect subscription.Dialect) 
(*ColumnAlignProcessor, error) {
        if dialect == "" {
                dialect = subscription.TreeDialect
@@ -85,34 +118,69 @@ func NewColumnAlignProcessor(dialect subscription.Dialect) 
(*ColumnAlignProcesso
                return nil, fmt.Errorf("subscription: invalid column alignment 
dialect %q", dialect)
        }
        return &ColumnAlignProcessor{
-               dialect: dialect,
-               lastValues: 
make(map[string]map[columnAlignDevice]map[columnAlignColumn]columnAlignValue),
+               dialect:    dialect,
+               lastValues: 
make(map[string]map[columnAlignDeviceKey][]columnAlignDeviceEntry),
        }, nil
 }
 
-// Process returns record messages with aligned, independently owned buffers.
-// It consumes and releases the original record messages only after every input
-// has been read successfully. Kind, commit identity, schema, raw row order, 
and
-// time projection are preserved. Watermarks pass through unchanged. Cached 
byte
-// values are copied so releasing an earlier message cannot invalidate the 
cache.
-// Access failures or model mismatches leave cache and input ownership 
unchanged.
+// Process returns aligned messages. For decoded subscription messages it
+// transfers the existing typed buffers to the result instead of copying the
+// Tablet. Kind, commit identity, schema, raw row order, and time projection 
are
+// preserved. Watermarks pass through unchanged.
 func (p *ColumnAlignProcessor) Process(messages []payload.SubscriptionMessage) 
([]payload.SubscriptionMessage, error) {
        p.mu.Lock()
        defer p.mu.Unlock()
-       incoming := make([]columnAlignMessage, 0, len(messages))
+       if err := p.validate(messages); err != nil {
+               return nil, err
+       }
+       processed, owned, err := record.TryMutateRecordMessages(messages, 
func(context payload.CommitContext, tablet record.TabletMutator) error {
+               return p.fillTablet(context.Topic, tablet.Database(), 
tabletMutator{tablet: tablet})
+       })
+       if owned {
+               if err != nil {
+                       return nil, err
+               }
+               return processed, nil
+       }
+       return p.copyAndAlign(messages)
+}
+
+func (p *ColumnAlignProcessor) validate(messages 
[]payload.SubscriptionMessage) error {
        for _, message := range messages {
                if message == nil {
-                       return nil, errors.New("subscription: column alignment 
received a nil message")
+                       return errors.New("subscription: column alignment 
received a nil message")
                }
-               entry := columnAlignMessage{message: message}
                if message.Kind() == payload.WatermarkMessage {
-                       incoming = append(incoming, entry)
+                       continue
+               }
+               batches, err := message.Records()
+               if err != nil {
+                       return err
+               }
+               for _, batch := range batches {
+                       if batch == nil {
+                               return errors.New("subscription: column 
alignment received a nil batch")
+                       }
+                       if batch.Dialect() != p.dialect {
+                               return fmt.Errorf("subscription: column 
alignment dialect %q does not match batch dialect %q", p.dialect, 
batch.Dialect())
+                       }
+               }
+       }
+       return nil
+}
+
+func (p *ColumnAlignProcessor) copyAndAlign(messages 
[]payload.SubscriptionMessage) ([]payload.SubscriptionMessage, error) {
+       incoming := make([]columnAlignCopiedMessage, 0, len(messages))
+       for _, message := range messages {
+               if message.Kind() == payload.WatermarkMessage {
+                       incoming = append(incoming, 
columnAlignCopiedMessage{message: message})
                        continue
                }
                batches, err := message.Records()
                if err != nil {
                        return nil, err
                }
+               entry := columnAlignCopiedMessage{message: message}
                for _, batch := range batches {
                        if batch == nil {
                                return nil, errors.New("subscription: column 
alignment received a nil batch")
@@ -121,14 +189,14 @@ func (p *ColumnAlignProcessor) Process(messages 
[]payload.SubscriptionMessage) (
                                return nil, fmt.Errorf("subscription: column 
alignment dialect %q does not match batch dialect %q", p.dialect, 
batch.Dialect())
                        }
                        tablet := record.Tablet{
-                               Target: batch.Target(),
-                               Columns: append([]payload.Column(nil), 
batch.Columns()...),
+                               Target:   batch.Target(),
+                               Columns:  append([]payload.Column(nil), 
batch.Columns()...),
                                RowCount: batch.Len(),
                        }
                        tablet.Timestamps = make([]int64, tablet.RowCount)
-                       tablet.Values = make([][]any, len(tablet.Columns))
-                       for column := range tablet.Values {
-                               tablet.Values[column] = make([]any, 
tablet.RowCount)
+                       tablet.Values = make([]record.TabletColumn, 
len(tablet.Columns))
+                       for column, schema := range tablet.Columns {
+                               tablet.Values[column] = 
record.NewTabletColumn(schema.DataType, tablet.RowCount)
                        }
                        for row := 0; row < tablet.RowCount; row++ {
                                tablet.Timestamps[row], err = 
batch.Timestamp(row)
@@ -140,13 +208,15 @@ func (p *ColumnAlignProcessor) Process(messages 
[]payload.SubscriptionMessage) (
                                        if err != nil {
                                                return nil, err
                                        }
-                                       tablet.Values[column][row] = 
cloneColumnAlignValue(value)
+                                       if !tablet.Values[column].Set(row, 
cloneColumnAlignValue(value)) {
+                                               return nil, 
fmt.Errorf("subscription: column alignment cannot store value for column %d", 
column)
+                                       }
                                }
                        }
-                       entry.batches = append(entry.batches, columnAlignBatch{
-                               database: batch.Database(),
+                       entry.batches = append(entry.batches, 
columnAlignCopiedBatch{
+                               database:     batch.Database(),
                                timeSelected: batch.TimeSelected(),
-                               tablet: tablet,
+                               tablet:       tablet,
                        })
                }
                incoming = append(incoming, entry)
@@ -157,10 +227,12 @@ func (p *ColumnAlignProcessor) Process(messages 
[]payload.SubscriptionMessage) (
                        result = append(result, entry.message)
                        continue
                }
-               context := entry.message.CommitContext()
+               context := record.ReadCommitContext(entry.message)
                var batches []payload.RecordBatch
                for _, batch := range entry.batches {
-                       p.fillTablet(context.Topic, batch.database, 
&batch.tablet)
+                       if err := p.fillTablet(context.Topic, batch.database, 
tabletReader{tablet: &batch.tablet}); err != nil {
+                               return nil, err
+                       }
                        batches = append(batches, 
record.NewRecordBatch(p.dialect, batch.database, batch.tablet, 
batch.timeSelected))
                }
                result = append(result, record.NewRecordMessage(context, 
batches))
@@ -173,92 +245,237 @@ func (p *ColumnAlignProcessor) Process(messages 
[]payload.SubscriptionMessage) (
        return result, nil
 }
 
-func (p *ColumnAlignProcessor) fillTablet(topic, database string, tablet 
*record.Tablet) {
+type columnAlignCopiedMessage struct {
+       message payload.SubscriptionMessage
+       batches []columnAlignCopiedBatch
+}
+
+type columnAlignCopiedBatch struct {
+       database     string
+       timeSelected bool
+       tablet       record.Tablet
+}
+
+func (p *ColumnAlignProcessor) fillTablet(topic, database string, tablet 
columnAlignTablet) error {
        topicCache := p.lastValues[topic]
        if topicCache == nil {
-               topicCache = 
make(map[columnAlignDevice]map[columnAlignColumn]columnAlignValue)
+               topicCache = 
make(map[columnAlignDeviceKey][]columnAlignDeviceEntry)
                p.lastValues[topic] = topicCache
        }
-       for row := 0; row < tablet.RowCount; row++ {
-               timestamp := tablet.Timestamps[row]
-               device := columnAlignDevice{target: tablet.Target}
+       for row := 0; row < tablet.rowCount(); row++ {
+               key := columnAlignDeviceKey{target: tablet.target()}
                if p.dialect == subscription.TableDialect {
-                       device.database = database
-                       device.tags = columnAlignTags(tablet, row)
+                       key.database = database
+                       key.tagHash = columnAlignTagHash(tablet, row)
                }
-               cache := topicCache[device]
+               bucket := topicCache[key]
+               cache, entry := columnAlignFindDevice(tablet, row, bucket, 
p.dialect == subscription.TableDialect)
                if cache == nil {
                        cache = make(map[columnAlignColumn]columnAlignValue)
-                       topicCache[device] = cache
+                       entry.values = cache
+                       topicCache[key] = append(bucket, entry)
                }
-               for column, schema := range tablet.Columns {
-                       // A schema type change cannot reuse an incompatible 
cached value.
-                       key := columnAlignColumn{name: schema.Name, dataType: 
schema.DataType}
-                       cached, ok := cache[key]
-                       value := tablet.Values[column][row]
+               timestamp := tablet.timestampAt(row)
+               for column, schema := range tablet.columns() {
+                       cellKey := columnAlignColumn{name: schema.Name, 
dataType: schema.DataType}
+                       cached, ok := cache[cellKey]
+                       value, valid := tablet.valueAt(row, column)
+                       if !valid {
+                               return fmt.Errorf("subscription: column 
alignment has no value for column %d", column)
+                       }
                        if value == nil {
-                               if ok && cached.timestamp < timestamp {
-                                       tablet.Values[column][row] = 
cloneColumnAlignValue(cached.value)
+                               if ok && cached.timestamp < timestamp && 
!tablet.setValueAt(row, column, cloneColumnAlignValue(cached.value)) {
+                                       return fmt.Errorf("subscription: column 
alignment cannot fill column %d", column)
                                }
                        } else if !ok || timestamp >= cached.timestamp {
-                               cache[key] = columnAlignValue{timestamp: 
timestamp, value: cloneColumnAlignValue(value)}
+                               cache[cellKey] = columnAlignValue{timestamp: 
timestamp, value: cloneColumnAlignValue(value)}
                        }
                }
        }
+       return nil
 }
 
-// columnAlignTags encodes ordered TAG segments without delimiter collisions.
-// Trailing NULLs are ignored, matching TsFile's table device identity; a NULL
-// segment remains distinct from an empty string and from embedded delimiters.
-func columnAlignTags(tablet *record.Tablet, row int) string {
-       var tags []any
+func columnAlignFindDevice(tablet columnAlignTablet, row int, bucket 
[]columnAlignDeviceEntry, table bool) (map[columnAlignColumn]columnAlignValue, 
columnAlignDeviceEntry) {
+       if !table {
+               if len(bucket) == 0 {
+                       return nil, columnAlignDeviceEntry{}
+               }
+               return bucket[0].values, bucket[0]
+       }
+       for _, entry := range bucket {
+               if columnAlignTagsEqual(tablet, row, entry.tags) {
+                       return entry.values, entry
+               }
+       }
+       tags := columnAlignTags(tablet, row)
+       return nil, columnAlignDeviceEntry{tags: tags}
+}
+
+const (
+       columnAlignHashOffset uint64 = 14695981039346656037
+       columnAlignHashPrime  uint64 = 1099511628211
+)
+
+func columnAlignTagHash(tablet columnAlignTablet, row int) uint64 {
+       lastNonNull := -1
+       for index, schema := range tablet.columns() {
+               if schema.Category != client.TAG {
+                       continue
+               }
+               value, _ := tablet.valueAt(row, index)
+               if value != nil {
+                       lastNonNull = index
+               }
+       }
+       hash := columnAlignHashOffset
+       for column, schema := range tablet.columns() {
+               if column > lastNonNull {
+                       break
+               }
+               if schema.Category != client.TAG {
+                       continue
+               }
+               value, _ := tablet.valueAt(row, column)
+               hash = columnAlignHashValue(hash, value)
+       }
+       return hash
+}
+
+func columnAlignTags(tablet columnAlignTablet, row int) []any {
+       tags := make([]any, 0)
        lastNonNull := 0
-       for column, schema := range tablet.Columns {
-               if schema.Category == client.TAG {
-                       value := tablet.Values[column][row]
-                       tags = append(tags, value)
+       for column, schema := range tablet.columns() {
+               if schema.Category != client.TAG {
+                       continue
+               }
+               value, _ := tablet.valueAt(row, column)
+               tags = append(tags, cloneColumnAlignValue(value))
+               if value != nil {
+                       lastNonNull = len(tags)
+               }
+       }
+       return tags[:lastNonNull]
+}
+
+func columnAlignTagsEqual(tablet columnAlignTablet, row int, tags []any) bool {
+       index := 0
+       lastNonNull := 0
+       for column, schema := range tablet.columns() {
+               if schema.Category != client.TAG {
+                       continue
+               }
+               value, _ := tablet.valueAt(row, column)
+               if index >= len(tags) {
                        if value != nil {
-                               lastNonNull = len(tags)
+                               return false
                        }
+               } else if !columnAlignEqualValue(tags[index], value) {
+                       return false
+               }
+               index++
+               if value != nil {
+                       lastNonNull = index
                }
        }
-       var key strings.Builder
-       for _, value := range tags[:lastNonNull] {
-               if value == nil {
-                       key.WriteString("-1:")
-                       continue
+       return lastNonNull == len(tags) && index >= len(tags)
+}
+
+func columnAlignHashValue(hash uint64, value any) uint64 {
+       mix := func(value uint64) uint64 {
+               for index := 0; index < 8; index++ {
+                       hash ^= (value >> uint(index*8)) & 0xff
+                       hash *= columnAlignHashPrime
                }
-               segment := fmt.Sprint(value)
-               if binary, ok := value.([]byte); ok {
-                       segment = string(binary)
+               return hash
+       }
+       if value == nil {
+               return mix(0)
+       }
+       switch value := value.(type) {
+       case bool:
+               if value {
+                       return mix(2)
                }
-               key.WriteString(strconv.Itoa(len(segment)))
-               key.WriteByte(':')
-               key.WriteString(segment)
+               return mix(1)
+       case int32:
+               return mix(3) ^ mix(uint64(uint32(value)))
+       case int64:
+               return mix(4) ^ mix(uint64(value))
+       case float32:
+               return mix(5) ^ mix(uint64(math.Float32bits(value)))
+       case float64:
+               return mix(6) ^ mix(math.Float64bits(value))
+       case string:
+               hash = mix(7)
+               for index := 0; index < len(value); index++ {
+                       hash ^= uint64(value[index])
+                       hash *= columnAlignHashPrime
+               }
+               return hash
+       case []byte:
+               hash = mix(8)
+               for _, item := range value {
+                       hash ^= uint64(item)
+                       hash *= columnAlignHashPrime
+               }
+               return hash
+       case time.Time:
+               return mix(9) ^ mix(uint64(value.Year())) ^ 
mix(uint64(value.Month())) ^ mix(uint64(value.Day()))
+       default:
+               return mix(10)
+       }
+}
+
+func columnAlignEqualValue(left, right any) bool {
+       switch left := left.(type) {
+       case nil:
+               return right == nil
+       case bool:
+               value, ok := right.(bool)
+               return ok && left == value
+       case int32:
+               value, ok := right.(int32)
+               return ok && left == value
+       case int64:
+               value, ok := right.(int64)
+               return ok && left == value
+       case float32:
+               value, ok := right.(float32)
+               return ok && math.Float32bits(left) == math.Float32bits(value)
+       case float64:
+               value, ok := right.(float64)
+               return ok && math.Float64bits(left) == math.Float64bits(value)
+       case string:
+               value, ok := right.(string)
+               return ok && left == value
+       case []byte:
+               value, ok := right.([]byte)
+               return ok && bytes.Equal(left, value)
+       case time.Time:
+               value, ok := right.(time.Time)
+               return ok && left.Equal(value)
+       default:
+               return false
        }
-       return key.String()
 }
 
 func cloneColumnAlignValue(value any) any {
        if binary, ok := value.([]byte); ok {
-               return append([]byte(nil), binary...)
+               return append([]byte{}, binary...)
        }
        return value
 }
 
-// Flush returns no messages because alignment does not buffer deliveries.
 func (p *ColumnAlignProcessor) Flush() ([]payload.SubscriptionMessage, error) {
        return nil, nil
 }
 
-// Reset clears cached values for every topic.
 func (p *ColumnAlignProcessor) Reset() {
        p.mu.Lock()
        defer p.mu.Unlock()
-       p.lastValues = 
make(map[string]map[columnAlignDevice]map[columnAlignColumn]columnAlignValue)
+       p.lastValues = 
make(map[string]map[columnAlignDeviceKey][]columnAlignDeviceEntry)
 }
 
-// ResetTopic clears only the target topic's cached alignment values.
 func (p *ColumnAlignProcessor) ResetTopic(topic string) {
        p.mu.Lock()
        defer p.mu.Unlock()
diff --git a/subscription/processor/watermark_processor.go 
b/subscription/processor/watermark_processor.go
index c15db1b..fef4a65 100644
--- a/subscription/processor/watermark_processor.go
+++ b/subscription/processor/watermark_processor.go
@@ -137,24 +137,39 @@ func (p *WatermarkProcessor) Process(messages 
[]payload.SubscriptionMessage) ([]
                        incoming = append(incoming, 
watermarkBufferedMessage{message: message, maximum: timestamp})
                        continue
                }
-               batches, err := message.Records()
+               maximum, found := int64(math.MinInt64), false
+               owned, err := record.TryReadRecordMessage(message, func(_ 
payload.CommitContext, batch record.TabletReader) error {
+                       for row := 0; row < batch.RowCount(); row++ {
+                               timestamp := batch.Timestamp(row)
+                               if !found || timestamp > maximum {
+                                       maximum = timestamp
+                               }
+                               found = true
+                       }
+                       return nil
+               })
                if err != nil {
                        return nil, err
                }
-               maximum, found := int64(math.MinInt64), false
-               for _, batch := range batches {
-                       if batch == nil {
-                               return nil, errors.New("subscription: watermark 
processor received a nil batch")
+               if !owned {
+                       batches, err := message.Records()
+                       if err != nil {
+                               return nil, err
                        }
-                       for row := 0; row < batch.Len(); row++ {
-                               timestamp, err := batch.Timestamp(row)
-                               if err != nil {
-                                       return nil, err
+                       for _, batch := range batches {
+                               if batch == nil {
+                                       return nil, errors.New("subscription: 
watermark processor received a nil batch")
                                }
-                               if !found || timestamp > maximum {
-                                       maximum = timestamp
+                               for row := 0; row < batch.Len(); row++ {
+                                       timestamp, err := batch.Timestamp(row)
+                                       if err != nil {
+                                               return nil, err
+                                       }
+                                       if !found || timestamp > maximum {
+                                               maximum = timestamp
+                                       }
+                                       found = true
                                }
-                               found = true
                        }
                }
                // As in Java, an empty message falls back to the wall clock in 
ms.
@@ -168,7 +183,7 @@ func (p *WatermarkProcessor) Process(messages 
[]payload.SubscriptionMessage) ([]
                incoming = append(incoming, watermarkBufferedMessage{message: 
message, maximum: maximum, size: size})
        }
        for _, entry := range incoming {
-               context := entry.message.CommitContext()
+               context := record.ReadCommitContext(entry.message)
                key := watermarkRegion{topic: context.Topic, node: 
context.DataNodeID, region: context.RegionID}
                previous, ok := p.regions[key]
                if !ok || entry.maximum > previous.timestamp {
@@ -222,9 +237,9 @@ func (p *WatermarkProcessor) Buffered() ProcessorBuffer {
        defer p.mu.Unlock()
        result := ProcessorBuffer{Count: len(p.buffer)}
        for _, entry := range p.buffer {
-               context := entry.message.CommitContext()
+               context := record.ReadCommitContext(entry.message)
                if record.Committable(context) {
-                       result.Contexts = append(result.Contexts, context)
+                       result.Contexts = append(result.Contexts, 
record.CloneCommitContext(context))
                }
        }
        return result
@@ -259,7 +274,7 @@ func (p *WatermarkProcessor) ResetTopic(topic string) {
        defer p.mu.Unlock()
        retained := p.buffer[:0]
        for _, entry := range p.buffer {
-               if entry.message.CommitContext().Topic == topic {
+               if record.ReadCommitContext(entry.message).Topic == topic {
                        p.bufferedBytes -= entry.size
                        entry.message.Release()
                } else {

Reply via email to