[jira] [Comment Edited] (IGNITE-28356) Get rid of redundant wire companion fields and hand-written marshalling in messages
[
https://issues.apache.org/jira/browse/IGNITE-28356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18100512#comment-18100512
]
Anton Vinogradov edited comment on IGNITE-28356 at 7/30/26 6:00 PM:
[Ignite PR Checker|https://ignite-pr-checker.is-a.dev/?pr=13408] verdict for PR
13408 · RunAll build [9240744|https://ci2.ignite.apache.org/build/9240744] ·
147 suites ran, 0 reused
(/) *No blockers* — nothing in this run looks caused by this PR. 28
pre-existing/flaky tests filtered out.
♻️ Settled after 1 auto re-run wave(s): #1 — 1 broken suite(s).
was (Author: av):
[Ignite PR Checker|https://ignite-pr-checker.is-a.dev/?pr=13408] verdict for PR
13408 · RunAll build [9240744|https://ci2.ignite.apache.org/build/9240744] ·
147 suites ran, 0 reused
(!) *1 broken suite(s)* (failed without a reliable run):
- Snapshots 6: JVM crash / out of memory · 1 failed test detected
(/) No test blockers otherwise; 28 pre-existing/flaky filtered out.
⏳ _Auto re-run *#1* in progress — 1 broken suite(s) re-queued (attempt 1/2), *≈
settled by 20:58 MSK*. This comment updates when they settle._
> Get rid of redundant wire companion fields and hand-written marshalling in
> messages
> ---
>
> Key: IGNITE-28356
> URL: https://issues.apache.org/jira/browse/IGNITE-28356
> Project: Ignite
> Issue Type: Task
>Reporter: Ilya Shishkov
>Assignee: Anton Vinogradov
>Priority: Minor
> Labels: IEP-132, ise
> Fix For: 2.19
>
> Time Spent: 18h 20m
> Remaining Estimate: 0h
>
> The message codegen (IEP-132) can marshal a logical field into its wire
> companion by itself, yet many
> messages still keep both the field and a hand-written
> \{{marshal()}}/\{{unmarshal()}} pair that fills the
> companion. Some of those pairs carry no logic at all, and some of the
> companions describe data that is
> never sent. Each section below is a problem found and the change that closes
> it.
> h3. A map whose values are always null
> {\{GridDistributedTxPrepareRequest#dhtVers}} was a map plus two companions.
> All three producers put
> {\{null}} as the value, so the consumer's \{{ver.getValue() == null ||
> !equals}} is always true.
> *Fix:* the field becomes \{{Collection dhtVerKeys}}. The value
> companion is gone and the
> request is \{{4 + 2N}} bytes shorter.
> h3. A key travelling next to its own value
> {\{GridNearTxPrepareResponse#ownedVals}} was a map plus two companions, and
> \{{CacheVersionedValue}}
> already carries \{{cacheId}}, so the cache id travelled twice.
> *Fix:* the key moves into the value as the new \{{KeyedVersionedValue}}, and
> the field becomes a single
> collection. The response is \{{4 + 6N}} bytes shorter.
> h3. An array companion for an ordered set
> {\{TxLocksRequest#txKeys}} and \{{TxLocksResponse#txKeys}} kept a \{{Set}}
> plus an array to write it, even
> though \{{writeCollection}} and \{{writeObjectArray}} emit identical bytes.
> *Fix:* the companions are gone. The runtime type stays \{{HashSet}}, so the
> sender-side dedup is
> preserved and the wire format does not move.
> h3. A hand-written zip of two parallel fields
> {\{GridNearGetRequest#keyMap}} was assembled by pairing \{{keys}} and
> \{{readersFlags}} by index, and taken
> apart again on the way out.
> *Fix:* the map stays where the code reads better, and the conversion moves
> into
> {\{@Marshalled(keys = ..., values = ...)}}.
> h3. A map copied entry by entry into an empty one
> {\{GridDhtTxPrepareRequest#owned}} was drained into \{{GridNearTxRemote}}
> with \{{putAll}}, although the
> request is done with it by then.
> *Fix:* \{{ownedVersions}} takes the map over instead of copying it, which
> drops a \{{GridLeanMap}} and
> {\{N}} nodes per prepare.
> h3. Blob marshalling the codegen already expresses
> {\{LazyServiceConfigurationMessage#affKey}}, \{{GenericValueMessage#val}},
> \{{QueryStartRequest#params}}
> and \{{QueryEntityMessage#dfltFieldValues}} each had a hand-written pair
> doing nothing but
> {\{U.marshal}}/\{{U.unmarshal}}.
> *Fix:* all four become \{{@Marshalled}}; the generated code matches the
> hand-written one call for call.
> {\{QueryEntityMessage}} keeps its \{{!F.isEmpty}} gate, moved into the
> constructor, because
> {\{QueryEntity}} initializes the defaults with an empty map while the
> generated gate is \{{!= null}}.
> h3. Collection marshalling in an atomic request
> {\{GridNearAtomicFullUpdateRequest#entryProcessors}} was marshalled by hand,
> so the request carried both
> the objects and their bytes until it was collected.
> *Fix:* the field becomes \{{@Marshalled}}, retyped to \{{List}}
> because the generated code calls
> {\{add(Object)}}. Two points worth reviewing:
> * the
[jira] [Comment Edited] (IGNITE-28356) Get rid of redundant wire companion fields and hand-written marshalling in messages
[
https://issues.apache.org/jira/browse/IGNITE-28356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18100076#comment-18100076
]
Anton Vinogradov edited comment on IGNITE-28356 at 7/29/26 2:26 PM:
[Ignite PR Checker|https://ignite-pr-checker.is-a.dev/?pr=13408] verdict for PR
13408 · RunAll build [9236765|https://ci2.ignite.apache.org/build/9236765] ·
147 suites ran, 0 reused
(x) *1 blocker(s) in 1 suite(s):*
- Cache 2: {{org.apache.ignite.testsuites.IgniteCacheTestSuite2:
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxPrepareDestroyedCacheTest.testPrepareOfDestroyedCacheAnsweredWithErrorResponse}}
♻️ Settled after 2 auto re-run wave(s): #1 — 2 blocker suite(s); #2 — 1 blocker
suite(s).
was (Author: av):
[Ignite PR Checker|https://ignite-pr-checker.is-a.dev/?pr=13408] verdict for PR
13408 · RunAll build [9236765|https://ci2.ignite.apache.org/build/9236765] ·
147 suites ran, 0 reused
(x) *1 blocker(s) in 1 suite(s):*
- Cache 2: {{org.apache.ignite.testsuites.IgniteCacheTestSuite2:
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxPrepareDestroyedCacheTest.testPrepareOfDestroyedCacheAnsweredWithErrorResponse}}
⏳ _Auto re-run *#2* in progress — 1 blocker suite(s) re-queued (attempt 2/2).
This comment updates when they settle._
_Earlier re-runs: #1 — 2 blocker suite(s)._
> Get rid of redundant wire companion fields and hand-written marshalling in
> messages
> ---
>
> Key: IGNITE-28356
> URL: https://issues.apache.org/jira/browse/IGNITE-28356
> Project: Ignite
> Issue Type: Task
>Reporter: Ilya Shishkov
>Assignee: Anton Vinogradov
>Priority: Minor
> Labels: IEP-132, ise
> Fix For: 2.19
>
> Time Spent: 15h
> Remaining Estimate: 0h
>
> The message codegen (IEP-132) can marshal a logical field into its wire
> companion by itself, yet many
> messages still keep both the field and a hand-written
> \{{marshal()}}/\{{unmarshal()}} pair that fills the
> companion. Some of those pairs carry no logic at all, and some of the
> companions describe data that is
> never sent. Each section below is a problem found and the change that closes
> it.
> h3. A map whose values are always null
> {\{GridDistributedTxPrepareRequest#dhtVers}} was a map plus two companions.
> All three producers put
> {\{null}} as the value, so the consumer's \{{ver.getValue() == null ||
> !equals}} is always true.
> *Fix:* the field becomes \{{Collection dhtVerKeys}}. The value
> companion is gone and the
> request is \{{4 + 2N}} bytes shorter.
> h3. A key travelling next to its own value
> {\{GridNearTxPrepareResponse#ownedVals}} was a map plus two companions, and
> \{{CacheVersionedValue}}
> already carries \{{cacheId}}, so the cache id travelled twice.
> *Fix:* the key moves into the value as the new \{{KeyedVersionedValue}}, and
> the field becomes a single
> collection. The response is \{{4 + 6N}} bytes shorter.
> h3. An array companion for an ordered set
> {\{TxLocksRequest#txKeys}} and \{{TxLocksResponse#txKeys}} kept a \{{Set}}
> plus an array to write it, even
> though \{{writeCollection}} and \{{writeObjectArray}} emit identical bytes.
> *Fix:* the companions are gone. The runtime type stays \{{HashSet}}, so the
> sender-side dedup is
> preserved and the wire format does not move.
> h3. A hand-written zip of two parallel fields
> {\{GridNearGetRequest#keyMap}} was assembled by pairing \{{keys}} and
> \{{readersFlags}} by index, and taken
> apart again on the way out.
> *Fix:* the map stays where the code reads better, and the conversion moves
> into
> {\{@Marshalled(keys = ..., values = ...)}}.
> h3. A map copied entry by entry into an empty one
> {\{GridDhtTxPrepareRequest#owned}} was drained into \{{GridNearTxRemote}}
> with \{{putAll}}, although the
> request is done with it by then.
> *Fix:* \{{ownedVersions}} takes the map over instead of copying it, which
> drops a \{{GridLeanMap}} and
> {\{N}} nodes per prepare.
> h3. Blob marshalling the codegen already expresses
> {\{LazyServiceConfigurationMessage#affKey}}, \{{GenericValueMessage#val}},
> \{{QueryStartRequest#params}}
> and \{{QueryEntityMessage#dfltFieldValues}} each had a hand-written pair
> doing nothing but
> {\{U.marshal}}/\{{U.unmarshal}}.
> *Fix:* all four become \{{@Marshalled}}; the generated code matches the
> hand-written one call for call.
> {\{QueryEntityMessage}} keeps its \{{!F.isEmpty}} gate, moved into the
> constructor, because
> {\{QueryEntity}} initializes the defaults with an empty map while the
> generated gate is \{{!= null}}.
> h3. Collection marshalling in an atomic request
> {\{GridNearAtomicFullUpdateRequest#entryProcessors}} was marshalled by hand,
> so
[jira] [Comment Edited] (IGNITE-28356) Get rid of redundant wire companion fields and hand-written marshalling in messages
[
https://issues.apache.org/jira/browse/IGNITE-28356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18100076#comment-18100076
]
Anton Vinogradov edited comment on IGNITE-28356 at 7/29/26 1:55 PM:
[Ignite PR Checker|https://ignite-pr-checker.is-a.dev/?pr=13408] verdict for PR
13408 · RunAll build [9236765|https://ci2.ignite.apache.org/build/9236765] ·
147 suites ran, 0 reused
(x) *1 blocker(s) in 1 suite(s):*
- Cache 2: {{org.apache.ignite.testsuites.IgniteCacheTestSuite2:
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxPrepareDestroyedCacheTest.testPrepareOfDestroyedCacheAnsweredWithErrorResponse}}
⏳ _Auto re-run *#2* in progress — 1 blocker suite(s) re-queued (attempt 2/2).
This comment updates when they settle._
_Earlier re-runs: #1 — 2 blocker suite(s)._
was (Author: av):
[Ignite PR Checker|https://ignite-pr-checker.is-a.dev/?pr=13408] verdict for PR
13408 · RunAll build [9236765|https://ci2.ignite.apache.org/build/9236765] ·
147 suites ran, 0 reused
(x) *2 blocker(s) in 2 suite(s):*
- Queries 5: {{org.apache.ignite.testsuites.IgniteBinaryCacheQueryTestSuite5:
org.apache.ignite.internal.processors.cache.index.DynamicEnableIndexingBasicSelfTest.testEnableDynamicIndexing[hasNear=false,nodeIdx=3,cacheMode=PARTITIONED,atomicityMode=ATOMIC]}}
- Cache 2: {{org.apache.ignite.testsuites.IgniteCacheTestSuite2:
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxPrepareDestroyedCacheTest.testPrepareOfDestroyedCacheAnsweredWithErrorResponse}}
⏳ _Auto re-run *#1* in progress — 2 blocker suite(s) re-queued (attempt 1/2).
This comment updates when they settle._
> Get rid of redundant wire companion fields and hand-written marshalling in
> messages
> ---
>
> Key: IGNITE-28356
> URL: https://issues.apache.org/jira/browse/IGNITE-28356
> Project: Ignite
> Issue Type: Task
>Reporter: Ilya Shishkov
>Assignee: Anton Vinogradov
>Priority: Minor
> Labels: IEP-132, ise
> Fix For: 2.19
>
> Time Spent: 15h
> Remaining Estimate: 0h
>
> The message codegen (IEP-132) can marshal a logical field into its wire
> companion by itself, yet many
> messages still keep both the field and a hand-written
> \{{marshal()}}/\{{unmarshal()}} pair that fills the
> companion. Some of those pairs carry no logic at all, and some of the
> companions describe data that is
> never sent. Each section below is a problem found and the change that closes
> it.
> h3. A map whose values are always null
> {\{GridDistributedTxPrepareRequest#dhtVers}} was a map plus two companions.
> All three producers put
> {\{null}} as the value, so the consumer's \{{ver.getValue() == null ||
> !equals}} is always true.
> *Fix:* the field becomes \{{Collection dhtVerKeys}}. The value
> companion is gone and the
> request is \{{4 + 2N}} bytes shorter.
> h3. A key travelling next to its own value
> {\{GridNearTxPrepareResponse#ownedVals}} was a map plus two companions, and
> \{{CacheVersionedValue}}
> already carries \{{cacheId}}, so the cache id travelled twice.
> *Fix:* the key moves into the value as the new \{{KeyedVersionedValue}}, and
> the field becomes a single
> collection. The response is \{{4 + 6N}} bytes shorter.
> h3. An array companion for an ordered set
> {\{TxLocksRequest#txKeys}} and \{{TxLocksResponse#txKeys}} kept a \{{Set}}
> plus an array to write it, even
> though \{{writeCollection}} and \{{writeObjectArray}} emit identical bytes.
> *Fix:* the companions are gone. The runtime type stays \{{HashSet}}, so the
> sender-side dedup is
> preserved and the wire format does not move.
> h3. A hand-written zip of two parallel fields
> {\{GridNearGetRequest#keyMap}} was assembled by pairing \{{keys}} and
> \{{readersFlags}} by index, and taken
> apart again on the way out.
> *Fix:* the map stays where the code reads better, and the conversion moves
> into
> {\{@Marshalled(keys = ..., values = ...)}}.
> h3. A map copied entry by entry into an empty one
> {\{GridDhtTxPrepareRequest#owned}} was drained into \{{GridNearTxRemote}}
> with \{{putAll}}, although the
> request is done with it by then.
> *Fix:* \{{ownedVersions}} takes the map over instead of copying it, which
> drops a \{{GridLeanMap}} and
> {\{N}} nodes per prepare.
> h3. Blob marshalling the codegen already expresses
> {\{LazyServiceConfigurationMessage#affKey}}, \{{GenericValueMessage#val}},
> \{{QueryStartRequest#params}}
> and \{{QueryEntityMessage#dfltFieldValues}} each had a hand-written pair
> doing nothing but
> {\{U.marshal}}/\{{U.unmarshal}}.
> *Fix:* all four become \{{@Marshalled}}; the generated code matches the
> hand-written one call for call.
> {\{QueryEntityMessage}} keeps its \{{!F.isEmpty}}
