[
https://issues.apache.org/jira/browse/IGNITE-28413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18070662#comment-18070662
]
Pavel Tupitsyn edited comment on IGNITE-28413 at 4/2/26 6:25 PM:
-----------------------------------------------------------------
* The benchmark only performs read operations (getAll on binary KV view), in 16
threads, sharing one client instance => the only writes come from keys and data
comes from socket. TBD check how the key looks like - is "field7" a part of key
or value?
* The servers run as separate processes and can't affect Netty on the client
* Netty ByteBuf can't be used once released (it throws an exception) =>
use-after-free ruled out
* Multiple requests failed with a similar error at the same time, but the
payload is different in every error. The endpoint is the same. Looks like
multiple responses have arrived, request headers were decoded on the Netty
thread correctly, then we jump to the continuation thread and see garbage data
there (readTx is the first thing we do on another thread).
was (Author: ptupitsyn):
* The benchmark only performs read operations (getAll on binary KV view), in 16
threads, sharing one client instance => the only writes come from keys and data
comes from socket. TBD check how the key looks like - is "field7" a part of key
or value?
* The servers run as separate processes and can't affect Netty on the client
* Netty ByteBuf can't be used once released (it throws an exception) =>
use-after-free ruled out
* Multiple requests failed with a similar error at the same time, but the
payload is different in every error. The endpoint is the same. Looks like
multiple responses have arrived, request headers were decoded on the Netty
thread correctly, then we jump to. continuation thread and see garbage data
there.
> Java client: Failed to deserialize server response for getAll. Expected Ext,
> but got Integer
> --------------------------------------------------------------------------------------------
>
> Key: IGNITE-28413
> URL: https://issues.apache.org/jira/browse/IGNITE-28413
> Project: Ignite
> Issue Type: Bug
> Components: thin clients ai3, transactions ai3
> Reporter: Pavel Tupitsyn
> Assignee: Pavel Tupitsyn
> Priority: Critical
> Labels: ignite-3
> Fix For: 3.2
>
>
> Under load in benchmark run on GET_ALL op:
> {code}
> SEVERE: Failed to deserialize server response
> [remoteAddress=172.25.4.97/172.25.4.97:10800, opCode=15]: Expected Ext, but
> got Integer (32) at pos 14: '6669656c64373a3231313638383532'
> org.msgpack.core.MessageTypeException: Expected Ext, but got Integer (32) at
> pos 14: '6669656c64373a3231313638383532'
> at
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unexpected(ClientMessageUnpacker.java:100)
> at
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unpackExtensionTypeHeader(ClientMessageUnpacker.java:430)
> at
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unpackUuid(ClientMessageUnpacker.java:682)
> at
> org.apache.ignite.internal.client.tx.DirectTxUtils.readTx(DirectTxUtils.java:209)
> at
> org.apache.ignite.internal.client.table.ClientTable.readSchemaAndReadData(ClientTable.java:785)
> at
> org.apache.ignite.internal.client.table.ClientTable.lambda$doSchemaOutInOpAsync$8(ClientTable.java:684)
> at
> org.apache.ignite.internal.client.TcpClientChannel.complete(TcpClientChannel.java:528)
> at
> org.apache.ignite.internal.client.TcpClientChannel.lambda$completeAsync$6(TcpClientChannel.java:499)
> at
> java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1423)
> at
> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
> at
> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
> at
> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
> at
> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
> at
> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
> {code}
> or
> {code}
> SEVERE: Failed to deserialize server response
> [remoteAddress=172.25.4.97/172.25.4.97:10800, opCode=15]: Expected boolean,
> but got Integer (73) at pos 22108:
> '38323832303934333a38363435303230373a3233373633383133313a323035393231383632373a2d323033353530373738393a2d313738757365723833303133373839313036313434343737373a6669656c64383a2d313534373732323139373a2d313938383430323636333a3337353333373334353a2d313534323733383936333a2d3835363432373100006adfce005e080100d30000000000000000ce002f0439d8031800a69d993a9d0100000000964e01acd8036c4e604f40bc0ab24bcaaf32c12dfd97cdea60011ac3c504160117007b00df004301a7010b026f02d30237039b03ff0375736572343738323030323430343538393738393939367573'
> org.msgpack.core.MessageTypeException: Expected boolean, but got Integer (73)
> at pos 22108:
> '38323832303934333a38363435303230373a3233373633383133313a323035393231383632373a2d323033353530373738393a2d313738757365723833303133373839313036313434343737373a6669656c64383a2d313534373732323139373a2d313938383430323636333a3337353333373334353a2d313534323733383936333a2d3835363432373100006adfce005e080100d30000000000000000ce002f0439d8031800a69d993a9d0100000000964e01acd8036c4e604f40bc0ab24bcaaf32c12dfd97cdea60011ac3c504160117007b00df004301a7010b026f02d30237039b03ff0375736572343738323030323430343538393738393939367573'
> at
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unexpected(ClientMessageUnpacker.java:100)
> at
> org.apache.ignite.internal.client.proto.ClientMessageUnpacker.unpackBoolean(ClientMessageUnpacker.java:233)
> at
> org.apache.ignite.internal.client.table.ClientTupleSerializer.readKvTuplesNullable(ClientTupleSerializer.java:433)
> at
> org.apache.ignite.internal.client.table.ClientKeyValueBinaryView.lambda$getAllAsync$3(ClientKeyValueBinaryView.java:129)
> at
> org.apache.ignite.internal.client.table.ClientTable.readSchemaAndReadData(ClientTable.java:804)
> at
> org.apache.ignite.internal.client.table.ClientTable.lambda$doSchemaOutInOpAsync$8(ClientTable.java:684)
> at
> org.apache.ignite.internal.client.TcpClientChannel.complete(TcpClientChannel.java:529)
> at
> org.apache.ignite.internal.client.TcpClientChannel.lambda$completeAsync$6(TcpClientChannel.java:500)
> at
> java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1423)
> at
> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
> at
> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
> at
> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
> at
> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
> at
> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
