Hi Calvin, It seems that the code is incomplete. First, what are the Instant, DeltaMap types? What are the types stored in the Map fields? Second, do you have some code processing tool generating some code for you? E.g. I see that version() implementation is missing in WorkingSetLedgerDelta – I assume it’s generated.
The reason why OptimizedMarshaller is used is most likely that some parts of your objects are Serializable with custom serialization (readObject()/writeObject() methods). Also, OptimizedMarshaller may be used when processing JDK classes. The logs should have a message about that though. Are you checking Ignite’s stdout or the log files? Ignite starts in a quiet mode by default, so not all messages are in the stdout. There is a path to the full log file printed in the stdout. I’m still not sure whether I grasp your use case in full though. Any chance you could create a small reproducer project and share it on GitHub? Thanks, Stan From: Calvin KL Wong, CLSA Sent: 10 июля 2018 г. 6:06 To: user@ignite.apache.org Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is usedforScanQuery Hi Stan, Thanks for your response. >> How did you check which marshaller is used in each case? i) from stack trace ii) step with a debugger >> Can you share the code of your POJO? Attached pojo.zip contains skeleton of the POJO. The object that was passed between ignite nodes is called WorkingSetLedgerDelta. It contains objects of other classes. >> Can you also share the logs/check them for warnings? There might be a >> warning saying that there is a problem in using BinaryMarshaller. I just checked them – did a grep of ‘-i marsh’, ‘-i warn’, ‘-i except’, ‘-i error’; no marshaller related problem found. Background: A newer version of NewOrderRequest extends AbstractRequest is deployed on the service node, but not on the client node. Client node sends a scan query to server node to get all WorkingSetLedgerDelta, which contains Request objects. About the attached stack traces of the client node 1. exception.when.using.pojo.with.scan.query.log Cache: IgniteCache<LedgerKey, LedgerDelta> Caught exception when trying to deserialize WorkingSetLedgerDelta 2. forced.exception.when.using.binary.object.with.scan.query.with.keep.binary.log Cache: IgniteCache<BinaryObject, BinaryObject>.withKeepBinary() I forced an exception from eclipse when I saw that we successfully unmarshalled bytes into the BinaryObjectImpl of WorkingSetLedgerDelta. The main difference between the two stack traces is at: with POJO, the logic unmarshalls WorkingSetLedgerDelta using ‘readSerializable’, which cannot handle versioning. at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:601) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:927) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) ~[?:1.8.0_152] at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] withKeepBinary, BinaryObjImpl object was received, the logic could unmarshall the BinaryObjImpl of WorkingSetLedgerDelta with no problem. at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:558) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:917) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) ~[?:1.8.0_152] at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59] Thanks, Calvin From: Stanislav Lukyanov [mailto:stanlukya...@gmail.com] Sent: Monday, July 09, 2018 9:04 PM To: user@ignite.apache.org Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery Hi Calvin, It should work the same for all queries. Ideally OptimizedMarshaller shouldn’t even be used (except for JDK classes). How did you check which marshaller is used in each case? Can you share the code of your POJO? Or perhaps a runnable reproducer? Can you also share the logs/check them for warnings? There might be a warning saying that there is a problem in using BinaryMarshaller. Thanks, Stan From: Calvin KL Wong, CLSA Sent: 9 июля 2018 г. 13:40 To: user@ignite.apache.org Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery Ok, I found that I can use binaryObject.deserialize() to get back the POJO. So, the only remaining question is: When a cache does not have ‘withKeepBinary’ set, why ContinuousQuery can handle class versioning fine whereas ScanQuery cannot? Is that an expected behaviour? Thanks, Calvin From: Calvin KL Wong, CLSA [mailto:calvin.kl.w...@clsa.com] Sent: Monday, July 09, 2018 11:50 AM To: user@ignite.apache.org Subject: OptimizedMarshaller instead of BinaryMarshaller is used for ScanQuery Hi, I have a cache of <Integer, POJO>. I found that when I use a ScanQuery on that cache, result will be deserialized using OptimizedMarshaller; Ignite sends ‘ordered’ message using GridCacheQueryResonseEntry. Whereas if I use ContinousQuery, result will be deserialized using BinaryMarshaller. My problem is that ScanQuery won’t be able to handle class versioning. Question: 1. I am using “keepBinary” and BinaryObject on the cache to avoid versioning related problem. Is that the recommended approach? Because it seems strange that “ScanQuery” and “ContinousQuery” have different behaviour. 2. What is the recommended way to convert BinaryObject into POJO? Ignite website talks about getting field of each object using ‘BinaryObject.field()’. However, Ignite uses ‘BinaryReaderExImpl implements BinaryReader’ internally. How can I get a reference to that object? Thanks, Calvin Calvin KL Wong Sr. Lead Engineer, Execution Services D +852 2600 7983 | M +852 9267 9471 | T +852 2600 8888 5/F, One Island East, 18 Westlands Road, Island East, Hong Kong clsa.com Insights. Liquidity. Capital. A CITIC Securities Company The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices. These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request. Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment. The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices. These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request. Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment. The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices. These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request. Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
