Ilya,
Yes, Rafael confirmed that the peer-class-loading is set.
Still does not work, this time complaining that the MyFilter class is not
found.
PeerClassLoading is active as far as I can tell. At least in both
server/client ignite config
Is this a known limitation or regression? Should we file an feature request
or bug?
Denis
On Thursday, July 2, 2020, Ilya Kasnacheev <ilya.kasnach...@gmail.com>
wrote:
> Hello!
>
> Did you enable peer class loading? You need to either enable peer class
> loading, or put your filter class(es) to all server nodes.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 2 июл. 2020 г. в 15:09, Rafael Troilo <rafael.tro...@heigit.org>:
>
>> Hi Denis,
>>
>> thank you for your quick response. Unfortunately even avoiding lambdas or
>> anonymous classes didn't solve this problem.
>> And was my first thought for this problem too.
>>
>> With:
>>
>> MyFilter.java
>>
>> import org.apache.ignite.lang.IgniteClosure;
>>
>> public class MyFilter<T> implements IgniteClosure<T, Long> {
>>
>> private static final long serialVersionUID = 1L;
>> private final long l;
>>
>> public MyFilter(long l) {
>> this.l = l;
>> }
>>
>> @Override
>> public Long apply(T e) {
>> return l;
>> }
>>
>> }
>>
>>
>> and
>>
>> try (Ignite ignite = Ignition.start("ignite.xml")) {
>> IgniteCache<Integer, String> cache = ignite.getOrCreateCache("
>> ScanTransformTest");
>> ...
>> // 2 Running the ScanQuery on the remote server. Works!
>> compute.broadcast(() -> {
>> try (QueryCursor<Long> cursor = cache.query(new
>> ScanQuery<>().setLocal(true), new MyFilter<>(234))) {
>> for (Long row : cursor) {
>> return row;
>> }
>> }
>> return 0;
>> })
>>
>> // 3 Still does not work.
>> try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), new
>> MyFilter<>(345L))) {
>> for (Long row : cursor) {
>> System.out.println(row);
>> break;
>> }
>> }
>> }
>>
>> Still does not work, this time complaining that the MyFilter class is not
>> found.
>> PeerClassLoading is active as far as I can tell. At least in both
>> server/client ignite config
>> ...
>> <property name="peerClassLoadingEnabled" value="true"/>
>> ...
>> it is enabled.
>>
>> I'm also not sure why the same class (IgniteClosure) works in the other
>> case but not in the Transformer or whats the differnt in case 2 to 3.
>> I guess making the MyFilter class available on the server will solve this
>> problem, but should not peerClassLoading work here?
>>
>> Is there any information I can provide or test I should do to help to
>> solve this problem.
>>
>> Thank you very much.
>>
>>
>> Stack Trace:
>> Exception in thread "main" javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: MyFilter
>> at org.apache.ignite.internal.processors.cache.GridCacheUtils.
>> convertToCacheException(GridCacheUtils.java:1317)
>> at org.apache.ignite.internal.processors.cache.query.
>> GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>> at org.apache.ignite.internal.processors.cache.query.
>> GridCacheDistributedQueryManager$5.onHasNext(
>> GridCacheDistributedQueryManager.java:645)
>> at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.
>> hasNextX(GridCloseableIteratorAdapter.java:53)
>> at org.apache.ignite.internal.util.lang.GridIteratorAdapter.
>> hasNext(GridIteratorAdapter.java:45)
>> at org.apache.ignite.internal.processors.cache.
>> AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>> at ScanQueryTest.main(ScanQueryTest.java:57)
>> Caused by: class org.apache.ignite.IgniteCheckedException: MyFilter
>> at org.apache.ignite.internal.util.IgniteUtils.unmarshal(
>> IgniteUtils.java:10310)
>> at org.apache.ignite.internal.processors.cache.query.
>> GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
>> at org.apache.ignite.internal.processors.cache.
>> GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
>> at org.apache.ignite.internal.processors.cache.
>> GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
>> at org.apache.ignite.internal.processors.cache.
>> GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
>> at org.apache.ignite.internal.processors.cache.
>> GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
>> at org.apache.ignite.internal.processors.cache.
>> GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>> at org.apache.ignite.internal.processors.cache.
>> GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
>> at org.apache.ignite.internal.managers.communication.
>> GridIoManager.invokeListener(GridIoManager.java:1843)
>> at org.apache.ignite.internal.managers.communication.
>> GridIoManager.processRegularMessage0(GridIoManager.java:1468)
>> at org.apache.ignite.internal.managers.communication.
>> GridIoManager.access$5200(GridIoManager.java:229)
>> at org.apache.ignite.internal.managers.communication.
>> GridIoManager$9.run(GridIoManager.java:1365)
>> at java.base/java.util.concurrent.ThreadPoolExecutor.
>> runWorker(ThreadPoolExecutor.java:1128)
>> at java.base/java.util.concurrent.ThreadPoolExecutor$
>> Worker.run(ThreadPoolExecutor.java:628)
>> at java.base/java.lang.Thread.run(Thread.java:834)
>> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>> MyFilter
>> at org.apache.ignite.internal.binary.BinaryContext.
>> descriptorForTypeId(BinaryContext.java:724)
>> at org.apache.ignite.internal.binary.BinaryContext.
>> descriptorForTypeId(BinaryContext.java:721)
>> at org.apache.ignite.internal.binary.BinaryReaderExImpl.
>> deserialize0(BinaryReaderExImpl.java:1757)
>> at org.apache.ignite.internal.binary.BinaryReaderExImpl.
>> deserialize(BinaryReaderExImpl.java:1716)
>> at org.apache.ignite.internal.binary.GridBinaryMarshaller.
>> deserialize(GridBinaryMarshaller.java:307)
>> at org.apache.ignite.internal.binary.BinaryMarshaller.
>> unmarshal0(BinaryMarshaller.java:101)
>> at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshalle
>> r.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
>> at org.apache.ignite.internal.util.IgniteUtils.unmarshal(
>> IgniteUtils.java:10304)
>> ... 14 more
>> Caused by: java.lang.ClassNotFoundException: MyFilter
>> at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(
>> BuiltinClassLoader.java:581)
>> at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.
>> loadClass(ClassLoaders.java:178)
>> at java.base/java.lang.ClassLoader.loadClass(
>> ClassLoader.java:522)
>> at java.base/java.lang.Class.forName0(Native Method)
>> at java.base/java.lang.Class.forName(Class.java:398)
>> at org.apache.ignite.internal.util.IgniteUtils.forName(
>> IgniteUtils.java:8828)
>> at org.apache.ignite.internal.MarshallerContextImpl.getClass(
>> MarshallerContextImpl.java:374)
>> at org.apache.ignite.internal.binary.BinaryContext.
>> descriptorForTypeId(BinaryContext.java:715)
>> ... 21 more
>>
>>
>>
>> On 7/1/20 7:09 PM, Denis Magda wrote:
>> > Hi Rafael,
>> >
>> > Lamdas and anonymous classes capture and serialize their
>> Inner/OuterClass
>> > with all the fields, and classes of the fields of those fields, etc. In
>> > your case, ScanQueryClass was captured during the lamda serialization
>> and
>> > that class was not found during deserialization on the other side:
>> >
>> > Caused by: class org.apache.ignite.binary.BinaryObjectException:
>> Failed to
>> > read field [name=capturingClass]
>> > at
>> > org.apache.ignite.internal.binary.BinaryFieldAccessor.
>> read(BinaryFieldAccessor.java:192)
>> > at
>> > org.apache.ignite.internal.binary.BinaryClassDescriptor.
>> read(BinaryClassDescriptor.java:874)
>> > ... 20 more
>> > Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>> > ScanQueryTest
>> > at
>> > org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(
>> BinaryContext.java:724)
>> > at
>> > org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(
>> BinaryContext.java:721)
>> > at
>> > org.apache.ignite.internal.binary.BinaryUtils.
>> doReadClass(BinaryUtils.java:1638)
>> > at
>> > org.apache.ignite.internal.binary.BinaryUtils.
>> doReadClass(BinaryUtils.java:1575)
>> > at
>> > org.apache.ignite.internal.binary.BinaryUtils.
>> doReadClass(BinaryUtils.java:1552)
>> > at
>> > org.apache.ignite.internal.binary.BinaryReaderExImpl.
>> readClass(BinaryReaderExImpl.java:383)
>> > at
>> > org.apache.ignite.internal.binary.BinaryFieldAccessor$
>> DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>> > at
>> > org.apache.ignite.internal.binary.BinaryFieldAccessor$
>> DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>> > at
>> > org.apache.ignite.internal.binary.BinaryFieldAccessor.
>> read(BinaryFieldAccessor.java:188)
>> > ... 21 more
>> > *Caused by: java.lang.ClassNotFoundException: ScanQueryTest*
>> > at
>> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(
>> BuiltinClassLoader.java:581)
>> > at
>> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.
>> loadClass(ClassLoaders.java:178)
>> > at java.base/java.lang.ClassLoader.loadClass(
>> ClassLoader.java:522)
>> > at java.base/java.lang.Class.forName0(Native Method)
>> > at java.base/java.lang.Class.forName(Class.java:398)
>> > at
>> > org.apache.ignite.internal.util.IgniteUtils.forName(
>> IgniteUtils.java:8828)
>> > at
>> > org.apache.ignite.internal.MarshallerContextImpl.getClass(
>> MarshallerContextImpl.java:374)
>> > at
>> > org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(
>> BinaryContext.java:715)
>> > ... 29 more
>> >
>> > My advice is to avoid using lambdas and anonymous classes due to this
>> > specificity. Instead, you can create "private static classes" and Ignite
>> > will be serializing an instance of that class only:
>> >
>> > private static class MyFilter implements IgniteClosure {
>> > ....
>> > }
>> >
>> > Lamdas are nice for demos and simple code samples where
>> Inner/OuterClasses
>> > are simple Java classes. While in real production environments, things
>> can
>> > get messy.
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Wed, Jul 1, 2020 at 9:36 AM Rafael Troilo <rafael.tro...@heigit.org>
>> > wrote:
>> >
>> >> Hey Guys,
>> >>
>> >> I have a strange problem with the Transform part of a ScanQuery that I
>> get
>> >> a deserialize object exception for the SerializedLambda.
>> >>> javax.cache.CacheException: class
>> >> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> >> [typeName=java.lang.invoke.SerializedLambda]
>> >>
>> >> Strange is that it works very well in other cases like:
>> >>
>> >> try (Ignite ignite = Ignition.start("ignite.xml")) {
>> >> IgniteCache<Integer,String> cache =
>> >> ignite.createCache("ScanTransformTest");
>> >> cache.put(1, "Hello");
>> >>
>> >> // 1 Just a broadcast with a IgniteClosure works!
>> >> compute.broadcast((a) -> 123L, "Hello").forEach(System.out::
>> println);
>> >>
>> >> // 2 Running the ScanQuery on the remote server. Works too!
>> >> compute.broadcast(() -> {
>> >> try(QueryCursor<Long> cursor = cache.query(new
>> >> ScanQuery<>().setLocal(true), (a) -> 234L)){
>> >> for( Long row : cursor) {
>> >> return row;
>> >> }
>> >> }
>> >> return 0;
>> >> }).forEach(System.out::println);
>> >>
>> >>
>> >> // 3 But running it from the client side it fails!
>> >> try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), (a) ->
>> >> 345L)){
>> >> for( Long row : cursor) {
>> >> System.out.println(row);
>> >> break;
>> >> }
>> >> }
>> >>
>> >> }
>> >>
>> >> Also changing the Lambda to a actually class implementing IgniteClosure
>> >> results in the same problem.
>> >>
>> >> Any idea what I'm doing wrong?
>> >>
>> >> We are running ignite ver. 2.8.0#20200226-sha1:341b01df
>> >> With java openjdk 11.0.7 2020-04-14
>> >>
>> >> Thank you!
>> >>
>> >> Best,
>> >> Rafael
>> >>
>> >>
>> >> Error Log:
>> >> Exception in thread "main" javax.cache.CacheException: class
>> >> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> >> [typeName=java.lang.invoke.SerializedLambda]
>> >> at
>> >> org.apache.ignite.internal.processors.cache.GridCacheUtils.
>> convertToCacheException(GridCacheUtils.java:1317)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.query.
>> GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.query.
>> GridCacheDistributedQueryManager$5.onHasNext(
>> GridCacheDistributedQueryManager.java:645)
>> >> at
>> >> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(
>> GridCloseableIteratorAdapter.java:53)
>> >> at
>> >> org.apache.ignite.internal.util.lang.GridIteratorAdapter.
>> hasNext(GridIteratorAdapter.java:45)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.
>> AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>> >> at ScanQueryTest.main(ScanQueryTest.java:49)
>> >> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>> >> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>> >> at
>> >> org.apache.ignite.internal.util.IgniteUtils.unmarshal(
>> IgniteUtils.java:10310)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.query.
>> GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.
>> GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.
>> GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.GridCacheIoManager.
>> handleMessage(GridCacheIoManager.java:392)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.GridCacheIoManager.
>> handleMessage(GridCacheIoManager.java:318)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.
>> GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>> >> at
>> >> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.
>> onMessage(GridCacheIoManager.java:308)
>> >> at
>> >> org.apache.ignite.internal.managers.communication.
>> GridIoManager.invokeListener(GridIoManager.java:1843)
>> >> at
>> >> org.apache.ignite.internal.managers.communication.GridIoManager.
>> processRegularMessage0(GridIoManager.java:1468)
>> >> at
>> >> org.apache.ignite.internal.managers.communication.
>> GridIoManager.access$5200(GridIoManager.java:229)
>> >> at
>> >> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(
>> GridIoManager.java:1365)
>> >> at
>> >> java.base/java.util.concurrent.ThreadPoolExecutor.
>> runWorker(ThreadPoolExecutor.java:1128)
>> >> at
>> >> java.base/java.util.concurrent.ThreadPoolExecutor$
>> Worker.run(ThreadPoolExecutor.java:628)
>> >> at java.base/java.lang.Thread.run(Thread.java:834)
>> >> Caused by: class org.apache.ignite.binary.BinaryObjectException:
>> Failed to
>> >> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryClassDescriptor.
>> read(BinaryClassDescriptor.java:913)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(
>> BinaryReaderExImpl.java:1764)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(
>> BinaryReaderExImpl.java:1716)
>> >> at
>> >> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(
>> GridBinaryMarshaller.java:307)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryMarshaller.
>> unmarshal0(BinaryMarshaller.java:101)
>> >> at
>> >> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshalle
>> r.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
>> >> at
>> >> org.apache.ignite.internal.util.IgniteUtils.unmarshal(
>> IgniteUtils.java:10304)
>> >> ... 14 more
>> >> Caused by: class org.apache.ignite.binary.BinaryObjectException:
>> Failed to
>> >> read field [name=capturingClass]
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryFieldAccessor.
>> read(BinaryFieldAccessor.java:192)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryClassDescriptor.
>> read(BinaryClassDescriptor.java:874)
>> >> ... 20 more
>> >> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>> >> ScanQueryTest
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(
>> BinaryContext.java:724)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(
>> BinaryContext.java:721)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryUtils.
>> doReadClass(BinaryUtils.java:1638)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryUtils.
>> doReadClass(BinaryUtils.java:1575)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryUtils.
>> doReadClass(BinaryUtils.java:1552)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryReaderExImpl.
>> readClass(BinaryReaderExImpl.java:383)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryFieldAccessor$
>> DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryFieldAccessor$
>> DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryFieldAccessor.
>> read(BinaryFieldAccessor.java:188)
>> >> ... 21 more
>> >> Caused by: java.lang.ClassNotFoundException: ScanQueryTest
>> >> at
>> >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(
>> BuiltinClassLoader.java:581)
>> >> at
>> >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.
>> loadClass(ClassLoaders.java:178)
>> >> at java.base/java.lang.ClassLoader.loadClass(
>> ClassLoader.java:522)
>> >> at java.base/java.lang.Class.forName0(Native Method)
>> >> at java.base/java.lang.Class.forName(Class.java:398)
>> >> at
>> >> org.apache.ignite.internal.util.IgniteUtils.forName(
>> IgniteUtils.java:8828)
>> >> at
>> >> org.apache.ignite.internal.MarshallerContextImpl.getClass(
>> MarshallerContextImpl.java:374)
>> >> at
>> >> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(
>> BinaryContext.java:715)
>> >> ... 29 more
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Rafael Troilo
>> >> HeiGIT gGmbH
>> >> Heidelberg Institute for Geoinformation Technology at Heidelberg
>> University
>> >>
>> >> https://heigit.org | rafael.tro...@heigit.org | phone +49-6221-533 484
>> >>
>> >> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg |
>> Germany
>> <https://www.google.com/maps/search/Schloss-Wolfsbrunnenweg+33+%7C+69118+Heidelberg+%7C+Germany?entry=gmail&source=g>
>> >> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
>> <https://www.google.com/maps/search/Berliner+Str.+45+%7C+69120+Heidelberg+%7C+Germany?entry=gmail&source=g>
>> >>
>> >> Amtsgericht Mannheim | HRB 733765
>> >> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>> >>
>> >
>>
>> --
>> Rafael Troilo
>> HeiGIT gGmbH
>> Heidelberg Institute for Geoinformation Technology at Heidelberg
>> University
>>
>> https://heigit.org | rafael.tro...@heigit.org | phone +49-6221-533 484
>>
>> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
>> <https://www.google.com/maps/search/Schloss-Wolfsbrunnenweg+33+%7C+69118+Heidelberg+%7C+Germany?entry=gmail&source=g>
>> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
>> <https://www.google.com/maps/search/Berliner+Str.+45+%7C+69120+Heidelberg+%7C+Germany?entry=gmail&source=g>
>>
>> Amtsgericht Mannheim | HRB 733765
>> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>>
>
--
-
Denis
