Hi Denis,
thank you for your quick response. Unfortunately even avoiding lambdas or
anonymous classes didn't solve this problem.
And was my first thought for this problem too.
With:
MyFilter.java
import org.apache.ignite.lang.IgniteClosure;
public class MyFilter<T> implements IgniteClosure<T, Long> {
private static final long serialVersionUID = 1L;
private final long l;
public MyFilter(long l) {
this.l = l;
}
@Override
public Long apply(T e) {
return l;
}
}
and
try (Ignite ignite = Ignition.start("ignite.xml")) {
IgniteCache<Integer, String> cache =
ignite.getOrCreateCache("ScanTransformTest");
...
// 2 Running the ScanQuery on the remote server. Works!
compute.broadcast(() -> {
try (QueryCursor<Long> cursor = cache.query(new
ScanQuery<>().setLocal(true), new MyFilter<>(234))) {
for (Long row : cursor) {
return row;
}
}
return 0;
})
// 3 Still does not work.
try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), new
MyFilter<>(345L))) {
for (Long row : cursor) {
System.out.println(row);
break;
}
}
}
Still does not work, this time complaining that the MyFilter class is not found.
PeerClassLoading is active as far as I can tell. At least in both server/client
ignite config
...
<property name="peerClassLoadingEnabled" value="true"/>
...
it is enabled.
I'm also not sure why the same class (IgniteClosure) works in the other case
but not in the Transformer or whats the differnt in case 2 to 3.
I guess making the MyFilter class available on the server will solve this
problem, but should not peerClassLoading work here?
Is there any information I can provide or test I should do to help to solve
this problem.
Thank you very much.
Stack Trace:
Exception in thread "main" javax.cache.CacheException: class
org.apache.ignite.IgniteCheckedException: MyFilter
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
at
org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
at
org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
at
org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
at
org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
at
org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
at ScanQueryTest.main(ScanQueryTest.java:57)
Caused by: class org.apache.ignite.IgniteCheckedException: MyFilter
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
at
org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: MyFilter
at
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
at
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
at
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
... 14 more
Caused by: java.lang.ClassNotFoundException: MyFilter
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at
org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
at
org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
at
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
... 21 more
On 7/1/20 7:09 PM, Denis Magda wrote:
> Hi Rafael,
>
> Lamdas and anonymous classes capture and serialize their Inner/OuterClass
> with all the fields, and classes of the fields of those fields, etc. In
> your case, ScanQueryClass was captured during the lamda serialization and
> that class was not found during deserialization on the other side:
>
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> read field [name=capturingClass]
> at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
> at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
> ... 20 more
> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
> ScanQueryTest
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
> at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
> at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
> at
> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
> ... 21 more
> *Caused by: java.lang.ClassNotFoundException: ScanQueryTest*
> at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
> at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
> at java.base/java.lang.Class.forName0(Native Method)
> at java.base/java.lang.Class.forName(Class.java:398)
> at
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
> at
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
> at
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
> ... 29 more
>
> My advice is to avoid using lambdas and anonymous classes due to this
> specificity. Instead, you can create "private static classes" and Ignite
> will be serializing an instance of that class only:
>
> private static class MyFilter implements IgniteClosure {
> ....
> }
>
> Lamdas are nice for demos and simple code samples where Inner/OuterClasses
> are simple Java classes. While in real production environments, things can
> get messy.
>
> -
> Denis
>
>
> On Wed, Jul 1, 2020 at 9:36 AM Rafael Troilo <rafael.tro...@heigit.org>
> wrote:
>
>> Hey Guys,
>>
>> I have a strange problem with the Transform part of a ScanQuery that I get
>> a deserialize object exception for the SerializedLambda.
>>> javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>>
>> Strange is that it works very well in other cases like:
>>
>> try (Ignite ignite = Ignition.start("ignite.xml")) {
>> IgniteCache<Integer,String> cache =
>> ignite.createCache("ScanTransformTest");
>> cache.put(1, "Hello");
>>
>> // 1 Just a broadcast with a IgniteClosure works!
>> compute.broadcast((a) -> 123L, "Hello").forEach(System.out::println);
>>
>> // 2 Running the ScanQuery on the remote server. Works too!
>> compute.broadcast(() -> {
>> try(QueryCursor<Long> cursor = cache.query(new
>> ScanQuery<>().setLocal(true), (a) -> 234L)){
>> for( Long row : cursor) {
>> return row;
>> }
>> }
>> return 0;
>> }).forEach(System.out::println);
>>
>>
>> // 3 But running it from the client side it fails!
>> try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), (a) ->
>> 345L)){
>> for( Long row : cursor) {
>> System.out.println(row);
>> break;
>> }
>> }
>>
>> }
>>
>> Also changing the Lambda to a actually class implementing IgniteClosure
>> results in the same problem.
>>
>> Any idea what I'm doing wrong?
>>
>> We are running ignite ver. 2.8.0#20200226-sha1:341b01df
>> With java openjdk 11.0.7 2020-04-14
>>
>> Thank you!
>>
>> Best,
>> Rafael
>>
>>
>> Error Log:
>> Exception in thread "main" javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>> [typeName=java.lang.invoke.SerializedLambda]
>> at
>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
>> at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>> at
>> org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
>> at
>> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
>> at
>> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
>> at
>> org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>> at ScanQueryTest.main(ScanQueryTest.java:49)
>> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>> at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
>> at
>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
>> at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
>> at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
>> at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
>> at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
>> at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>> at
>> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
>> at
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>> at
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>> at java.base/java.lang.Thread.run(Thread.java:834)
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>> at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
>> at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>> at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> at
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
>> at
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
>> at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
>> at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
>> ... 14 more
>> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
>> read field [name=capturingClass]
>> at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>> at
>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>> ... 20 more
>> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>> ScanQueryTest
>> at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>> at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>> at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>> at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>> at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>> at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>> at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>> at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>> at
>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>> ... 21 more
>> Caused by: java.lang.ClassNotFoundException: ScanQueryTest
>> at
>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>> at
>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>> at java.base/java.lang.Class.forName0(Native Method)
>> at java.base/java.lang.Class.forName(Class.java:398)
>> at
>> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>> at
>> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>> at
>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>> ... 29 more
>>
>>
>>
>>
>>
>> --
>> Rafael Troilo
>> HeiGIT gGmbH
>> Heidelberg Institute for Geoinformation Technology at Heidelberg University
>>
>> https://heigit.org | rafael.tro...@heigit.org | phone +49-6221-533 484
>>
>> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
>> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
>>
>> Amtsgericht Mannheim | HRB 733765
>> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>>
>
--
Rafael Troilo
HeiGIT gGmbH
Heidelberg Institute for Geoinformation Technology at Heidelberg University
https://heigit.org | rafael.tro...@heigit.org | phone +49-6221-533 484
Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
Amtsgericht Mannheim | HRB 733765
Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
