Hello!
I believe the root cause is this part of log:
Caused by: java.nio.channels.ClosedByInterruptException
at
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:746)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727)
at
org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.read(RandomAccessFileIO.java:62)
at
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:322)
... 60 more
Here, a thread tht was using Persistence file channel got interrupted and
it became dysfunctional after that. Then there's a multitude of problems
caused by this.
This should not generally happen under normal load, however if you're
seeing such errors, consider setting the following Java argument for Ignite:
-DIGNITE_USE_ASYNC_FILE_IO_FACTORY=true
Please try to re-run with this setting, see if you still have any issues
Regards,
--
Ilya Kasnacheev
2018-01-29 11:06 GMT+03:00 Arseny Kovalchuk <[email protected]>:
> Hi guys.
>
> We've got 2.3 with persistence enabled. 5 server nodes, 3 client query
> nodes, 1 writer client node (it gets events from Kafka and puts them to
> Ignite with key-val API). All client and server nodes are Pods in
> Kubernetes.
>
> Cluster's worked well for a wile and then has stuck with the error which
> sounded as
>
> Suppressed: org.apache.ignite.IgniteCheckedException: Failed to update
> keys on primary node.
> Caused by: java.lang.ClassCastException: org.apache.ignite.internal.
> processors.cache.persistence.tree.io.DataPageIO cannot be cast to
> org.apache.ignite.internal.processors.cache.persistence.tree.io.BPlusIO
>
> At this stage the cluster doesn't accept data. Please take a look at
> attached logs.
>
> client-log.txt - log of the client, which tries to put data to cluster
> log-instance-N.txt - log of each server node. See ignite-instance-1.txt
> for the error on server node.
>
> Any known issues about that?
>
> Sincerely,
>
>
> Arseny Kovalchuk
>
> Senior Software Engineer at Synesis
> skype: arseny.kovalchuk
> mobile: +375 (29) 666-16-16
> LinkedIn Profile <http://www.linkedin.com/in/arsenykovalchuk/en>
>
