Hello.
We use the Camel "master" component with a File lock cluster service. The
service is configured using a blueprint and stores its files on
"/tmp/MY-LOCKS/*". The service is transparently bound to all the Camel's
contexts.
Versions :
- Camel 3.22.1
- Karaf 4.4.4
- Java 17
Our Karaf runs inside a K8 s pods, with a single instance for now in this
configuration. The file lock folder is on the container's "/tmp", and not
on a shared volume (PVC).
We have more than 20 deployments working without any problem on several
Kubernetes clusters, but an error appears randomly on two namespaces of one
cluster. See the stack trace at the end of the message. We observe that all
the routes that depend on master are frozen (file endpoint for example are
not consuming existing files), but error messages only appear when some
change happens on a route : for example stopping or suspending a context.
>From the pod's container, a lslocks indeed doesn't show the expected locks
even if files matching the distinct routes are present in the configured
folder, as we can see on all the working instances.
We've checked the possible differences between this cluster and other
working clusters, but we can't identify significative ones (same container
image, same workers OS, and so on).
Has someone already experienced this kind of error ?
What could we check to identify when and why all the locks seem to be lost ?
Thanks in advance for your help.
Regards.
Stack trace :
DefaultShutdownStrategy | 143 - org.apache.camel.camel-base-engine
- 3.22.1 | | Error occurred while shutting down route: acq_appvte002-f027.
This exception will be ignored.
java.lang.RuntimeException: org.apache.camel.RuntimeCamelException:
java.nio.channels.ClosedChannelException
at
org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.lambda$new$1(AbstractCamelClusterService.java:261)
~[?:?]
at
org.apache.camel.util.ReferenceCount.release(ReferenceCount.java:71)
~[!/:3.22.1]
at
org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.release(AbstractCamelClusterService.java:281)
~[?:?]
at
org.apache.camel.support.cluster.AbstractCamelClusterService.lambda$releaseView$4(AbstractCamelClusterService.java:168)
~[?:?]
at
org.apache.camel.util.concurrent.LockHelper.doWithWriteLock(LockHelper.java:84)
~[!/:3.22.1]
at
org.apache.camel.support.cluster.AbstractCamelClusterService.releaseView(AbstractCamelClusterService.java:162)
~[?:?]
at
org.apache.camel.component.master.MasterConsumer.doStop(MasterConsumer.java:94)
~[?:?]
at
org.apache.camel.support.service.BaseService.stop(BaseService.java:160)
~[!/:3.22.1]
at
org.apache.camel.support.service.ServiceHelper.stopService(ServiceHelper.java:162)
~[!/:3.22.1]
at
org.apache.camel.impl.engine.DefaultShutdownStrategy.shutdownNow(DefaultShutdownStrategy.java:433)
[!/:3.22.1]
at
org.apache.camel.impl.engine.DefaultShutdownStrategy$ShutdownTask.run(DefaultShutdownStrategy.java:636)
[!/:3.22.1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[?:?]
at java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: org.apache.camel.RuntimeCamelException:
java.nio.channels.ClosedChannelException
at
org.apache.camel.RuntimeCamelException.wrapRuntimeException(RuntimeCamelException.java:66)
~[!/:3.22.1]
at
org.apache.camel.support.service.BaseService.doFail(BaseService.java:413)
~[!/:3.22.1]
at
org.apache.camel.support.service.BaseService.fail(BaseService.java:342)
~[!/:3.22.1]
at
org.apache.camel.support.service.BaseService.stop(BaseService.java:165)
~[!/:3.22.1]
at
org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.stopView(AbstractCamelClusterService.java:296)
~[?:?]
at
org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.lambda$new$1(AbstractCamelClusterService.java:259)
~[?:?]
... 15 more
Caused by: java.nio.channels.ClosedChannelException
at sun.nio.ch.FileLockImpl.release(FileLockImpl.java:58) ~[?:?]
at
org.apache.camel.component.file.cluster.FileLockClusterView.closeInternal(FileLockClusterView.java:110)
~[?:?]
at
org.apache.camel.component.file.cluster.FileLockClusterView.doStop(FileLockClusterView.java:97)
~[?:?]
at
org.apache.camel.support.service.BaseService.stop(BaseService.java:160)
~[!/:3.22.1]
at
org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.stopView(AbstractCamelClusterService.java:296)
~[?:?]
at
org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.lambda$new$1(AbstractCamelClusterService.java:259)
~[?:?]
... 15 more
