Yeah I know the reason behind the exception.
I don't see a reason for this exception to kill the entire
FlowCapableInventoryProvider thread, and by that causing serious deadlocks in
the system.
This is the thread that polls from the Notifications Queue, it is stuck because
the FlowCapableInventoryProvider is dead:
"pool-31-thread-1" prio=10 tid=0x00007fd9d4f02000 nid=0x17e7 waiting on
condition [0x00007fda80f70000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000072b8d69c8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown
Source)
at java.util.concurrent.LinkedBlockingDeque.putLast(Unknown Source)
at java.util.concurrent.LinkedBlockingDeque.put(Unknown Source)
at
org.opendaylight.openflowplugin.applications.inventory.manager.FlowCapableInventoryProvider.enqueue(FlowCapableInventoryProvider.java:60)
at
org.opendaylight.openflowplugin.applications.inventory.manager.NodeChangeCommiter.onNodeRemoved(NodeChangeCommiter.java:125)
- locked <0x000000072b8d6a40> (a
org.opendaylight.openflowplugin.applications.inventory.manager.NodeChangeCommiter)
at
org.opendaylight.yangtools.yang.binding.util.NotificationListenerInvoker.invokeNotification(NotificationListenerInvoker.java:91)
at
org.opendaylight.controller.md.sal.binding.impl.BindingDOMNotificationListenerAdapter.onNotification(BindingDOMNotificationListenerAdapter.java:44)
at
org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouterEvent.deliverNotification(DOMNotificationRouterEvent.java:50)
at
org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter$1.onEvent(DOMNotificationRouter.java:68)
at
org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter$1.onEvent(DOMNotificationRouter.java:65)
at
com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
So the Notifications Queue is full, and then we can't put new Notifications:
"nioEventLoopGroup-11-83" prio=10 tid=0x00007fd698061800 nid=0xb7bf runnable
[0x00007fd5a6997000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
at
com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136)
at
com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246)
at
org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter.putNotification(DOMNotificationRouter.java:200)
at
org.opendaylight.controller.md.sal.binding.impl.BindingDOMNotificationPublishServiceAdapter.putNotification(BindingDOMNotificationPublishServiceAdapter.java:51)
at
org.opendaylight.controller.md.sal.binding.compat.HeliumNotificationProviderServiceAdapter.publish(HeliumNotificationProviderServiceAdapter.java:33)
at
org.opendaylight.openflowplugin.openflow.md.core.sal.OFRpcTaskUtil$1FutureCallbackImpl.onSuccess(OFRpcTaskUtil.java:115)
at
org.opendaylight.openflowplugin.openflow.md.core.sal.OFRpcTaskUtil$1FutureCallbackImpl.onSuccess(OFRpcTaskUtil.java:101)
....
From: Robert Varga [mailto:[email protected]]
Sent: Tuesday, January 19, 2016 7:15 PM
To: Sela, Guy; controller-dev; [email protected]
Subject: Re: [controller-dev] FlowCapableInventoryProvider - Thread Dies
On 01/19/2016 06:02 PM, Sela, Guy wrote:
Hi,
I looked at the code of FlowCapableInventoryProvider.
The thread that polls from the queue (Named "FlowCapableInventoryProvider")
died at our system, because of an exception in one of the handlers.
Why isn't the run() loop protected from exceptions?
FlowCapableInventoryProvider
Nothing protects the run() from Exceptions.
@Override
public void run() {
try {
for (; ; ) {
InventoryOperation op = queue.take();
int ops = 0;
final ArrayList<InventoryOperation> opsToApply = new
ArrayList<>(MAX_BATCH);
do {
opsToApply.add(op);
ops++;
if (ops < MAX_BATCH) {
op = queue.poll();
} else {
op = null;
}
} while (op != null);
submitOperations(opsToApply);
}
} catch (final InterruptedException e) {
LOG.info("Processing interrupted, terminating", e);
}
// Drain all events, making sure any blocked threads are unblocked
while (!queue.isEmpty()) {
queue.poll();
}
}
The exception:
Exception in thread "FlowCapableInventoryProvider"
java.lang.IllegalArgumentException: Instance identifier references
(urn:opendaylight:inventory?revision=2013-08-19)node-connector[{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:4:3}]
but data identifier is
ImmutableMapEntryNode{nodeIdentifier=(urn:opendaylight:inventory?revision=2013-08-19)node-connector[{(urn:opendaylight:inventory?revision=2013-08-19)id=3}],
value=[ImmutableLeafNode{nodeIdentifier=(urn:opendaylight:inventory?revision=2013-08-19)id,
value=3, attributes={}}], attributes={}}
You have a mismatch between the instance identifier where the data is being
written to and the data itself. The two IDs (highlighted) have to match.
Bye,
Robert
_______________________________________________
openflowplugin-dev mailing list
[email protected]
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
