Sure: https://bugs.opendaylight.org/show_bug.cgi?id=5020
From: Abhijit Kumbhare [mailto:[email protected]] Sent: Wednesday, January 20, 2016 2:10 AM To: Anil Vishnoi Cc: Sela, Guy; controller-dev; [email protected] Subject: Re: [controller-dev] FlowCapableInventoryProvider - Thread Dies Yes - please add a bug. On Wed, Jan 20, 2016 at 12:08 AM, Anil Vishnoi <[email protected]<mailto:[email protected]>> wrote: Hi Guy, Can you please open a bug here for the issue https://bugs.opendaylight.org/enter_bug.cgi?product=openflowplugin Thanks Anil On Tue, Jan 19, 2016 at 9:18 AM, Sela, Guy <[email protected]<mailto:[email protected]>> wrote: Yeah I know the reason behind the exception. I don’t see a reason for this exception to kill the entire FlowCapableInventoryProvider thread, and by that causing serious deadlocks in the system. This is the thread that polls from the Notifications Queue, it is stuck because the FlowCapableInventoryProvider is dead: "pool-31-thread-1" prio=10 tid=0x00007fd9d4f02000 nid=0x17e7 waiting on condition [0x00007fda80f70000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000072b8d69c8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.LinkedBlockingDeque.putLast(Unknown Source) at java.util.concurrent.LinkedBlockingDeque.put(Unknown Source) at org.opendaylight.openflowplugin.applications.inventory.manager.FlowCapableInventoryProvider.enqueue(FlowCapableInventoryProvider.java:60) at org.opendaylight.openflowplugin.applications.inventory.manager.NodeChangeCommiter.onNodeRemoved(NodeChangeCommiter.java:125) - locked <0x000000072b8d6a40> (a org.opendaylight.openflowplugin.applications.inventory.manager.NodeChangeCommiter) at org.opendaylight.yangtools.yang.binding.util.NotificationListenerInvoker.invokeNotification(NotificationListenerInvoker.java:91) at org.opendaylight.controller.md.sal.binding.impl.BindingDOMNotificationListenerAdapter.onNotification(BindingDOMNotificationListenerAdapter.java:44) at org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouterEvent.deliverNotification(DOMNotificationRouterEvent.java:50) at org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter$1.onEvent(DOMNotificationRouter.java:68) at org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter$1.onEvent(DOMNotificationRouter.java:65) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) So the Notifications Queue is full, and then we can’t put new Notifications: "nioEventLoopGroup-11-83" prio=10 tid=0x00007fd698061800 nid=0xb7bf runnable [0x00007fd5a6997000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105) at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246) at org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter.putNotification(DOMNotificationRouter.java:200) at org.opendaylight.controller.md.sal.binding.impl.BindingDOMNotificationPublishServiceAdapter.putNotification(BindingDOMNotificationPublishServiceAdapter.java:51) at org.opendaylight.controller.md.sal.binding.compat.HeliumNotificationProviderServiceAdapter.publish(HeliumNotificationProviderServiceAdapter.java:33) at org.opendaylight.openflowplugin.openflow.md.core.sal.OFRpcTaskUtil$1FutureCallbackImpl.onSuccess(OFRpcTaskUtil.java:115) at org.opendaylight.openflowplugin.openflow.md.core.sal.OFRpcTaskUtil$1FutureCallbackImpl.onSuccess(OFRpcTaskUtil.java:101) …. From: Robert Varga [mailto:[email protected]<mailto:[email protected]>] Sent: Tuesday, January 19, 2016 7:15 PM To: Sela, Guy; controller-dev; [email protected]<mailto:[email protected]> Subject: Re: [controller-dev] FlowCapableInventoryProvider - Thread Dies On 01/19/2016 06:02 PM, Sela, Guy wrote: Hi, I looked at the code of FlowCapableInventoryProvider. The thread that polls from the queue (Named “FlowCapableInventoryProvider”) died at our system, because of an exception in one of the handlers. Why isn’t the run() loop protected from exceptions? FlowCapableInventoryProvider Nothing protects the run() from Exceptions. @Override public void run() { try { for (; ; ) { InventoryOperation op = queue.take(); int ops = 0; final ArrayList<InventoryOperation> opsToApply = new ArrayList<>(MAX_BATCH); do { opsToApply.add(op); ops++; if (ops < MAX_BATCH) { op = queue.poll(); } else { op = null; } } while (op != null); submitOperations(opsToApply); } } catch (final InterruptedException e) { LOG.info("Processing interrupted, terminating", e); } // Drain all events, making sure any blocked threads are unblocked while (!queue.isEmpty()) { queue.poll(); } } The exception: Exception in thread "FlowCapableInventoryProvider" java.lang.IllegalArgumentException: Instance identifier references (urn:opendaylight:inventory?revision=2013-08-19)node-connector[{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:4:3}] but data identifier is ImmutableMapEntryNode{nodeIdentifier=(urn:opendaylight:inventory?revision=2013-08-19)node-connector[{(urn:opendaylight:inventory?revision=2013-08-19)id=3}], value=[ImmutableLeafNode{nodeIdentifier=(urn:opendaylight:inventory?revision=2013-08-19)id, value=3, attributes={}}], attributes={}} You have a mismatch between the instance identifier where the data is being written to and the data itself. The two IDs (highlighted) have to match. Bye, Robert _______________________________________________ controller-dev mailing list [email protected]<mailto:[email protected]> https://lists.opendaylight.org/mailman/listinfo/controller-dev -- Thanks Anil _______________________________________________ controller-dev mailing list [email protected]<mailto:[email protected]> https://lists.opendaylight.org/mailman/listinfo/controller-dev
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
