Hi Guy, Can you please open a bug here for the issue https://bugs.opendaylight.org/enter_bug.cgi?product=openflowplugin
Thanks Anil On Tue, Jan 19, 2016 at 9:18 AM, Sela, Guy <[email protected]> wrote: > Yeah I know the reason behind the exception. > > I don’t see a reason for this exception to kill the entire > FlowCapableInventoryProvider thread, and by that causing serious deadlocks > in the system. > > > > This is the thread that polls from the Notifications Queue, it is stuck > because the FlowCapableInventoryProvider is dead: > > "pool-31-thread-1" prio=10 tid=0x00007fd9d4f02000 nid=0x17e7 waiting on > condition [0x00007fda80f70000] > > java.lang.Thread.State: WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000072b8d69c8> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at java.util.concurrent.locks.LockSupport.park(Unknown Source) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown > Source) > > at java.util.concurrent.LinkedBlockingDeque.putLast(Unknown Source) > > at java.util.concurrent.LinkedBlockingDeque.put(Unknown Source) > > * at > org.opendaylight.openflowplugin.applications.inventory.manager.FlowCapableInventoryProvider.enqueue(FlowCapableInventoryProvider.java:60)* > > at > org.opendaylight.openflowplugin.applications.inventory.manager.NodeChangeCommiter.onNodeRemoved(NodeChangeCommiter.java:125) > > - locked <0x000000072b8d6a40> (a > org.opendaylight.openflowplugin.applications.inventory.manager.NodeChangeCommiter) > > at > org.opendaylight.yangtools.yang.binding.util.NotificationListenerInvoker.invokeNotification(NotificationListenerInvoker.java:91) > > at > org.opendaylight.controller.md.sal.binding.impl.BindingDOMNotificationListenerAdapter.onNotification(BindingDOMNotificationListenerAdapter.java:44) > > at > org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouterEvent.deliverNotification(DOMNotificationRouterEvent.java:50) > > at > org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter$1.onEvent(DOMNotificationRouter.java:68) > > at > org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter$1.onEvent(DOMNotificationRouter.java:65) > > at > com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) > > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > > at java.lang.Thread.run(Unknown Source) > > > > So the Notifications Queue is full, and then we can’t put new > Notifications: > > "nioEventLoopGroup-11-83" prio=10 tid=0x00007fd698061800 nid=0xb7bf > runnable [0x00007fd5a6997000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) > > at > com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136) > > at > com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105) > > * at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246)* > > * at > org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter.putNotification(DOMNotificationRouter.java:200)* > > at > org.opendaylight.controller.md.sal.binding.impl.BindingDOMNotificationPublishServiceAdapter.putNotification(BindingDOMNotificationPublishServiceAdapter.java:51) > > at > org.opendaylight.controller.md.sal.binding.compat.HeliumNotificationProviderServiceAdapter.publish(HeliumNotificationProviderServiceAdapter.java:33) > > at > org.opendaylight.openflowplugin.openflow.md.core.sal.OFRpcTaskUtil$1FutureCallbackImpl.onSuccess(OFRpcTaskUtil.java:115) > > at > org.opendaylight.openflowplugin.openflow.md.core.sal.OFRpcTaskUtil$1FutureCallbackImpl.onSuccess(OFRpcTaskUtil.java:101) > > …. > > > > *From:* Robert Varga [mailto:[email protected]] > *Sent:* Tuesday, January 19, 2016 7:15 PM > *To:* Sela, Guy; controller-dev; [email protected] > *Subject:* Re: [controller-dev] FlowCapableInventoryProvider - Thread Dies > > > > On 01/19/2016 06:02 PM, Sela, Guy wrote: > > Hi, > > I looked at the code of FlowCapableInventoryProvider. > > The thread that polls from the queue (Named > “FlowCapableInventoryProvider”) died at our system, because of an exception > in one of the handlers. > > Why isn’t the run() loop protected from exceptions? > > > > *FlowCapableInventoryProvider* > > > > Nothing protects the run() from Exceptions. > > > > @Override > > public void run() { > > try { > > for (; ; ) { > > InventoryOperation op = queue.take(); > > int ops = 0; > > final ArrayList<InventoryOperation> opsToApply = new > ArrayList<>(MAX_BATCH); > > do { > > opsToApply.add(op); > > ops++; > > if (ops < MAX_BATCH) { > > op = queue.poll(); > > } else { > > op = null; > > } > > } while (op != null); > > submitOperations(opsToApply); > > } > > } catch (final InterruptedException e) { > > LOG.info("Processing interrupted, terminating", e); > > } > > > > // Drain all events, making sure any blocked threads are unblocked > > while (!queue.isEmpty()) { > > queue.poll(); > > } > > } > > > > The exception: > > Exception in thread "FlowCapableInventoryProvider" > java.lang.IllegalArgumentException: Instance identifier references > (urn:opendaylight:inventory?revision=2013-08-19)node-connector[{(urn:opendaylight:inventory?revision=2013-08-19) > *id=openflow:4:3*}] but data identifier is > ImmutableMapEntryNode{nodeIdentifier=(urn:opendaylight:inventory?revision=2013-08-19)node-connector[{(urn:opendaylight:inventory?revision=2013-08-19) > *id=3*}], > value=[ImmutableLeafNode{nodeIdentifier=(urn:opendaylight:inventory?revision=2013-08-19)id, > value=3, attributes={}}], attributes={}} > > > You have a mismatch between the instance identifier where the data is > being written to and the data itself. The two IDs (highlighted) have to > match. > > Bye, > Robert > > _______________________________________________ > controller-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/controller-dev > > -- Thanks Anil
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
