Kishor Patil created STORM-2873:
-----------------------------------
Summary: Backpressure implentation delete ephemeral too frequently
Key: STORM-2873
URL: https://issues.apache.org/jira/browse/STORM-2873
Project: Apache Storm
Issue Type: Bug
Reporter: Kishor Patil
Assignee: Kishor Patil
The backpressure implementation deletes the znode when not relevant but that
hits zookeeper issue of too frequent deletion and creation or same path for
ephemeral znode. Below is exception we get when zk hits this issue:
{code}
017-09-18 15:00:34.980 b.s.util WorkerBackpressureThread [WARN] Expecting
exception of class: class
org.apache.storm.shade.org.apache.zookeeper.KeeperException$NoNodeException,
but exception chain only contains: (#<NoAuthException
org.apache.storm.shade.org.apache.zookeeper.KeeperException$NoAuthException:
KeeperErrorCode = NoAuth for
/backpressure/TestTopology--170916-005358-117-1505523256/d11af335-6e03-48b6-801e-7369acfe9ffd-127.0.0.1-6721>)
2017-09-18 15:00:34.980 b.s.d.worker WorkerBackpressureThread [ERROR]
workerBackpressure update failed when connecting to ZK ... will retry
java.lang.RuntimeException:
org.apache.storm.shade.org.apache.zookeeper.KeeperException$NoAuthException:
KeeperErrorCode = NoAuth for
/backpressure/TestTopology--170916-005358-117-1505523256/d11af335-6e03-48b6-801e-7369acfe9ffd-127.0.0.1-6721
at backtype.storm.util$wrap_in_runtime.invoke(util.clj:52)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at backtype.storm.zookeeper$delete_node.doInvoke(zookeeper.clj:110)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at clojure.lang.RestFn.invoke(RestFn.java:464) ~[clojure-1.6.0.jar:?]
at backtype.storm.zookeeper$delete_recursive.invoke(zookeeper.clj:189)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at
backtype.storm.cluster_state.zookeeper_state_factory$_mkState$reify__4207.delete_node(zookeeper_state_factory.clj:117)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at sun.reflect.GeneratedMethodAccessor860.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
~[clojure-1.6.0.jar:?]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
~[clojure-1.6.0.jar:?]
at
org.apache.storm.pacemaker.pacemaker_state_factory$_mkState$reify__4254.delete_node(pacemaker_state_factory.clj:174)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at sun.reflect.GeneratedMethodAccessor859.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
~[clojure-1.6.0.jar:?]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
~[clojure-1.6.0.jar:?]
at
backtype.storm.cluster$mk_storm_cluster_state$reify__3873.worker_backpressure_BANG_(cluster.clj:421)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at sun.reflect.GeneratedMethodAccessor857.invoke(Unknown Source) ~[?:?]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
~[clojure-1.6.0.jar:?]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
~[clojure-1.6.0.jar:?]
at
backtype.storm.daemon.worker$mk_backpressure_handler$fn__7117.invoke(worker.clj:161)
[storm-core-0.10.2.y.jar:0.10.2.y]
at
backtype.storm.disruptor$worker_backpressure_handler$reify__6432.onEvent(disruptor.clj:57)
[storm-core-0.10.2.y.jar:0.10.2.y]
at
backtype.storm.utils.WorkerBackpressureThread.run(WorkerBackpressureThread.java:64)
[storm-core-0.10.2.y.jar:0.10.2.y]
Caused by:
org.apache.storm.shade.org.apache.zookeeper.KeeperException$NoAuthException:
KeeperErrorCode = NoAuth for
/backpressure/TestTopology--170916-005358-117-1505523256/d11af335-6e03-48b6-801e-7369acfe9ffd-127.0.0.1-6721
at
org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at
org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at
org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at
org.apache.storm.shade.org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:239)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at
org.apache.storm.shade.org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:234)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at
org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at
org.apache.storm.shade.org.apache.curator.framework.imps.DeleteBuilderImpl.pathInForeground(DeleteBuilderImpl.java:230)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at
org.apache.storm.shade.org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:215)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at
org.apache.storm.shade.org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:42)
~[storm-core-0.10.2.y.jar:0.10.2.y]
at backtype.storm.zookeeper$delete_node.doInvoke(zookeeper.clj:107)
~[storm-core-0.10.2.y.jar:0.10.2.y]
... 23 more
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)