Hi Ethan,
Please find the configuration detail ********************************************** #Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. ########### These MUST be filled in for a storm configuration storm.zookeeper.servers: - "ZK1" - "ZK2" - "ZK3" nimbus.seeds: ["host1","host2"] ui.port : 8081 storm.log.dir: "/var/log/storm" storm.local.dir: "/var/log/storm/tmp" supervisor.slots.ports: - 6700 - 6701 - 6702 - 6703 - 6704 - 6705 - 6706 - 6707 - 6708 - 6709 - 6710 - 6711 - 6712 - 6713 - 6714 - 6715 - 6716 - 6717 worker.heap.memory.mb: 1639 topology.worker.max.heap.size.mb: 1639 worker.childopts: "-Xmx2048m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump" worker.gc.childopts: "" topology.min.replication.count: 2 # # # ##### These may optionally be filled in: # ## List of custom serializations # topology.kryo.register: # - org.mycompany.MyType # - org.mycompany.MyType2: org.mycompany.MyType2Serializer # ## List of custom kryo decorators # topology.kryo.decorators: # - org.mycompany.MyDecorator # ## Locations of the drpc servers # drpc.servers: # - "server1" # - "server2" ## Metrics Consumers # topology.metrics.consumer.register: # - class: "org.apache.storm.metric.LoggingMetricsConsumer" # parallelism.hint: 1 # - class: "org.mycompany.MyMetricsConsumer" # parallelism.hint: 1 # argument: # - endpoint: "metrics-collector.mycompany.org" ********************************************************************************************* On Thu, 30 Jan 2020 at 03:07, Ethan Li <[email protected]> wrote: > > I am not sure. Can you provide your configs? > > > > > On Jan 28, 2020, at 6:33 PM, Narasimhan Chengalvarayan > > <[email protected]> wrote: > > > > Hi Team, > > > > Do you have any idea, In storm apache 1.1.0 we have set worker size as > > 2 GB , Once we upgrade to 1.2.1 .It was coming as 65MB only. please > > help us .DO we need to follow different configuration setting for > > storm 1.2.1 or it is a bug. > > > > On Mon, 27 Jan 2020 at 16:44, Narasimhan Chengalvarayan > > <[email protected]> wrote: > >> > >> Hi Team, > >> > >> In storm 1.2.1 version, worker memory is showing as 65MB. But we have > >> set worker memory has 2GB. > >> > >> On Fri, 24 Jan 2020 at 01:25, Ethan Li <[email protected]> wrote: > >>> > >>> > >>> 1) What is stored in Workerbeats znode? > >>> > >>> > >>> Worker periodically sends heartbeat to zookeeper under workerbeats node. > >>> > >>> 2) Which settings control the frequency of workerbeats update > >>> > >>> > >>> > >>> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/jvm/org/apache/storm/Config.java#L1534-L1539 > >>> task.heartbeat.frequency.secs Default to 3 > >>> > >>> 3)What will be the impact if the frequency is reduced > >>> > >>> > >>> Nimbus get the worker status from workerbeat znode to know if executors > >>> on workers are alive or not. > >>> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/jvm/org/apache/storm/Config.java#L595-L601 > >>> If heartbeat exceeds nimbus.task.timeout.secs (default to 30), nimbus > >>> will think the certain executor is dead and try to reschedule. > >>> > >>> To reduce the issue on zookeeper, a pacemaker component was introduced. > >>> https://github.com/apache/storm/blob/master/docs/Pacemaker.md > >>> You might want to use it too. > >>> > >>> Thanks > >>> > >>> > >>> On Dec 10, 2019, at 4:36 PM, Surajeet Dev <[email protected]> > >>> wrote: > >>> > >>> We upgraded Storm version to 1.2.1 , and since then have been > >>> consistently observing Zookeeper session timeouts . > >>> > >>> On analysis , we observed that there is high frequency of updates on > >>> workerbeats znode with data upto size of 50KB. This causes the Garbage > >>> Collector to kickoff lasting more than 15 secs , resulting in Zookeper > >>> session timeout > >>> > >>> I understand , increasing the session timeout will alleviate the issue , > >>> but we have already done that twice > >>> > >>> My questions are: > >>> > >>> 1) What is stored in Workerbeats znode? > >>> 2) Which settings control the frequency of workerbeats update > >>> 3)What will be the impact if the frequency is reduced > >>> > >>> > >>> > >> > >> > >> -- > >> Thanks > >> C.Narasimhan > >> 09739123245 > > > > > > > > -- > > Thanks > > C.Narasimhan > > 09739123245 > -- Thanks C.Narasimhan 09739123245
