Hi Dima, You probably want to increase the -Xmx setting in "worker.childopts", which is available in ambari under $Server:8080/#/main/services/STORM/configs.
Jon On Tue, Nov 8, 2016 at 2:47 PM DimaKovalyov <[email protected]> wrote: > Github user DimaKovalyov commented on the issue: > > https://github.com/apache/incubator-metron/pull/318 > > Thank you James, > > > Once you have data in your kafka queue this should go away. > That is true! Once I create a topic and stream data through it the > error is gone. > > My data is now going to enrichment and both bolts and spouts (all of > them) are having this weird error: > `java.lang.OutOfMemoryError: unable to create new native thread at > java.lang.Thread.start0(Native Method) at > java.lang.Thread.start(Thread.java:714) at > org.apache.zookeeper.ClientCnxn.start(ClientCnxn.java:417) at > org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:450) at > ... > java.lang.Thread.run(Thread.java:745)` > ` > > And supervisor crashes also after 5-10 minutes with: > ``` > 2016-11-08 14:25:56.125 o.a.s.event [ERROR] Error when processing event > java.lang.OutOfMemoryError: unable to create new native thread > at java.lang.Thread.start0(Native Method) > at java.lang.Thread.start(Thread.java:714) > at > java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) > at > java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368) > at java.lang.UNIXProcess.initStreams(UNIXProcess.java:289) > at java.lang.UNIXProcess.lambda$new$2(UNIXProcess.java:259) > at java.security.AccessController.doPrivileged(Native Method) > at java.lang.UNIXProcess.<init>(UNIXProcess.java:258) > at java.lang.ProcessImpl.start(ProcessImpl.java:134) > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) > at java.lang.Runtime.exec(Runtime.java:620) > at org.apache.storm.shade.org > .apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:58) > at org.apache.storm.shade.org > .apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:254) > at org.apache.storm.shade.org > .apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:319) > at org.apache.storm.shade.org > .apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:160) > at org.apache.storm.shade.org > .apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:147) > at > org.apache.storm.util$exec_command_BANG_.invoke(util.clj:402) > at > org.apache.storm.util$send_signal_to_process.invoke(util.clj:429) > at > org.apache.storm.util$kill_process_with_sig_term.invoke(util.clj:454) > at > org.apache.storm.daemon.supervisor$shutdown_worker.invoke(supervisor.clj:290) > at > org.apache.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:435) > at clojure.core$partial$fn__4527.invoke(core.clj:2492) > at > org.apache.storm.event$event_manager$fn__7248.invoke(event.clj:40) > at clojure.lang.AFn.run(AFn.java:22) > at java.lang.Thread.run(Thread.java:745) > > ``` > Even though I have more than 30 GB RAM available. Do I need to tune > Storm for better memory usage? > Please advise. > > - Dima > > > --- > If your project is set up for it, you can reply to this email and have your > reply appear on GitHub as well. If your project does not have this feature > enabled and wishes so, or if the feature is enabled but not working, please > contact infrastructure at [email protected] or file a JIRA ticket > with INFRA. > --- > -- Jon Sent from my mobile device
