Github user DimaKovalyov commented on the issue:
https://github.com/apache/incubator-metron/pull/318
Thank you James,
> Once you have data in your kafka queue this should go away.
That is true! Once I create a topic and stream data through it the error is
gone.
My data is now going to enrichment and both bolts and spouts (all of them)
are having this weird error:
`java.lang.OutOfMemoryError: unable to create new native thread at
java.lang.Thread.start0(Native Method) at
java.lang.Thread.start(Thread.java:714) at
org.apache.zookeeper.ClientCnxn.start(ClientCnxn.java:417) at
org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:450) at
...
java.lang.Thread.run(Thread.java:745)`
`
And supervisor crashes also after 5-10 minutes with:
```
2016-11-08 14:25:56.125 o.a.s.event [ERROR] Error when processing event
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
at java.lang.UNIXProcess.initStreams(UNIXProcess.java:289)
at java.lang.UNIXProcess.lambda$new$2(UNIXProcess.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:258)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
at java.lang.Runtime.exec(Runtime.java:620)
at
org.apache.storm.shade.org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:58)
at
org.apache.storm.shade.org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:254)
at
org.apache.storm.shade.org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:319)
at
org.apache.storm.shade.org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:160)
at
org.apache.storm.shade.org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:147)
at org.apache.storm.util$exec_command_BANG_.invoke(util.clj:402)
at org.apache.storm.util$send_signal_to_process.invoke(util.clj:429)
at
org.apache.storm.util$kill_process_with_sig_term.invoke(util.clj:454)
at
org.apache.storm.daemon.supervisor$shutdown_worker.invoke(supervisor.clj:290)
at
org.apache.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:435)
at clojure.core$partial$fn__4527.invoke(core.clj:2492)
at
org.apache.storm.event$event_manager$fn__7248.invoke(event.clj:40)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:745)
```
Even though I have more than 30 GB RAM available. Do I need to tune Storm
for better memory usage?
Please advise.
- Dima
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
