My linux machine is Centos 6.7 with components: - HDFS 2.7.1.2.3 - MapReduce2 2.7.1.2.3 - YARN 2.7.1.2.3 - Tez 0.7.0.2.3 - HBase 1.1.1.2.3 - ZooKeeper 3.4.6.2.3 - Storm 0.10.0 - Kafka 0.8.2.2.3
I've just noticed that I don't have hive installed. Is hive necessary? Also I've found that storm relays on logback, not on log4j (also I did not changed any logging settings anywhere). The only log4j file that i've found is one from Metron-DataServices (also didn't change it). wt., 5.04.2016 o 16:52 użytkownik Casey Stella <[email protected]> napisał: > This would actually be the storm log4j properties. We don't, to my > knowledge, ship a log4j properties with the jar. Can you tell a bit about > your setup? (distribution or barring that apache versions of storm)? > > On Tue, Apr 5, 2016 at 10:48 AM, Kuba Sienkiewicz <[email protected]> > wrote: > >> Hi, >> I know metron is still very young so you don't need to be sorry :) >> Dev mailing list does not work for me :/ I get hella lot of emails but no >> responses from you. >> Regarding to your answer I didn't changed anything in metron source and >> as far as I know, without any special options, log4j.xml is located in jar >> (so is taken from source). >> >> Best Regards >> >> >> wt., 5.04.2016 o 15:49 użytkownik Casey Stella <[email protected]> >> napisał: >> >>> + dev mailing list. >>> >>> On Tue, Apr 5, 2016 at 7:52 AM, Casey Stella <[email protected]> wrote: >>> >>>> Hi Jakub, >>>> >>>> Sorry you are having problems. Regarding the stack trace, what does >>>> your logging properties file look like? Specifically, are you in a >>>> situation where you might have duplicate entries in it like in >>>> http://stackoverflow.com/questions/8243137/log4jerror-attempted-to-append-to-closed-appender-named >>>> >>>> Casey >>>> >>>> On Tue, Apr 5, 2016 at 05:02 Kuba Sienkiewicz <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> it indeed not made to me. When I'll have a free weekend I'm going to >>>>> commit fixes for ansible playbooks. I'm sorry to say that but they are >>>>> full >>>>> of bugs (even spelling bugs like ddl instead of dll). >>>>> But bug which bothers me the most is the bug with storm architecture. >>>>> It's problem with enrichment bolt. I'm including stack trace. >>>>> >>>>> enrichmentJoinBolt: >>>>> >>>>> >>>>> 2016-04-05 04:57:49.125 STDERR [INFO] 2016-04-05 04:57:49,118 ERROR >>>>> Unable to write to stream >>>>> /var/log/storm/enrichment-4-1459322961-worker-6701.log for appender A1 >>>>> 2016-04-05 04:57:49.169 STDERR [INFO] 2016-04-05 04:57:49,134 ERROR An >>>>> exception occurred processing Appender A1 >>>>> org.apache.logging.log4j.core.appender.AppenderLoggingException: Error >>>>> flushing stream /var/log/storm/enrichment-4-1459322961-worker-6701.log >>>>> 2016-04-05 04:57:49.170 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.appender.OutputStreamManager.flush(OutputStreamManager.java:159) >>>>> 2016-04-05 04:57:49.171 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:112) >>>>> 2016-04-05 04:57:49.171 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:88) >>>>> 2016-04-05 04:57:49.172 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:99) >>>>> 2016-04-05 04:57:49.173 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:430) >>>>> 2016-04-05 04:57:49.174 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:409) >>>>> 2016-04-05 04:57:49.175 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:367) >>>>> 2016-04-05 04:57:49.185 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.Logger.logMessage(Logger.java:112) >>>>> 2016-04-05 04:57:49.185 STDERR [INFO] at >>>>> org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:727) >>>>> 2016-04-05 04:57:49.203 STDERR [INFO] at >>>>> org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:716) >>>>> 2016-04-05 04:57:49.205 STDERR [INFO] at >>>>> org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:198) >>>>> 2016-04-05 04:57:49.216 STDERR [INFO] at >>>>> clojure.tools.logging$eval1$fn__7.invoke(NO_SOURCE_FILE:0) >>>>> 2016-04-05 04:57:49.226 STDERR [INFO] at >>>>> clojure.tools.logging.impl$fn__28$G__8__39.invoke(impl.clj:16) >>>>> 2016-04-05 04:57:49.227 STDERR [INFO] at >>>>> clojure.tools.logging$log_STAR_.invoke(logging.clj:59) >>>>> 2016-04-05 04:57:49.239 STDERR [INFO] at >>>>> backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:336) >>>>> 2016-04-05 04:57:49.247 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$fn__5828$exec_fn__1317__auto__$reify__5830$iter__5835__5839$fn__5840.invoke(worker.clj:541) >>>>> 2016-04-05 04:57:49.252 STDERR [INFO] at >>>>> clojure.lang.LazySeq.sval(LazySeq.java:40) >>>>> 2016-04-05 04:57:49.260 STDERR [INFO] at >>>>> clojure.lang.LazySeq.seq(LazySeq.java:49) >>>>> 2016-04-05 04:57:49.266 STDERR [INFO] at >>>>> clojure.lang.Cons.next(Cons.java:39) >>>>> 2016-04-05 04:57:49.271 STDERR [INFO] at >>>>> clojure.lang.RT.next(RT.java:598) >>>>> 2016-04-05 04:57:49.430 STDERR [INFO] at >>>>> clojure.core$next.invoke(core.clj:64) >>>>> 2016-04-05 04:57:49.436 STDERR [INFO] at >>>>> clojure.core$dorun.invoke(core.clj:2856) >>>>> 2016-04-05 04:57:49.437 STDERR [INFO] at >>>>> clojure.core$doall.invoke(core.clj:2871) >>>>> 2016-04-05 04:57:49.448 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$fn__5828$exec_fn__1317__auto__$reify__5830.run(worker.clj:541) >>>>> 2016-04-05 04:57:49.450 STDERR [INFO] at >>>>> java.security.AccessController.doPrivileged(Native Method) >>>>> 2016-04-05 04:57:49.461 STDERR [INFO] at >>>>> javax.security.auth.Subject.doAs(Subject.java:422) >>>>> 2016-04-05 04:57:49.461 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$fn__5828$exec_fn__1317__auto____5829.invoke(worker.clj:515) >>>>> 2016-04-05 04:57:49.462 STDERR [INFO] at >>>>> clojure.lang.AFn.applyToHelper(AFn.java:178) >>>>> 2016-04-05 04:57:49.469 STDERR [INFO] at >>>>> clojure.lang.AFn.applyTo(AFn.java:144) >>>>> 2016-04-05 04:57:49.469 STDERR [INFO] at >>>>> clojure.core$apply.invoke(core.clj:624) >>>>> 2016-04-05 04:57:49.469 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$fn__5828$mk_worker__5914.doInvoke(worker.clj:491) >>>>> 2016-04-05 04:57:49.480 STDERR [INFO] at >>>>> clojure.lang.RestFn.invoke(RestFn.java:512) >>>>> 2016-04-05 04:57:49.480 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$_main.invoke(worker.clj:641) >>>>> 2016-04-05 04:57:49.480 STDERR [INFO] at >>>>> clojure.lang.AFn.applyToHelper(AFn.java:165) >>>>> 2016-04-05 04:57:49.494 STDERR [INFO] at >>>>> clojure.lang.AFn.applyTo(AFn.java:144) >>>>> 2016-04-05 04:57:49.495 STDERR [INFO] at >>>>> backtype.storm.daemon.worker.main(Unknown Source) >>>>> 2016-04-05 04:57:49.509 STDERR [INFO] Caused by: java.io.IOException: >>>>> Stream Closed >>>>> 2016-04-05 04:57:49.510 STDERR [INFO] at >>>>> java.io.FileOutputStream.writeBytes(Native Method) >>>>> 2016-04-05 04:57:49.523 STDERR [INFO] at >>>>> java.io.FileOutputStream.write(FileOutputStream.java:326) >>>>> 2016-04-05 04:57:49.534 STDERR [INFO] at >>>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) >>>>> 2016-04-05 04:57:49.535 STDERR [INFO] at >>>>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) >>>>> 2016-04-05 04:57:49.553 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.appender.OutputStreamManager.flush(OutputStreamManager.java:156) >>>>> 2016-04-05 04:57:49.553 STDERR [INFO] ... 35 more >>>>> 2016-04-05 04:57:49.571 STDERR [INFO] >>>>> 2016-04-05 04:57:49.572 STDERR [INFO] 2016-04-05 04:57:49,212 ERROR >>>>> Attempted to append to non-started appender syslog >>>>> >>>>> enrichmentSplitBolt >>>>> <http://192.168.66.2:8744/component.html?id=enrichmentSplitBolt&topology_id=enrichment-4-1459322961>: >>>>> >>>>> 2016-04-05 04:57:49.125 STDERR [INFO] 2016-04-05 04:57:49,118 ERROR >>>>> Unable to write to stream >>>>> /var/log/storm/enrichment-4-1459322961-worker-6701.log for appender A1 >>>>> 2016-04-05 04:57:49.169 STDERR [INFO] 2016-04-05 04:57:49,134 ERROR An >>>>> exception occurred processing Appender A1 >>>>> org.apache.logging.log4j.core.appender.AppenderLoggingException: Error >>>>> flushing stream /var/log/storm/enrichment-4-1459322961-worker-6701.log >>>>> 2016-04-05 04:57:49.170 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.appender.OutputStreamManager.flush(OutputStreamManager.java:159) >>>>> 2016-04-05 04:57:49.171 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:112) >>>>> 2016-04-05 04:57:49.171 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:88) >>>>> 2016-04-05 04:57:49.172 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:99) >>>>> 2016-04-05 04:57:49.173 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:430) >>>>> 2016-04-05 04:57:49.174 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:409) >>>>> 2016-04-05 04:57:49.175 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:367) >>>>> 2016-04-05 04:57:49.185 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.Logger.logMessage(Logger.java:112) >>>>> 2016-04-05 04:57:49.185 STDERR [INFO] at >>>>> org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:727) >>>>> 2016-04-05 04:57:49.203 STDERR [INFO] at >>>>> org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:716) >>>>> 2016-04-05 04:57:49.205 STDERR [INFO] at >>>>> org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:198) >>>>> 2016-04-05 04:57:49.216 STDERR [INFO] at >>>>> clojure.tools.logging$eval1$fn__7.invoke(NO_SOURCE_FILE:0) >>>>> 2016-04-05 04:57:49.226 STDERR [INFO] at >>>>> clojure.tools.logging.impl$fn__28$G__8__39.invoke(impl.clj:16) >>>>> 2016-04-05 04:57:49.227 STDERR [INFO] at >>>>> clojure.tools.logging$log_STAR_.invoke(logging.clj:59) >>>>> 2016-04-05 04:57:49.239 STDERR [INFO] at >>>>> backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:336) >>>>> 2016-04-05 04:57:49.247 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$fn__5828$exec_fn__1317__auto__$reify__5830$iter__5835__5839$fn__5840.invoke(worker.clj:541) >>>>> 2016-04-05 04:57:49.252 STDERR [INFO] at >>>>> clojure.lang.LazySeq.sval(LazySeq.java:40) >>>>> 2016-04-05 04:57:49.260 STDERR [INFO] at >>>>> clojure.lang.LazySeq.seq(LazySeq.java:49) >>>>> 2016-04-05 04:57:49.266 STDERR [INFO] at >>>>> clojure.lang.Cons.next(Cons.java:39) >>>>> 2016-04-05 04:57:49.271 STDERR [INFO] at >>>>> clojure.lang.RT.next(RT.java:598) >>>>> 2016-04-05 04:57:49.430 STDERR [INFO] at >>>>> clojure.core$next.invoke(core.clj:64) >>>>> 2016-04-05 04:57:49.436 STDERR [INFO] at >>>>> clojure.core$dorun.invoke(core.clj:2856) >>>>> 2016-04-05 04:57:49.437 STDERR [INFO] at >>>>> clojure.core$doall.invoke(core.clj:2871) >>>>> 2016-04-05 04:57:49.448 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$fn__5828$exec_fn__1317__auto__$reify__5830.run(worker.clj:541) >>>>> 2016-04-05 04:57:49.450 STDERR [INFO] at >>>>> java.security.AccessController.doPrivileged(Native Method) >>>>> 2016-04-05 04:57:49.461 STDERR [INFO] at >>>>> javax.security.auth.Subject.doAs(Subject.java:422) >>>>> 2016-04-05 04:57:49.461 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$fn__5828$exec_fn__1317__auto____5829.invoke(worker.clj:515) >>>>> 2016-04-05 04:57:49.462 STDERR [INFO] at >>>>> clojure.lang.AFn.applyToHelper(AFn.java:178) >>>>> 2016-04-05 04:57:49.469 STDERR [INFO] at >>>>> clojure.lang.AFn.applyTo(AFn.java:144) >>>>> 2016-04-05 04:57:49.469 STDERR [INFO] at >>>>> clojure.core$apply.invoke(core.clj:624) >>>>> 2016-04-05 04:57:49.469 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$fn__5828$mk_worker__5914.doInvoke(worker.clj:491) >>>>> 2016-04-05 04:57:49.480 STDERR [INFO] at >>>>> clojure.lang.RestFn.invoke(RestFn.java:512) >>>>> 2016-04-05 04:57:49.480 STDERR [INFO] at >>>>> backtype.storm.daemon.worker$_main.invoke(worker.clj:641) >>>>> 2016-04-05 04:57:49.480 STDERR [INFO] at >>>>> clojure.lang.AFn.applyToHelper(AFn.java:165) >>>>> 2016-04-05 04:57:49.494 STDERR [INFO] at >>>>> clojure.lang.AFn.applyTo(AFn.java:144) >>>>> 2016-04-05 04:57:49.495 STDERR [INFO] at >>>>> backtype.storm.daemon.worker.main(Unknown Source) >>>>> 2016-04-05 04:57:49.509 STDERR [INFO] Caused by: java.io.IOException: >>>>> Stream Closed >>>>> 2016-04-05 04:57:49.510 STDERR [INFO] at >>>>> java.io.FileOutputStream.writeBytes(Native Method) >>>>> 2016-04-05 04:57:49.523 STDERR [INFO] at >>>>> java.io.FileOutputStream.write(FileOutputStream.java:326) >>>>> 2016-04-05 04:57:49.534 STDERR [INFO] at >>>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) >>>>> 2016-04-05 04:57:49.535 STDERR [INFO] at >>>>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) >>>>> 2016-04-05 04:57:49.553 STDERR [INFO] at >>>>> org.apache.logging.log4j.core.appender.OutputStreamManager.flush(OutputStreamManager.java:156) >>>>> 2016-04-05 04:57:49.553 STDERR [INFO] ... 35 more >>>>> 2016-04-05 04:57:49.571 STDERR [INFO] >>>>> 2016-04-05 04:57:49.572 STDERR [INFO] 2016-04-05 04:57:49,212 ERROR >>>>> Attempted to append to non-started appender syslog >>>>> 2016-04-05 04:58:31.273 STDERR [INFO] JMXetricAgent instrumented JVM, see >>>>> https://github.com/ganglia/jmxetric >>>>> 2016-04-05 <https://github.com/ganglia/jmxetric%0D2016-04-05> >>>>> 04:58:31.541 STDERR [INFO] Apr 05, 2016 4:58:30 AM >>>>> info.ganglia.gmetric4j.GMonitor start >>>>> 2016-04-05 04:58:31.544 STDERR [INFO] INFO: Setting up 1 samplers >>>>> >>>>> >>> >
