Andrew, Is any of this not standard log4j, and nifi-specific? On Jun 6, 2016 2:34 AM, "Andrew Psaltis" <[email protected]> wrote:
> You are correct, sorry about that Stephane! > > On Mon, Jun 6, 2016 at 8:23 AM, Stéphane Maarek <[email protected] > > wrote: > >> Actually, it seems I only needed to change one line (versus two) >> >> <appender name="BOOTSTRAP_FILE" >> class="ch.qos.logback.core.rolling.RollingFileAppender"> >> <file>/mnt/xvdf/logs/nifi-bootstrap.log</file> >> <rollingPolicy >> class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> >> <!-- >> For daily rollover, use 'user_%d.log'. >> For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'. >> To GZIP rolled files, replace '.log' with '.log.gz'. >> To ZIP rolled files, replace '.log' with '.log.zip'. >> --> >> >> <fileNamePattern>./logs/nifi-bootstrap_%d.log</fileNamePattern> >> >> On Mon, Jun 6, 2016 at 4:20 PM Stéphane Maarek <[email protected]> >> wrote: >> >>> Hi Andrew, >>> >>> After changing the file I'm getting the following error. I changed the >>> lines you indicated. >>> >>> Failed to auto configure default logger context >>> Reported exception: >>> ch.qos.logback.core.joran.spi.JoranException: Problem parsing XML >>> document. See previously reported errors. >>> at >>> ch.qos.logback.core.joran.event.SaxEventRecorder.recordEvents(SaxEventRecorder.java:67) >>> at >>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:134) >>> at >>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:99) >>> at >>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:49) >>> at >>> ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:77) >>> at >>> ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:152) >>> at >>> org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:85) >>> at >>> org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55) >>> at org.slf4j.LoggerFactory.bind(LoggerFactory.java:141) >>> at >>> org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:120) >>> at >>> org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:331) >>> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:283) >>> at org.apache.nifi.bootstrap.RunNiFi.<init>(RunNiFi.java:117) >>> at org.apache.nifi.bootstrap.RunNiFi.main(RunNiFi.java:199) >>> Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: >>> 1; Premature end of file. >>> at >>> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1239) >>> at >>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:642) >>> at >>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:326) >>> at >>> ch.qos.logback.core.joran.event.SaxEventRecorder.recordEvents(SaxEventRecorder.java:61) >>> ... 13 more >>> 06:16:11,055 |-INFO in ch.qos.logback.classic.LoggerContext[default] - >>> Could NOT find resource [logback.groovy] >>> 06:16:11,055 |-INFO in ch.qos.logback.classic.LoggerContext[default] - >>> Could NOT find resource [logback-test.xml] >>> 06:16:11,055 |-INFO in ch.qos.logback.classic.LoggerContext[default] - >>> Found resource [logback.xml] at [file:/home/ec2-user/nifi/conf/logback.xml] >>> 06:16:11,127 |-ERROR in >>> ch.qos.logback.core.joran.event.SaxEventRecorder@682ea4b1 - XML_PARSING >>> - Parsing fatal error on line 1 and column 1 org.xml.sax.SAXParseException; >>> lineNumber: 1; columnNumber: 1; Premature end of file. >>> at org.xml.sax.SAXParseException: Premature end of file. >>> at at >>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198) >>> at at >>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177) >>> at at >>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400) >>> at at >>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327) >>> at at >>> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1438) >>> at at >>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:1019) >>> at at >>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) >>> at at >>> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:118) >>> at at >>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510) >>> at at >>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848) >>> at at >>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777) >>> at at >>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) >>> at at >>> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) >>> at at >>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:642) >>> at at >>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:326) >>> at at >>> ch.qos.logback.core.joran.event.SaxEventRecorder.recordEvents(SaxEventRecorder.java:61) >>> at at >>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:134) >>> at at >>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:99) >>> at at >>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:49) >>> at at >>> ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:77) >>> at at >>> ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:152) >>> at at >>> org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:85) >>> at at >>> org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55) >>> at at org.slf4j.LoggerFactory.bind(LoggerFactory.java:141) >>> at at >>> org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:120) >>> at at >>> org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:331) >>> at at >>> org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:283) >>> at at >>> org.apache.nifi.bootstrap.RunNiFi.<init>(RunNiFi.java:117) >>> at at >>> org.apache.nifi.bootstrap.RunNiFi.main(RunNiFi.java:199) >>> >>> >>> On Mon, Jun 6, 2016 at 4:10 PM Andrew Psaltis <[email protected]> >>> wrote: >>> >>>> Hi Stephane, >>>> In the conf directory of your nifi install there is a logback.xml file >>>> that needs to be edited. This configuration file controls the internal >>>> logging done by nifi. For each of the appenders you will see an XML >>>> elements that reference the "logs" directory. For example here is the >>>> config for the app.log >>>> >>>> <appender name="APP_FILE" >>>> class="ch.qos.logback.core.rolling.RollingFileAppender"> >>>> * <file>logs/nifi-app.log</file> <--- This should be changed* >>>> <rollingPolicy >>>> class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> >>>> <!-- >>>> For daily rollover, use 'app_%d.log'. >>>> For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'. >>>> To GZIP rolled files, replace '.log' with '.log.gz'. >>>> To ZIP rolled files, replace '.log' with '.log.zip'. >>>> --> >>>> * >>>> <fileNamePattern>./logs/nifi-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern> >>>> <--This should be changed* >>>> <timeBasedFileNamingAndTriggeringPolicy >>>> class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> >>>> <maxFileSize>100MB</maxFileSize> >>>> </timeBasedFileNamingAndTriggeringPolicy> >>>> <!-- keep 30 log files worth of history --> >>>> <maxHistory>30</maxHistory> >>>> </rollingPolicy> >>>> <encoder >>>> class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> >>>> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> >>>> <immediateFlush>true</immediateFlush> >>>> </encoder> >>>> </appender> >>>> >>>> >>>> You would want to change the references above in to your attached >>>> volume. I have marked them with bold above. In the logback config file you >>>> will see two other appenders as well, these same changes should be done >>>> there as well. >>>> >>>> Thanks, >>>> Andrew >>>> >>>> >>>> On Mon, Jun 6, 2016 at 6:56 AM, Stéphane Maarek < >>>> [email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'm running NiFi on a t1.micro in AWS. I have attached a 100GB EBS >>>>> volume at /mnt/xvdf and all the NiFi databases are there. I have a problem >>>>> though. It seems that NiFi keeps an internal log and I can't change the >>>>> position of it, so it's quickly clogging my t1.micro disk. See below. How >>>>> can I improve / change that? And what's the "work" folder for? >>>>> >>>>> [ec2-user@ ~]$ du -s * >>>>> 5271404 nifi >>>>> [ec2-user@ ~]$ cd nifi >>>>> [ec2-user@ nifi]$ du -s * >>>>> 28 bin >>>>> 756 conf >>>>> 8412 docs >>>>> 427468 lib >>>>> 64 LICENSE >>>>> 4331244 logs >>>>> 44 NOTICE >>>>> 8 README >>>>> 144 state >>>>> 503232 work >>>>> >>>>> >>>>> Thanks! >>>>> Stephane >>>>> >>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> Andrew >>>> >>>> Subscribe to my book: Streaming Data <http://manning.com/psaltis> >>>> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> >>>> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata> >>>> >>> > > > -- > Thanks, > Andrew > > Subscribe to my book: Streaming Data <http://manning.com/psaltis> > <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> > twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata> >
