Hi Pat, It is all standard logback, described here: http://logback.qos.ch/
On Mon, Jun 6, 2016 at 12:18 PM, Pat Trainor <[email protected]> wrote: > Andrew, > Is any of this not standard log4j, and nifi-specific? > On Jun 6, 2016 2:34 AM, "Andrew Psaltis" <[email protected]> wrote: > >> You are correct, sorry about that Stephane! >> >> On Mon, Jun 6, 2016 at 8:23 AM, Stéphane Maarek < >> [email protected]> wrote: >> >>> Actually, it seems I only needed to change one line (versus two) >>> >>> <appender name="BOOTSTRAP_FILE" >>> class="ch.qos.logback.core.rolling.RollingFileAppender"> >>> <file>/mnt/xvdf/logs/nifi-bootstrap.log</file> >>> <rollingPolicy >>> class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> >>> <!-- >>> For daily rollover, use 'user_%d.log'. >>> For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'. >>> To GZIP rolled files, replace '.log' with '.log.gz'. >>> To ZIP rolled files, replace '.log' with '.log.zip'. >>> --> >>> >>> <fileNamePattern>./logs/nifi-bootstrap_%d.log</fileNamePattern> >>> >>> On Mon, Jun 6, 2016 at 4:20 PM Stéphane Maarek < >>> [email protected]> wrote: >>> >>>> Hi Andrew, >>>> >>>> After changing the file I'm getting the following error. I changed the >>>> lines you indicated. >>>> >>>> Failed to auto configure default logger context >>>> Reported exception: >>>> ch.qos.logback.core.joran.spi.JoranException: Problem parsing XML >>>> document. See previously reported errors. >>>> at >>>> ch.qos.logback.core.joran.event.SaxEventRecorder.recordEvents(SaxEventRecorder.java:67) >>>> at >>>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:134) >>>> at >>>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:99) >>>> at >>>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:49) >>>> at >>>> ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:77) >>>> at >>>> ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:152) >>>> at >>>> org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:85) >>>> at >>>> org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55) >>>> at org.slf4j.LoggerFactory.bind(LoggerFactory.java:141) >>>> at >>>> org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:120) >>>> at >>>> org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:331) >>>> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:283) >>>> at org.apache.nifi.bootstrap.RunNiFi.<init>(RunNiFi.java:117) >>>> at org.apache.nifi.bootstrap.RunNiFi.main(RunNiFi.java:199) >>>> Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: >>>> 1; Premature end of file. >>>> at >>>> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1239) >>>> at >>>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:642) >>>> at >>>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:326) >>>> at >>>> ch.qos.logback.core.joran.event.SaxEventRecorder.recordEvents(SaxEventRecorder.java:61) >>>> ... 13 more >>>> 06:16:11,055 |-INFO in ch.qos.logback.classic.LoggerContext[default] - >>>> Could NOT find resource [logback.groovy] >>>> 06:16:11,055 |-INFO in ch.qos.logback.classic.LoggerContext[default] - >>>> Could NOT find resource [logback-test.xml] >>>> 06:16:11,055 |-INFO in ch.qos.logback.classic.LoggerContext[default] - >>>> Found resource [logback.xml] at [file:/home/ec2-user/nifi/conf/logback.xml] >>>> 06:16:11,127 |-ERROR in >>>> ch.qos.logback.core.joran.event.SaxEventRecorder@682ea4b1 - >>>> XML_PARSING - Parsing fatal error on line 1 and column 1 >>>> org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature >>>> end of file. >>>> at org.xml.sax.SAXParseException: Premature end of file. >>>> at at >>>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198) >>>> at at >>>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177) >>>> at at >>>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400) >>>> at at >>>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327) >>>> at at >>>> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1438) >>>> at at >>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:1019) >>>> at at >>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) >>>> at at >>>> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:118) >>>> at at >>>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510) >>>> at at >>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848) >>>> at at >>>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777) >>>> at at >>>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) >>>> at at >>>> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) >>>> at at >>>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:642) >>>> at at >>>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:326) >>>> at at >>>> ch.qos.logback.core.joran.event.SaxEventRecorder.recordEvents(SaxEventRecorder.java:61) >>>> at at >>>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:134) >>>> at at >>>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:99) >>>> at at >>>> ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:49) >>>> at at >>>> ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:77) >>>> at at >>>> ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:152) >>>> at at >>>> org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:85) >>>> at at >>>> org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55) >>>> at at org.slf4j.LoggerFactory.bind(LoggerFactory.java:141) >>>> at at >>>> org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:120) >>>> at at >>>> org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:331) >>>> at at >>>> org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:283) >>>> at at >>>> org.apache.nifi.bootstrap.RunNiFi.<init>(RunNiFi.java:117) >>>> at at >>>> org.apache.nifi.bootstrap.RunNiFi.main(RunNiFi.java:199) >>>> >>>> >>>> On Mon, Jun 6, 2016 at 4:10 PM Andrew Psaltis <[email protected]> >>>> wrote: >>>> >>>>> Hi Stephane, >>>>> In the conf directory of your nifi install there is a logback.xml file >>>>> that needs to be edited. This configuration file controls the internal >>>>> logging done by nifi. For each of the appenders you will see an XML >>>>> elements that reference the "logs" directory. For example here is the >>>>> config for the app.log >>>>> >>>>> <appender name="APP_FILE" >>>>> class="ch.qos.logback.core.rolling.RollingFileAppender"> >>>>> * <file>logs/nifi-app.log</file> <--- This should be changed* >>>>> <rollingPolicy >>>>> class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> >>>>> <!-- >>>>> For daily rollover, use 'app_%d.log'. >>>>> For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'. >>>>> To GZIP rolled files, replace '.log' with '.log.gz'. >>>>> To ZIP rolled files, replace '.log' with '.log.zip'. >>>>> --> >>>>> * >>>>> <fileNamePattern>./logs/nifi-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern> >>>>> <--This should be changed* >>>>> <timeBasedFileNamingAndTriggeringPolicy >>>>> class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> >>>>> <maxFileSize>100MB</maxFileSize> >>>>> </timeBasedFileNamingAndTriggeringPolicy> >>>>> <!-- keep 30 log files worth of history --> >>>>> <maxHistory>30</maxHistory> >>>>> </rollingPolicy> >>>>> <encoder >>>>> class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> >>>>> <pattern>%date %level [%thread] %logger{40} >>>>> %msg%n</pattern> >>>>> <immediateFlush>true</immediateFlush> >>>>> </encoder> >>>>> </appender> >>>>> >>>>> >>>>> You would want to change the references above in to your attached >>>>> volume. I have marked them with bold above. In the logback config file you >>>>> will see two other appenders as well, these same changes should be done >>>>> there as well. >>>>> >>>>> Thanks, >>>>> Andrew >>>>> >>>>> >>>>> On Mon, Jun 6, 2016 at 6:56 AM, Stéphane Maarek < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm running NiFi on a t1.micro in AWS. I have attached a 100GB EBS >>>>>> volume at /mnt/xvdf and all the NiFi databases are there. I have a >>>>>> problem >>>>>> though. It seems that NiFi keeps an internal log and I can't change the >>>>>> position of it, so it's quickly clogging my t1.micro disk. See below. How >>>>>> can I improve / change that? And what's the "work" folder for? >>>>>> >>>>>> [ec2-user@ ~]$ du -s * >>>>>> 5271404 nifi >>>>>> [ec2-user@ ~]$ cd nifi >>>>>> [ec2-user@ nifi]$ du -s * >>>>>> 28 bin >>>>>> 756 conf >>>>>> 8412 docs >>>>>> 427468 lib >>>>>> 64 LICENSE >>>>>> 4331244 logs >>>>>> 44 NOTICE >>>>>> 8 README >>>>>> 144 state >>>>>> 503232 work >>>>>> >>>>>> >>>>>> Thanks! >>>>>> Stephane >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks, >>>>> Andrew >>>>> >>>>> Subscribe to my book: Streaming Data <http://manning.com/psaltis> >>>>> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> >>>>> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata> >>>>> >>>> >> >> >> -- >> Thanks, >> Andrew >> >> Subscribe to my book: Streaming Data <http://manning.com/psaltis> >> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> >> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata> >> > -- Thanks, Andrew Subscribe to my book: Streaming Data <http://manning.com/psaltis> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
