Oracle DB is already a central part of our production setup. It stores ETL metadata, including etl task logs. I can't change it to some other storage.
Going to do TailFile and further app.log parsing to put it then in Oracle anyway thanks for suggestions Kirill On Mon, Mar 4, 2019 at 9:20 PM Mike Thomsen <[email protected]> wrote: > Kirill, > > This is a good place to get started with ELK: > > https://www.elastic.co/solutions/logging > > It uses ElasticSearch for storage. Check out some of their demos to get a > feel for what they offer. It's all open source and runs on very modest > hardware. It's also external to NiFi and something you can hook into every > part of your deployment rather than changing anything with NiFi. > > Mike > > On Mon, Mar 4, 2019 at 11:06 AM K Makeev <[email protected]> wrote: > >> My use case is that i need to put ExecuteSQL ERROS in database. >> >> Logback says that logging to database is their standard feature. >> >> Never used ELK. Could it accomplish such a task? >> Where should i start if i want to setup it with NIFI? >> >> Thanks >> Kirill >> >> >> On Mon, Mar 4, 2019 at 6:42 PM Mike Thomsen <[email protected]> >> wrote: >> >>> If this is a standard log analysis use case, I would recommend the ELK >>> stack instead (File Beat + Logstash + Elastic and Kibana). >>> >>> On Mon, Mar 4, 2019 at 10:21 AM K Makeev <[email protected]> wrote: >>> >>>> Hi all >>>> >>>> Using Nifi 1.5 >>>> I want my certain processors Erorrs to be logged to Oracle database >>>> >>>> So i set up separate dbaappender in logback.xml. >>>> I also created tables in database where logs should go with these >>>> scripts >>>> >>>> https://github.com/qos-ch/logback/blob/master/logback-classic/src/main/resources/ch/qos/logback/classic/db/script/oracle.sql >>>> >>>> Those are my additional entries to logback.xml >>>> >>>> <appender name="ora-pd" class="ch.qos.logback.classic.db.DBAppender"> >>>> <appender name="DB" class="ch.qos.logback.classic.db.DBAppender"> >>>> <connectionSource >>>> class="ch.qos.logback.core.db.DriverManagerConnectionSource"> >>>> <driverClass>oracle.jdbc.driver.OracleDriver</driverClass> >>>> <url>jdbc:oracle:thin:@myhost:1521/myservice_name</url> >>>> <user>myuser</user> >>>> <password>mypassword</password> >>>> </connectionSource> >>>> </appender> >>>> >>>> <logger name="org.apache.nifi.processors.standard.ExecuteSQL" >>>> level="ERROR"/> >>>> <appender-ref ref="ora-pd" /> >>>> </logger> >>>> >>>> I've restarted NIFI. But it still logs ExecuteSQL errors to >>>> nifi-app.log. >>>> >>>> May be i should check some logback errors in its own log. But i don't >>>> know where it is. >>>> May be it doesn't see Oracle driver or smth. Nifi itself connects to >>>> these database properly. >>>> >>>> How should i set up database logging properly? >>>> >>>> Kirill >>>> >>>
