Hi folks, thanks for the explanations! I've opened a JIRA ticket describing
this issues plus a few more that I faced since I started learning NiFi (
https://issues.apache.org/jira/browse/NIFI-4569). It got a little verbose
so forgive-me and feel free to ignore it. ;)
Most of my hit backs I attribute to my lack of experience with Java/groovy
and really to flaws of NiFi.
@Matt any chance of a future post series "service scripting cookbook"? ;)
Jokes apart it would be a great asset to have a post or document showcasing
the most common service interfaces.
Thanks again for the help!
2017-11-05 14:14 GMT-02:00 Matt Burgess <mattyb...@apache.org>:
> Eric,
>
> Because LookupService implements ControllerService, you must implement
> initialize(ControllerServiceInitializationContext context), which
> Andy's script provides an empty body for. However that context object
> has a method called getLogger() on it, so you can override the
> initialize() method and save off the logger for later use:
>
> class GroovyLookupService implements StringLookupService {
>
> def log
>
> @Override
> void initialize(ControllerServiceInitializationContext context) throws
> InitializationException {
> log = context.logger
> }
>
> // Other stuff
> }
>
> I admit it's a bit awkward to get at, but the ScriptedLookupService is
> more like InvokeScriptedProcessor (ISP) than ExecuteScript; the latter
> has a "log" binding so you can make immediate use of it; the former is
> more like a full implementation of the interface, so you usually get
> the logger a different way (i.e. an init method of some kind). Having
> said that, the ISP-based scripting components also attempt to call
> setLogger(ComponentLog logger) on your object if that method exists
> and the script engine is Invocable (Groovy is).
>
> I can look at adding the "log" object as a binding in all scripting
> components, I chose not to for the ISP-based processors in order to
> allow more flexibility with how/when logging is done (i.e. "log" is a
> global logger but ISP-based components have class definitions possibly
> with their own logger). But if it is easier in these cases to just
> have a "log" object to use, then that's fine with me, please feel free
> to mention that in the Jira.
>
> Regarding the awkwardness of setting log values, it might be nice to
> have a script (either in NiFi or available for download somewhere) to
> be able to update those without having to scour the logback.xml. Next
> time I have a few minutes I'll whip up a Groovy script to try it out.
> BTW if you didn't want to mark all processors as INFO (just your
> ScriptedLookupService), you can add a separate entry in logback.xml
> for the fully-qualified class name
> (org.apache.nifi.lookup.script.ScriptedLookupService in this case) and
> just set that level to INFO. Then the other processors would remain at
> WARN and not "clog the log" while you're looking for something
> specific. I'll make sure my script handles that (I envision a Groovy
> script that loads in the whole logback DOM, finds the node if it
> exists and update the level, or add a node if it does not, then writes
> it back out to logback.xml).
>
> Regards,
> Matt
>
>
>
> On Sun, Nov 5, 2017 at 9:45 AM, Joe Witt <joe.w...@gmail.com> wrote:
> > eric
> >
> > can you please file a JIRA to reflect the awkward process you had to
> > go through so we can improve it.
> >
> > Thanks
> >
> > On Sun, Nov 5, 2017 at 9:37 AM, Eric Chaves <e...@uolet.com> wrote:
> >> Ok, I managed to make to make it work. I had to add "log.dir=./logs" to
> >> bootstrap.conf file in order to have the logs generated. I got a little
> lost
> >> because this property is not mentioned at the docs but I could figured
> it
> >> out reading the logback.xml
> >>
> >> Once the logs began to be generated I could see that an exception was
> being
> >> raised in my scripts because I was trying to access "log" which does not
> >> seem to be available in Service controllers. Once I removed the
> offending
> >> log line, the code worked. Bottom line my attempt to debug the code was
> >> actually breaking it. ;)
> >>
> >> Thanks!
> >>
> >> 2017-11-05 12:14 GMT-02:00 Mike Thomsen <mikerthom...@gmail.com>:
> >>>
> >>> They go to logs/nifi-app.log. I think <logger
> >>> name="org.apache.nifi.processors" level="INFO"/> is actually WARN by
> >>> default. You'll need to tinker with the log levels (it's not straight
> >>> enable/disable) to get that working.
> >>>
> >>> On Sun, Nov 5, 2017 at 8:54 AM, Eric Chaves <e...@uolet.com> wrote:
> >>>>
> >>>> Hi Mike, I'm running nifi using the official docker image (version
> 1.4.0)
> >>>> and my logs folder is empty. I was looking at bootstrap.conf,
> logback.xml
> >>>> and nifi.properties but couldn't found any config value that may
> >>>> disable/enable log. Where should those logs be going?
> >>>>
> >>>> 2017-11-04 12:55 GMT-02:00 Mike Thomsen <mikerthom...@gmail.com>:
> >>>>>
> >>>>> You may need to update the logback xml file in the conf folder.
> There is
> >>>>> a line in there for the processor package. Might be too high for
> info.
> >>>>>
> >>>>> On Sat, Nov 4, 2017 at 10:50 AM Eric Chaves <e...@uolet.com> wrote:
> >>>>>>
> >>>>>> Hi folks,
> >>>>>>
> >>>>>> I'm trying to adapt the flow described at
> >>>>>> https://community.hortonworks.com/articles/138632/data-flow-
> enrichment-with-nifi-lookuprecord-proces.html
> >>>>>> using ScriptedLookupService as replacement for
> SimpleKeyValueLookupService
> >>>>>> to lookup city names and enrich and incoming record.
> >>>>>>
> >>>>>> When I ran the flow with KeyValueLookupService the field gets
> enriched
> >>>>>> properly but when I use my scriptedlookup the value always come
> back as
> >>>>>> null. The script was pretty simple and I can't figure out where is
> my
> >>>>>> error. I also tried the ScriptLookup (just the script, not the
> flow) by
> >>>>>> AloPresto at
> >>>>>> https://gist.github.com/alopresto/78eb1a2c2b878f75f61481269af38a9f
> with the
> >>>>>> same resutls.
> >>>>>>
> >>>>>> I'm trying to log.info the execution to figure out my mistakes but
> the
> >>>>>> logs are going nowhere. How can I enable logging for services?
> >>>>>>
> >>>>>> Does anyone spot an error?
> >>>>>>
> >>>>>> ---[service-lookup.groovy]---
> >>>>>> import org.apache.nifi.lookup.StringLookupService
> >>>>>>
> >>>>>> class GroovyLookupService implements StringLookupService {
> >>>>>>
> >>>>>> def lookupTable = [
> >>>>>> '1': 'Paris',
> >>>>>> '2': 'Lyon',
> >>>>>> '3': 'Marseille',
> >>>>>> '4': 'Toulouse',
> >>>>>> '5': 'Nice'
> >>>>>> ]
> >>>>>>
> >>>>>> @Override
> >>>>>> Optional<String> lookup(final String key) {
> >>>>>> log.warn('key value: ', key)
> >>>>>> return Optional.ofNullable(lookupTable[key])
> >>>>>> }
> >>>>>> }
> >>>>>>
> >>>>>> lookupService = new GroovyLookupService()
> >>>>>> ---
> >>>>>>
> >>>>
> >>>
> >>
>
