One consideration for why it does not support EL is due to client the processor is wrapping that registers with a given endpoint. EL would require this disconnect/reconnection process to potentially happen on every FlowFile presented to the processor (some smart caching could certainly lessen the effect). Currently, filtering and such is very much integrated with the lifecycle of the processor. A more dynamic processor could be achieved, but will come with a few caveats.
On Thu, Aug 25, 2016 at 1:03 PM, Sven Davison <svendavi...@gmail.com> wrote: > thats, close to the same flow i was looking at really. but was chucked out > for lack of EL support w/in GetTwitter. The good news is... we're learning! > > On Thu, Aug 25, 2016 at 12:52 PM, Andy LoPresto <alopre...@apache.org> > wrote: > >> Hi Sven, >> >> Someone may have a more streamlined solution, but I’d suggest taking a >> look at ExecuteSQL [1] to read from the database, ConvertAvroToJSON [2] to >> convert the output of the SQL query to JSON, and EvaluateJsonPath [3] to >> extract the specific values you are interested in. Then use UpdateAttribute >> [4] to populate those values from the flowfile content to an attribute, and >> finally use GetTwitter [5] to filter on those values. >> >> However, at this time the query fields in GetTwitter do not support >> Expression Language, so you will have to: >> >> * Modify the source of GetTwitter to support EL >> * Raise a Jira requesting this feature >> * Write a small script wrapping GetTwitter using ExecuteScript [6] to >> populate those values >> >> Sorry it’s not a cleaner solution. I would encourage you to raise the >> Jira [7] to have GetTwitter support EL in the query properties. It’s likely >> I am overlooking a potential simpler flow, but without EL support in >> GetTwitter, I don’t see an easy way forward. >> >> [1] https://nifi.apache.org/docs/nifi-docs/components/org.ap >> ache.nifi.processors.standard.ExecuteSQL/index.html >> [2] https://nifi.apache.org/docs/nifi-docs/components/org.ap >> ache.nifi.processors.avro.ConvertAvroToJSON/index.html >> [3] https://nifi.apache.org/docs/nifi-docs/components/org.ap >> ache.nifi.processors.standard.EvaluateJsonPath/index.html >> [4] https://nifi.apache.org/docs/nifi-docs/components/org.ap >> ache.nifi.processors.attributes.UpdateAttribute/index.html >> [5] https://nifi.apache.org/docs/nifi-docs/components/org.ap >> ache.nifi.processors.twitter.GetTwitter/index.html >> [6] https://nifi.apache.org/docs/nifi-docs/components/org.ap >> ache.nifi.processors.script.ExecuteScript/index.html >> [7] https://issues.apache.org/jira/secure/CreateIssue!default.jspa >> >> Andy LoPresto >> alopre...@apache.org >> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> On Aug 25, 2016, at 9:10 AM, Sven Davison <svendavi...@gmail.com> wrote: >> >> i have a GetTwitter processor which works wonders. I'm tracking a few >> people and a couple hash tags but i'm also pulling all hashtags out of the >> posts and tracking how many times i saw it and when the last time was that >> i saw it. >> >> example tweet: "hello world #earth #usa" >> >> if i'm watching #usa, i'll still get both tags and put them into my >> database. using the tag as the id, a count for how many times it's been >> seen and a lastSeen field for when it was last seen. >> >> what i would like to do, is dynamically follow new tags upon condition X. >> Say... once #earth gets more than 500 posts and only if the tag was seen in >> the last 7 days. I can make a view in MySQL to build the result set, but >> how do i get that result set into nifi, to follow those tags that will >> change. >> >> >> >> >
