Hi, Thanks for your approval! I created a ticket for that. For now, containing the proposal. Later I will update it as I go on, I hope that is okay with you.
https://issues.apache.org/jira/browse/NIFI-7188 <https://issues.apache.org/jira/browse/NIFI-7188> As this is not the only thing on my plate right now, I did not include graph db or other bigger changes, but I would like to keep it slim. If it works out for us, I keep the possibility open to extend it :) Best Regards, Bence > On 2020. Feb 21., at 23:51, Emanuel Oliveira <[email protected]> wrote: > > I like it :) > > Best Regards, > *Emanuel Oliveira* > > > > On Fri, Feb 21, 2020 at 10:37 PM [email protected] <[email protected]> > wrote: > >> Martin, >> >> correct me if I’m wrong, but Neo4J is not open source. Wouldn't it be >> generally better to use Janusgraph for this? >> Especially since many major companies use Janusgraph: IBM Cloud, Netflix, >> Uber... >> >> A complete and fully functional lineage is already available with Apache >> Atlas. In my opinion it would be more complicated to build it in NiFi than >> to integrate Apache Atlas, whereas it is already standard in NiFi to bridge >> to Atlas. >> >> Or did I misunderstand something? >> >> Mit freundlichen Grüßen / best regards >> Kay-Uwe Moosheimer >> >>> Am 21.02.2020 um 19:42 schrieb Martin Ebert <[email protected]>: >>> >>> We still think about building a graph based search (Neo4j) in top of >> NiFi. >>> Would be also fantastic to have it within NiFi. >>> >>> There are plenty of examples >>> >> https://blog.grandstack.io/using-neo4js-full-text-search-with-graphql-e3fa484de2ea >>> From the idea it could go in this direction - of course much more >>> rudimentary. Then one would have the possibility to have only the results >>> displayed as text or to find out exploratory connections (graph layout). >>> The built-in data lineage function of NiFi would also benefit from the >>> power of Neo4j. >>> >>> Simon Bence <[email protected]> schrieb am Fr., 21. Feb. 2020, >> 19:00: >>> >>>> Dear Community, >>>> >>>> In my project, I do use relatively high number of processors and process >>>> groups. The current search function on the NiFi UI has no >> capabilitites to >>>> narrow the results based on the group, which would make the results more >>>> relevant, so I would like to propose a possible solution. Please if you >>>> have any comment on this, do not hesitate to share it. >>>> >>>> The general approach would be to keep the current text box and extend >> the >>>> server side capabilities to process search query in the similar manner >> for >>>> example the Google search behaves.This extensions I would call >> "filters". >>>> For now I am interested in the ones I will mention below, but I think, >> it >>>> is only a matter of small work for further extend the solution with >> further >>>> ones. >>>> >>>> In order to distinguish the filters from the rest of the search query, I >>>> propose to put them at the beginning of the query and use the >>>> [a-zA-Z0-9\.]{1..n}\:[a-zA-Z0-9\.]{1..n} format. For example a filter >> might >>>> look the following: lorem:ipsum >>>> >>>> Adding this, the search query should look like the following: >>>> >>>> filter1:value filter2:value rest of the query >>>> >>>> As for processing the filters, I suggest the following behaviour: >>>> >>>> - Without filters the current behaviour should be kept >>>> - Everything after the filters should be handled as the search term >>>> - After the first "non filter word", anything should be considered as >> part >>>> of the search term (meaning: to keep the text parsing simple, I would >> not >>>> go in the direction to support filters at the end of the query, etc.) >>>> - The ordering of the filters should have no effect on the result >>>> - Filter duplications should be eliminated >>>> - In case a filter appears multiple times in the query, the first >> occasion >>>> will be used >>>> - Unknown filters should be ignored >>>> - Only adding filters will not end up with result, at least one >> character >>>> must appear as search term >>>> >>>> Suggested filters: >>>> >>>> scope >>>> Narrows the search based on the user's currently active process group. >> The >>>> allowed values are: "all" and "here". All produces the current >> behaviour, >>>> thus no filtering happens, but "here" should use the current process >> group >>>> as "root" of the search, ignoring everything else (including parent >> group). >>>> Note: This needs a minimal frontend change, because as I did see, >> currently >>>> the current group is not sent with the search query. >>>> >>>> group >>>> Narrows the search for a given processing group, if it exists. The >>>> behaviour is recursive, thus the result will include the contained >> groups >>>> as well. If it is a non-existing group, the result list should be empty. >>>> >>>> properties >>>> Controls if properties values are included or not. If not provided, the >>>> property values will be included. This is because in a lot of cases >> there >>>> is a huge number of results come from property names. >>>> >>>> - Valid values for inclusion: yes, true, include, 1 >>>> - Valid values for exclusion: no, none, false, exclude, 0 >>>> >>>> It is possible that the range of possible values should be limited (and >> not >>>> being ambiguous), but I see a merit of "permissiveness" here as it is >>>> simpler to remember. >>>> >>>> Also some example: >>>> >>>> 1. >>>> scope:here properties:exclude lorem ipsum >>>> This should search only in the current group (and it's children), >> excluding >>>> properties and return with components containing the "lorem ipsum" >>>> expression. >>>> >>>> 2. >>>> group:myGroup someQuery >>>> This should result the finding of components with someQuery expression, >> but >>>> only within the myGroup group, even if it is not the active one. >>>> >>>> 3. >>>> scope:all properties:include lorem >>>> This should behave the same as "lorem" without filters. >>>> >>>> Thanks for reading, I am interested to hear your opinion! >>>> >>>> Kind regards, >>>> Bence >>>> >> >>
