[
https://issues.apache.org/jira/browse/MINIFI-244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936636#comment-15936636
]
Joseph Witt commented on MINIFI-244:
------------------------------------
Ah ok yeah now I'm tracking on where you're heading.
- i'm good with the point about consistency being good but not required between
processors in minifi/nifi. And actually i can see why in minifi this processor
could be the difference between being able to do a certain flow or not within a
given resource constraint whereas in nifi it would be more useful than required.
- the notion of this in general does make sense too. It is for when we just
want to learn/understand/extract something buried inside a given archive to
help us decide we want to route something a certain way for followon action
(transformation/send/drop/etc..). And this could indeed be far faster for the
cases it targets.
- the downside of this is that it really struggles against the notion of highly
cohesive processors. It would require a set of components which do lens type
operations on a certain set of archive types which would duplicate other
similar ones which expect the data already split out. We could possibly have a
ControllerService instead which defines the lens and understands a given
archive type and then have the base processors built to utilize that lens
instead of just going after the raw object. Not sure but I think there is a
concept there to play with.
Anyway - this is pretty cool/innovative thinking. Glad you brought it up
> Create ArchiveLens processor
> ----------------------------
>
> Key: MINIFI-244
> URL: https://issues.apache.org/jira/browse/MINIFI-244
> Project: Apache NiFi MiNiFi
> Issue Type: Task
> Components: C++, Extensions
> Reporter: Andrew Christianson
> Assignee: Andrew Christianson
> Priority: Minor
>
> Create an ArchiveLens processor. A concise, though informal, definition of a
> lens is as follows:
> "Essentially, they represent the act of “peering into” or “focusing in on”
> some particular piece/path of a complex data object such that you can more
> precisely target particular operations without losing the context or
> structure of the overall data you’re working with."
> https://medium.com/@dtipson/functional-lenses-d1aba9e52254#.hdgsvbraq
> Why an ArchiveLens in MiNiFi? Simply put, it will enable us to "focus in on"
> an entry in the archive, perform processing *in-context* of that entry, then
> re-focus on the overall archive. This allows for transformation or other
> processing of an entry in the archive without losing the overall context of
> the archive.
> Initial format support is tar, due to its simplicity and ubiquity.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
