Ameer,
Depending on how you implemented the custom framework, you may be able to
easily drop it in place into a custom NiFi processor. Without knowing much
about your implementation details, if you can act on Java streams, Strings,
byte arrays and things like that it will probably be very straight forward
to drop in place.
This is a really simple of how you could bring it in depending on how
encapsulated your business logic is:
@Override
public void onTrigger(ProcessContext context, ProcessSession session)
throws ProcessException {
FlowFile input = session.get();
if (input == null) {
return;
}
FlowFile output = session.create(input);
try (InputStream is = session.read(input);
OutputStream os = session.write(output)
) {
transformerPojo.transform(is, os);
is.close();
os.close();
session.transfer(input, REL_ORIGINAL); //If you created an
"original relationship"
session.transfer(output, REL_SUCCESS);
} catch (Exception ex) {
session.remove(output);
session.transfer(input, REL_FAILURE);
}
}
That's the general idea, and that approach can scale to your disk space
limits. Hope that helps put it into perspective.
Mike
On Thu, Nov 1, 2018 at 10:16 AM Nathan Gough <thena...@gmail.com> wrote:
> Hi Ameer,
>
> This blog by Mark Payne describes how to manipulate record based data like
> CSV using schemas:
> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi. This
> would probably be the most efficient method. And another here:
> https://bryanbende.com/development/2017/06/20/apache-nifi-records-and-schema-registries
> .
>
> An alternative option would be to port your custom java code into your own
> NiFi processor:
>
> https://medium.com/hashmapinc/creating-custom-processors-and-controllers-in-apache-nifi-e14148740ea
> under 'Steps for Creating a Custom Apache NiFi Processor'
> https://nifi.apache.org/developer-guide.html
>
> Nathan
>
> On 10/31/18, 5:02 PM, "Ameer Mawia" <ameer.ma...@gmail.com> wrote:
>
> We have a use case where we take data from a source(text data in csv
> format), do transformation and manipulation of textual record, and
> output
> the data in another (csv)format. This is being done by a Java based
> custom
> framework, written specifically for this *transformation* piece.
>
> Recently as Apache NIFI is being adopted at enterprise level by the
> organisation, we have been asked to try *Apache NIFI* and see if can
> use
> that as a replacement to this custom tool?
>
> *My question is*:
>
> - How much leverage does *Apache NIFI *provides on the flowfile
> *content
> *manipulation?
>
> I understand *NIFI *is good for creating data flow pipeline, but is it
> good
> for *extensive TEXT Transformation* as well? So far I have not found
> obvious way to achieve that.
>
> Appreciate the feedback.
>
> Thanks,
>
> --
> http://ca.linkedin.com/in/ameermawia
> Toronto, ON
>
>
>
>
