Since I'm not so sure about your exact use case, I have just created a
rough template based on the simple example flow that I had posted earlier
which is GenerateFlowfile -> UpdateAttribute -> GetMongo. I have attached
the template here.
-
Sivaprasanna
On Wed, May 2, 2018 at 2:55 PM, Brajendra Mishra <
[email protected]> wrote:
> Hi Sivaprasanna,
>
> Could you please provide me the sample template for the same, where I can
> pass parameters (and get those parameters' value to process further) to
> GetMongoDB processor?
> It would be a great help for us.
>
> Brajendra Mishra
> Persistent Systems Ltd.
>
> -----Original Message-----
> From: Sivaprasanna <[email protected]>
> Sent: Wednesday, May 02, 2018 2:28 PM
> To: [email protected]
> Subject: Re: GetMongoDB : How to pass parameters as input to GetMongoDB
> processor
>
> Hi.
>
> GetMongo can take input. So technically you can use a processor before and
> then connect it to GetMongo.
>
> A simple example :
> GenerateFlowfile -> UpdateAttribute -> GetMongo
>
> In the UpdateAttribute, you can add attributes for the database and
> collection and then use them in GetMong using NiFi Expression Language.
>
> Let me know, if that doesn’t help.
>
> -
> Sivaprasanna
>
> On Wed, 2 May 2018 at 1:26 PM, Brajendra Mishra <
> [email protected]> wrote:
>
> > Hi Team,
> > We have found there is only 'GetMongoDB' processor to connect and
> > query to MongoDB in Apache NiFi.
> > Hence, we this processor does not take any type or input.
> >
> > Do we have another type to Apache NiFi processor which can take
> > parameters as input (details of MongoDB, query, instance etc.) from
> other processor?
> > If not then please suggest when such type of processor can be expected
> > in upcoming release?
> >
> > Brajendra Mishra
> > Persistent Systems Ltd.
> >
> > DISCLAIMER
> > ==========
> > This e-mail may contain privileged and confidential information which
> > is the property of Persistent Systems Ltd. It is intended only for the
> > use of the individual or entity to which it is addressed. If you are
> > not the intended recipient, you are not authorized to read, retain,
> > copy, print, distribute or use this message. If you have received this
> > communication in error, please notify the sender and delete all copies
> of this message.
> > Persistent Systems Ltd. does not accept any liability for virus
> > infected mails.
> >
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description></description>
<groupId>f6620488-0162-1000-acf2-cb751ab6e617</groupId>
<name>Hello Mongo</name>
<snippet>
<connections>
<id>944a5b06-7f3e-383e-0000-000000000000</id>
<parentGroupId>ab629e9a-a9e2-3dcc-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>ec33b40f-e496-3032-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>c97a1213-a63e-38ea-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>955cdf67-0ef5-32af-0000-000000000000</id>
<parentGroupId>ab629e9a-a9e2-3dcc-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>f9778f4b-ba15-3efb-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>ec33b40f-e496-3032-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>fe76ca18-692e-3adb-0000-000000000000</id>
<parentGroupId>ab629e9a-a9e2-3dcc-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<bends>
<x>502.5</x>
<y>429.0</y>
</bends>
<destination>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>53ce9b6e-7c8f-358c-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>f9778f4b-ba15-3efb-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>20112edd-ab5f-3ab0-0000-000000000000</id>
<parentGroupId>ab629e9a-a9e2-3dcc-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>53ce9b6e-7c8f-358c-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>original</selectedRelationships>
<source>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>f9778f4b-ba15-3efb-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>5a4151fd-910f-3ef9-0000-000000000000</id>
<parentGroupId>ab629e9a-a9e2-3dcc-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<bends>
<x>502.5</x>
<y>279.0</y>
</bends>
<destination>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>53ce9b6e-7c8f-358c-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>failure</selectedRelationships>
<source>
<groupId>ab629e9a-a9e2-3dcc-0000-000000000000</groupId>
<id>f9778f4b-ba15-3efb-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<processors>
<id>c97a1213-a63e-38ea-0000-000000000000</id>
<parentGroupId>ab629e9a-a9e2-3dcc-0000-000000000000</parentGroupId>
<position>
<x>2.0</x>
<y>4.0</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.6.0</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>File Size</key>
<value>
<name>File Size</name>
</value>
</entry>
<entry>
<key>Batch Size</key>
<value>
<name>Batch Size</name>
</value>
</entry>
<entry>
<key>Data Format</key>
<value>
<name>Data Format</name>
</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>
<name>Unique FlowFiles</name>
</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
<value>
<name>generate-ff-custom-text</name>
</value>
</entry>
<entry>
<key>character-set</key>
<value>
<name>character-set</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>File Size</key>
<value>0B</value>
</entry>
<entry>
<key>Batch Size</key>
<value>1</value>
</entry>
<entry>
<key>Data Format</key>
<value>Text</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>false</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
</entry>
<entry>
<key>character-set</key>
<value>UTF-8</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>60 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>GenerateFlowFile</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.standard.GenerateFlowFile</type>
</processors>
<processors>
<id>ec33b40f-e496-3032-0000-000000000000</id>
<parentGroupId>ab629e9a-a9e2-3dcc-0000-000000000000</parentGroupId>
<position>
<x>652.0</x>
<y>0.0</y>
</position>
<bundle>
<artifact>nifi-update-attribute-nar</artifact>
<group>org.apache.nifi</group>
<version>1.6.0</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Delete Attributes Expression</key>
<value>
<name>Delete Attributes Expression</name>
</value>
</entry>
<entry>
<key>Store State</key>
<value>
<name>Store State</name>
</value>
</entry>
<entry>
<key>Stateful Variables Initial Value</key>
<value>
<name>Stateful Variables Initial Value</name>
</value>
</entry>
<entry>
<key>mongodb.collection</key>
<value>
<name>mongodb.collection</name>
</value>
</entry>
<entry>
<key>mongodb.database</key>
<value>
<name>mongodb.database</name>
</value>
</entry>
<entry>
<key>mongodb.query</key>
<value>
<name>mongodb.query</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Delete Attributes Expression</key>
</entry>
<entry>
<key>Store State</key>
<value>Do not store state</value>
</entry>
<entry>
<key>Stateful Variables Initial Value</key>
</entry>
<entry>
<key>mongodb.collection</key>
<value>yourcollection</value>
</entry>
<entry>
<key>mongodb.database</key>
<value>yourdb</value>
</entry>
<entry>
<key>mongodb.query</key>
<value>yourquery</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>UpdateAttribute</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.attributes.UpdateAttribute</type>
</processors>
<processors>
<id>f9778f4b-ba15-3efb-0000-000000000000</id>
<parentGroupId>ab629e9a-a9e2-3dcc-0000-000000000000</parentGroupId>
<position>
<x>655.0</x>
<y>291.0</y>
</position>
<bundle>
<artifact>nifi-mongodb-nar</artifact>
<group>org.apache.nifi</group>
<version>1.6.0</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Mongo URI</key>
<value>
<name>Mongo URI</name>
</value>
</entry>
<entry>
<key>Mongo Database Name</key>
<value>
<name>Mongo Database Name</name>
</value>
</entry>
<entry>
<key>Mongo Collection Name</key>
<value>
<name>Mongo Collection Name</name>
</value>
</entry>
<entry>
<key>ssl-context-service</key>
<value>
<identifiesControllerService>org.apache.nifi.ssl.SSLContextService</identifiesControllerService>
<name>ssl-context-service</name>
</value>
</entry>
<entry>
<key>ssl-client-auth</key>
<value>
<name>ssl-client-auth</name>
</value>
</entry>
<entry>
<key>json-type</key>
<value>
<name>json-type</name>
</value>
</entry>
<entry>
<key>use-pretty-printing</key>
<value>
<name>use-pretty-printing</name>
</value>
</entry>
<entry>
<key>mongo-charset</key>
<value>
<name>mongo-charset</name>
</value>
</entry>
<entry>
<key>Query</key>
<value>
<name>Query</name>
</value>
</entry>
<entry>
<key>mongo-query-attribute</key>
<value>
<name>mongo-query-attribute</name>
</value>
</entry>
<entry>
<key>Projection</key>
<value>
<name>Projection</name>
</value>
</entry>
<entry>
<key>Sort</key>
<value>
<name>Sort</name>
</value>
</entry>
<entry>
<key>Limit</key>
<value>
<name>Limit</name>
</value>
</entry>
<entry>
<key>Batch Size</key>
<value>
<name>Batch Size</name>
</value>
</entry>
<entry>
<key>results-per-flowfile</key>
<value>
<name>results-per-flowfile</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Mongo URI</key>
<value>localhost:27017</value>
</entry>
<entry>
<key>Mongo Database Name</key>
<value>${mongodb.database}</value>
</entry>
<entry>
<key>Mongo Collection Name</key>
<value>${mongodb.collection}</value>
</entry>
<entry>
<key>ssl-context-service</key>
</entry>
<entry>
<key>ssl-client-auth</key>
<value>REQUIRED</value>
</entry>
<entry>
<key>json-type</key>
<value>Extended</value>
</entry>
<entry>
<key>use-pretty-printing</key>
<value>true</value>
</entry>
<entry>
<key>mongo-charset</key>
<value>UTF-8</value>
</entry>
<entry>
<key>Query</key>
<value>${mongodb.query}</value>
</entry>
<entry>
<key>mongo-query-attribute</key>
</entry>
<entry>
<key>Projection</key>
</entry>
<entry>
<key>Sort</key>
</entry>
<entry>
<key>Limit</key>
</entry>
<entry>
<key>Batch Size</key>
</entry>
<entry>
<key>results-per-flowfile</key>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>GetMongo</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>failure</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>original</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.mongodb.GetMongo</type>
</processors>
<processors>
<id>53ce9b6e-7c8f-358c-0000-000000000000</id>
<parentGroupId>ab629e9a-a9e2-3dcc-0000-000000000000</parentGroupId>
<position>
<x>0.0</x>
<y>287.0</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.6.0</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Log Level</key>
<value>
<name>Log Level</name>
</value>
</entry>
<entry>
<key>Log Payload</key>
<value>
<name>Log Payload</name>
</value>
</entry>
<entry>
<key>Attributes to Log</key>
<value>
<name>Attributes to Log</name>
</value>
</entry>
<entry>
<key>attributes-to-log-regex</key>
<value>
<name>attributes-to-log-regex</name>
</value>
</entry>
<entry>
<key>Attributes to Ignore</key>
<value>
<name>Attributes to Ignore</name>
</value>
</entry>
<entry>
<key>attributes-to-ignore-regex</key>
<value>
<name>attributes-to-ignore-regex</name>
</value>
</entry>
<entry>
<key>Log prefix</key>
<value>
<name>Log prefix</name>
</value>
</entry>
<entry>
<key>character-set</key>
<value>
<name>character-set</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Log Level</key>
<value>info</value>
</entry>
<entry>
<key>Log Payload</key>
<value>false</value>
</entry>
<entry>
<key>Attributes to Log</key>
</entry>
<entry>
<key>attributes-to-log-regex</key>
<value>.*</value>
</entry>
<entry>
<key>Attributes to Ignore</key>
</entry>
<entry>
<key>attributes-to-ignore-regex</key>
</entry>
<entry>
<key>Log prefix</key>
</entry>
<entry>
<key>character-set</key>
<value>windows-1252</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>LogAttribute</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.standard.LogAttribute</type>
</processors>
</snippet>
<timestamp>05/02/2018 16:08:46 IST</timestamp>
</template>
