That might require 1.6.0. Also, your image didn't come through in your
response to Sivaprasanna so resend that too.
On Wed, May 2, 2018 at 8:37 AM Brajendra Mishra <
[email protected]> wrote:
> Hi Mike,
>
> Thanks a lot for responding.
>
> On your statement
> "That is its new default behavior if you leave the query field blank and
> have an incoming connection from another processor. That would be a good
> way to integrate the flow with another application"
>
> Could you please share a sample template for the same?
>
>
> Brajendra Mishra
> Persistent Systems Ltd.
>
> -----Original Message-----
> From: Mike Thomsen <[email protected]>
> Sent: Wednesday, May 02, 2018 5:58 PM
> To: [email protected]
> Subject: Re: GetMongoDB : How to pass parameters as input to GetMongoDB
> processor
>
> GetMongo can also use the body of a flowfile for the query. That is its
> new default behavior if you leave the query field blank and have an
> incoming connection from another processor. That would be a good way to
> integrate the flow with another application. For example, you could add
> FetchKafka to the flow and have your applications post messages to Kafka
> with the queries they want it to run and FetchKafka would send that JSON to
> GetMongo as it comes in. Or you could build a REST service that writes the
> JSON to disk and use GetFile to load it. Lots of ways to do this.
>
> On Wed, May 2, 2018 at 6:42 AM Sivaprasanna <[email protected]>
> wrote:
>
> > Since I'm not so sure about your exact use case, I have just created a
> > rough template based on the simple example flow that I had posted
> > earlier which is GenerateFlowfile -> UpdateAttribute -> GetMongo. I
> > have attached the template here.
> >
> > -
> > Sivaprasanna
> >
> > On Wed, May 2, 2018 at 2:55 PM, Brajendra Mishra <
> > [email protected]> wrote:
> >
> >> Hi Sivaprasanna,
> >>
> >> Could you please provide me the sample template for the same, where I
> >> can pass parameters (and get those parameters' value to process
> >> further) to GetMongoDB processor?
> >> It would be a great help for us.
> >>
> >> Brajendra Mishra
> >> Persistent Systems Ltd.
> >>
> >> -----Original Message-----
> >> From: Sivaprasanna <[email protected]>
> >> Sent: Wednesday, May 02, 2018 2:28 PM
> >> To: [email protected]
> >> Subject: Re: GetMongoDB : How to pass parameters as input to
> >> GetMongoDB processor
> >>
> >> Hi.
> >>
> >> GetMongo can take input. So technically you can use a processor
> >> before and then connect it to GetMongo.
> >>
> >> A simple example :
> >> GenerateFlowfile -> UpdateAttribute -> GetMongo
> >>
> >> In the UpdateAttribute, you can add attributes for the database and
> >> collection and then use them in GetMong using NiFi Expression Language.
> >>
> >> Let me know, if that doesn’t help.
> >>
> >> -
> >> Sivaprasanna
> >>
> >> On Wed, 2 May 2018 at 1:26 PM, Brajendra Mishra <
> >> [email protected]> wrote:
> >>
> >> > Hi Team,
> >> > We have found there is only 'GetMongoDB' processor to connect and
> >> > query to MongoDB in Apache NiFi.
> >> > Hence, we this processor does not take any type or input.
> >> >
> >> > Do we have another type to Apache NiFi processor which can take
> >> > parameters as input (details of MongoDB, query, instance etc.) from
> >> other processor?
> >> > If not then please suggest when such type of processor can be
> >> > expected in upcoming release?
> >> >
> >> > Brajendra Mishra
> >> > Persistent Systems Ltd.
> >> >
> >> > DISCLAIMER
> >> > ==========
> >> > This e-mail may contain privileged and confidential information
> >> > which is the property of Persistent Systems Ltd. It is intended
> >> > only for the use of the individual or entity to which it is
> >> > addressed. If you are not the intended recipient, you are not
> >> > authorized to read, retain, copy, print, distribute or use this
> >> > message. If you have received this communication in error, please
> >> > notify the sender and delete all copies
> >> of this message.
> >> > Persistent Systems Ltd. does not accept any liability for virus
> >> > infected mails.
> >> >
> >> DISCLAIMER
> >> ==========
> >> This e-mail may contain privileged and confidential information which
> >> is the property of Persistent Systems Ltd. It is intended only for
> >> the use of the individual or entity to which it is addressed. If you
> >> are not the intended recipient, you are not authorized to read,
> >> retain, copy, print, distribute or use this message. If you have
> >> received this communication in error, please notify the sender and
> delete all copies of this message.
> >> Persistent Systems Ltd. does not accept any liability for virus
> >> infected mails.
> >>
> >
> >
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description></description>
<groupId>f7f2cefb-0162-1000-5360-32dc881c424f</groupId>
<name>GetMongo with input</name>
<snippet>
<connections>
<id>429ddec4-3650-3b76-0000-000000000000</id>
<parentGroupId>5bb9a3e0-6639-326e-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>5bb9a3e0-6639-326e-0000-000000000000</groupId>
<id>8f073700-234a-3b8f-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>5bb9a3e0-6639-326e-0000-000000000000</groupId>
<id>1b3af30a-8ced-3f47-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>6a9fb33f-c6e4-3244-0000-000000000000</id>
<parentGroupId>5bb9a3e0-6639-326e-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>5bb9a3e0-6639-326e-0000-000000000000</groupId>
<id>00d30fc0-dd9c-3224-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>5bb9a3e0-6639-326e-0000-000000000000</groupId>
<id>8f073700-234a-3b8f-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<processors>
<id>8f073700-234a-3b8f-0000-000000000000</id>
<parentGroupId>5bb9a3e0-6639-326e-0000-000000000000</parentGroupId>
<position>
<x>439.99013978648566</x>
<y>261.0930979464067</y>
</position>
<bundle>
<artifact>nifi-mongodb-nar</artifact>
<group>org.apache.nifi</group>
<version>1.7.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Mongo URI</key>
<value>
<name>Mongo URI</name>
</value>
</entry>
<entry>
<key>Mongo Database Name</key>
<value>
<name>Mongo Database Name</name>
</value>
</entry>
<entry>
<key>Mongo Collection Name</key>
<value>
<name>Mongo Collection Name</name>
</value>
</entry>
<entry>
<key>ssl-context-service</key>
<value>
<identifiesControllerService>org.apache.nifi.ssl.SSLContextService</identifiesControllerService>
<name>ssl-context-service</name>
</value>
</entry>
<entry>
<key>ssl-client-auth</key>
<value>
<name>ssl-client-auth</name>
</value>
</entry>
<entry>
<key>json-type</key>
<value>
<name>json-type</name>
</value>
</entry>
<entry>
<key>use-pretty-printing</key>
<value>
<name>use-pretty-printing</name>
</value>
</entry>
<entry>
<key>mongo-charset</key>
<value>
<name>mongo-charset</name>
</value>
</entry>
<entry>
<key>Query</key>
<value>
<name>Query</name>
</value>
</entry>
<entry>
<key>mongo-query-attribute</key>
<value>
<name>mongo-query-attribute</name>
</value>
</entry>
<entry>
<key>Projection</key>
<value>
<name>Projection</name>
</value>
</entry>
<entry>
<key>Sort</key>
<value>
<name>Sort</name>
</value>
</entry>
<entry>
<key>Limit</key>
<value>
<name>Limit</name>
</value>
</entry>
<entry>
<key>Batch Size</key>
<value>
<name>Batch Size</name>
</value>
</entry>
<entry>
<key>results-per-flowfile</key>
<value>
<name>results-per-flowfile</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Mongo URI</key>
<value>mongodb://localhost:27018</value>
</entry>
<entry>
<key>Mongo Database Name</key>
<value>scraper</value>
</entry>
<entry>
<key>Mongo Collection Name</key>
<value>events</value>
</entry>
<entry>
<key>ssl-context-service</key>
</entry>
<entry>
<key>ssl-client-auth</key>
<value>REQUIRED</value>
</entry>
<entry>
<key>json-type</key>
<value>Standard</value>
</entry>
<entry>
<key>use-pretty-printing</key>
<value>true</value>
</entry>
<entry>
<key>mongo-charset</key>
<value>UTF-8</value>
</entry>
<entry>
<key>Query</key>
</entry>
<entry>
<key>mongo-query-attribute</key>
</entry>
<entry>
<key>Projection</key>
</entry>
<entry>
<key>Sort</key>
</entry>
<entry>
<key>Limit</key>
</entry>
<entry>
<key>Batch Size</key>
</entry>
<entry>
<key>results-per-flowfile</key>
<value>250</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>GetMongo</name>
<relationships>
<autoTerminate>true</autoTerminate>
<name>failure</name>
</relationships>
<relationships>
<autoTerminate>true</autoTerminate>
<name>original</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>RUNNING</state>
<style/>
<type>org.apache.nifi.processors.mongodb.GetMongo</type>
</processors>
<processors>
<id>00d30fc0-dd9c-3224-0000-000000000000</id>
<parentGroupId>5bb9a3e0-6639-326e-0000-000000000000</parentGroupId>
<position>
<x>971.5399825576017</x>
<y>159.85952401280883</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.7.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Log Level</key>
<value>
<name>Log Level</name>
</value>
</entry>
<entry>
<key>Log Payload</key>
<value>
<name>Log Payload</name>
</value>
</entry>
<entry>
<key>Attributes to Log</key>
<value>
<name>Attributes to Log</name>
</value>
</entry>
<entry>
<key>attributes-to-log-regex</key>
<value>
<name>attributes-to-log-regex</name>
</value>
</entry>
<entry>
<key>Attributes to Ignore</key>
<value>
<name>Attributes to Ignore</name>
</value>
</entry>
<entry>
<key>attributes-to-ignore-regex</key>
<value>
<name>attributes-to-ignore-regex</name>
</value>
</entry>
<entry>
<key>Log prefix</key>
<value>
<name>Log prefix</name>
</value>
</entry>
<entry>
<key>character-set</key>
<value>
<name>character-set</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Log Level</key>
<value>info</value>
</entry>
<entry>
<key>Log Payload</key>
<value>false</value>
</entry>
<entry>
<key>Attributes to Log</key>
</entry>
<entry>
<key>attributes-to-log-regex</key>
<value>.*</value>
</entry>
<entry>
<key>Attributes to Ignore</key>
</entry>
<entry>
<key>attributes-to-ignore-regex</key>
</entry>
<entry>
<key>Log prefix</key>
</entry>
<entry>
<key>character-set</key>
<value>UTF-8</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>LogAttribute</name>
<relationships>
<autoTerminate>true</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.standard.LogAttribute</type>
</processors>
<processors>
<id>1b3af30a-8ced-3f47-0000-000000000000</id>
<parentGroupId>5bb9a3e0-6639-326e-0000-000000000000</parentGroupId>
<position>
<x>0.0</x>
<y>0.0</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.7.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>File Size</key>
<value>
<name>File Size</name>
</value>
</entry>
<entry>
<key>Batch Size</key>
<value>
<name>Batch Size</name>
</value>
</entry>
<entry>
<key>Data Format</key>
<value>
<name>Data Format</name>
</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>
<name>Unique FlowFiles</name>
</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
<value>
<name>generate-ff-custom-text</name>
</value>
</entry>
<entry>
<key>character-set</key>
<value>
<name>character-set</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>File Size</key>
<value>0B</value>
</entry>
<entry>
<key>Batch Size</key>
<value>1</value>
</entry>
<entry>
<key>Data Format</key>
<value>Text</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>false</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
<value>{ "site": { "$in": ["CNN", "FoxNews"] }}</value>
</entry>
<entry>
<key>character-set</key>
<value>UTF-8</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>15 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>GenerateFlowFile</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style/>
<type>org.apache.nifi.processors.standard.GenerateFlowFile</type>
</processors>
</snippet>
<timestamp>05/02/2018 08:48:45 EDT</timestamp>
</template>
