I've added my additional comments. Looks good enough now.
Thad +ThadGuidry <https://www.google.com/+ThadGuidry> On Tue, Jun 28, 2016 at 8:20 AM, Bryan Bende <[email protected]> wrote: > Thanks Thad. I created this JIRA: > https://issues.apache.org/jira/browse/NIFI-2135 > > If I said anything wrong there please let me know, or add to the JIRA. > > > > > On Tue, Jun 28, 2016 at 8:59 AM, Thad Guidry <[email protected]> wrote: > >> Yes, you want updateOne() >> https://docs.mongodb.com/manual/reference/method/db.collection.updateOne/ >> Not replaceOne() >> https://docs.mongodb.com/manual/reference/method/db.collection.replaceOne/ >> >> However, also look at the upsert option itself as well which does a >> replace if NO MATCH >> https://api.mongodb.com/java/3.2/com/mongodb/client/model/UpdateOptions.html >> which basically prescribes to this kind of handling >> https://docs.mongodb.com/manual/reference/method/db.collection.update/ >> >> >> Thad >> +ThadGuidry <https://www.google.com/+ThadGuidry> >> >> On Tue, Jun 28, 2016 at 7:26 AM, Bryan Bende <[email protected]> wrote: >> >>> I'm not very familiar with MongoDB, but I can see that when using >>> "update" mode PutMongo does this: >>> >>> else { >>> // update >>> final boolean upsert = context.getProperty(UPSERT).asBoolean(); >>> final String updateKey = >>> context.getProperty(UPDATE_QUERY_KEY).getValue(); >>> final Document query = new Document(updateKey, doc.get(updateKey)); >>> >>> collection.replaceOne(query, doc, new >>> UpdateOptions().upsert(upsert)); >>> logger.info("updated {} into MongoDB", new Object[] { flowFile }); >>> } >>> >>> I'm wondering if that collection.replaceOne() is the problem, I see >>> there is collection.updateOne() which sounds more correct here. >>> >>> If someone with more MongoDB experience could verify this I would be >>> happy to open a JIRA and get this changed. >>> >>> -Bryan >>> >>> On Tue, Jun 28, 2016 at 5:32 AM, Asanka Sanjaya Herath < >>> [email protected]> wrote: >>> >>>> Hi Bryan, >>>> >>>> Your suggestion was worked fine. I have another question. This is not >>>> related to the subject, but it is related to PutMongo processor. How can I >>>> use put mongo processor to add a new key value pair to an existing >>>> document? The flow file contains the document object Id. I have set 'mode' >>>> property to 'update' and 'upsert' property to false and 'update query key' >>>> property to '_id'. Flow file content is something like this. >>>> >>>> { >>>> _id:ObjectId(577216f0154b943fe8068079) >>>> expired:true >>>> } >>>> >>>> Without inserting the 'expired:true', it replaces the whole document >>>> with the given one. So is there a way to insert the new key value pair to >>>> collection without replacing the whole collection in MongoDB using putmongo >>>> processor? Your concern regarding this is highly appreciated. >>>> >>>> >>>> >>>> On Mon, Jun 27, 2016 at 6:43 PM, Asanka Sanjaya Herath < >>>> [email protected]> wrote: >>>> >>>>> Hi Bryan, >>>>> Thank you for the input. That really helps. I'll try that. >>>>> >>>>> On Mon, Jun 27, 2016 at 6:31 PM, Bryan Bende <[email protected]> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> Right now AttributesToJson does treat everything as strings. I >>>>>> remember a previous discussion about adding support for different types, >>>>>> but I can't find a JIRA that references this. >>>>>> >>>>>> One option to work around this could be to use ReplaceText to >>>>>> construct the JSON, instead of AttributesToJson. You could set the >>>>>> Replacement Value property to something like: >>>>>> >>>>>> { >>>>>> "dataSourceId" : "${datasource}", >>>>>> "filename" : "${filename}", >>>>>> "sent_date" : ${sent_date}, >>>>>> "uuid" : "${uuid}", >>>>>> "originalSource" : "${originalsource}" >>>>>> } >>>>>> >>>>>> Of course using the appropriate attribute names. >>>>>> >>>>>> Another option is that in the upcoming 0.7.0 release, there is a new >>>>>> processor to transform JSON using JOLT. With that processor you may be >>>>>> able >>>>>> to take the output of AttributesToJson and apply a transform that >>>>>> converts >>>>>> the date field to remove the quotes. >>>>>> >>>>>> Hope that helps. >>>>>> >>>>>> -Bryan >>>>>> >>>>>> On Mon, Jun 27, 2016 at 8:16 AM, Asanka Sanjaya Herath < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> I'm trying to insert a flow file to MongoDb which has a date record >>>>>>> as an attribute. First I sent that flow file through an attribute to >>>>>>> JSON >>>>>>> processor, so that all attributes are now converted to a Json document >>>>>>> in >>>>>>> flow file body. When I insert that flow file to Mongodb using PutMongo >>>>>>> processor, it saves the "sent_date" attribute as a String. I want this >>>>>>> to >>>>>>> be saved as an ISO date object. >>>>>>> >>>>>>> My flow file looked like this. >>>>>>> >>>>>>> { >>>>>>> "dataSourceId" : "xxxx", >>>>>>> "filename" : "979f7bc5-a395-4396-9625-69fdb2c806c6", >>>>>>> "sent_date" : "Mon Jan 18 04:50:50 IST 2016", >>>>>>> "uuid" : "77a5ef56-8b23-40ee-93b5-78c6323e0e1c", >>>>>>> "originalSource" : "ImportedZip" >>>>>>> } >>>>>>> >>>>>>> Then I prepend "ISODate" to "sent_date" attribute using another >>>>>>> processor. So now my flow file content looks like this. >>>>>>> { >>>>>>> "dataSourceId" : "xxxx", >>>>>>> "filename" : "979f7bc5-a395-4396-9625-69fdb2c806c6", >>>>>>> "sent_date" : "ISODate('Mon Jan 18 04:50:50 IST 2016')", >>>>>>> "uuid" : "77a5ef56-8b23-40ee-93b5-78c6323e0e1c", >>>>>>> "originalSource" : "ImportedZip" >>>>>>> } >>>>>>> >>>>>>> But still It is saved as a string in MongoDB, because of the double >>>>>>> quotation marks. Is there a way to remove those double quotation marks >>>>>>> when >>>>>>> convert using AttributeToJson processor? >>>>>>> >>>>>>> Any help is appreciated. >>>>>>> >>>>>>> -- >>>>>>> Thanks, >>>>>>> Regards, >>>>>>> ASH >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks, >>>>> Regards, >>>>> ASH >>>>> >>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> Regards, >>>> ASH >>>> >>> >>> >> >
