That doesn't sound too bad. Thanks, guys. -Will
> On Jan 18, 2016, at 1:46 PM, Danny Sokolsky <[email protected]> > wrote: > > And if your properties are only being used by cpf, then the last step can > skip moving over the properties to the prod database, thus saving you those > fragments in that database. > > -Danny > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Geert Josten > Sent: Monday, January 18, 2016 11:38 AM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] CPF pre-commit action? > > Hi Will, > > It doesn¹t have to be that much more complicated. Just add a ¹staging¹ > database, put your CPF there, and insert docs into that. Add one extra state > and pipeline action at the end of your chain that copies the final docs into > a different database, duplicating any properties.. > > It would become slightly more complicated if there are linked docs that needs > to be copied as well, but as long as you keep db uris the same, you don¹t > need to touch the contents of any of the docs.. > > Kind regards, > Geert > > On 1/18/16, 6:33 PM, "[email protected] on behalf of > Will Thompson" <[email protected] on behalf of > [email protected]> wrote: > >> Geert, >> >> I hadn't considered that, but it would definitely achieve the desired >> behavior. Thanks for the suggestion. It would add a lot more complexity >> to the pipeline, but its sounds feasible. >> >> -Will >> >>> On Jan 15, 2016, at 1:38 PM, Geert Josten >>> <[email protected]> >>> wrote: >>> >>> Have you considered taking an approach similar to Info Studio: insert >>> unprocessed docs in an separate database in which you have all the CPF >>> pipelines you need. Not until finishing of processing it gets copied >>> into the target database. Info Studio was using the Fab database for >>> that purpose.. >>> >>> Cheers >>> >>> On 1/15/16, 5:06 PM, "[email protected] on >>> behalf of Will Thompson" <[email protected] on >>> behalf of [email protected]> wrote: >>> >>>> Geert, >>>> >>>> We're using CPF because some steps may necessitate human >>>> intervention, in which case they go into a queue, get resolved, a >>>> state change occurs, and the pipeline marches on. But are you >>>> suggesting a pre-commit trigger to supplement the existing CPF >>>> pipeline? >>>> >>>> Danny, >>>> >>>> The main issue for us is user experience. For example, in most cases >>>> document X will already exist in the database and users can see it as >>>> part of the application. We want to replace document X with an >>>> updated version and not have it appear visible to users until it has >>>> been transformed. Hiding it in a collection would hide it from users >>>> as well. >>>> I said "version" so maybe DLS seems like a good fit? But I'm still >>>> unsure of how to begin the pipeline with the document in a >>>> checked-out state. >>>> >>>> We do have a solution to this, which is essentially: the application >>>> takes care of it. But I wanted to see if it were possible to push >>>> that logic back into CPF and simplify our API such that other >>>> applications could utilize the pipeline with basic insert and delete >>>> operations. >>>> >>>> Thanks, >>>> >>>> Will >>>> >>>> >>>>> On Jan 15, 2016, at 5:26 AM, Geert Josten >>>>> <[email protected]> >>>>> wrote: >>>>> >>>>> Or how about KISS: just use one pre-commit trigger that does all >>>>> the transformation/processing in one blow.. >>>>> >>>>> Kind regards, >>>>> Geert >>>>> >>>>> On 1/15/16, 12:42 AM, "[email protected] on >>>>> behalf of Danny Sokolsky" <[email protected] >>>>> on behalf of [email protected]> wrote: >>>>> >>>>>> Hi Will, >>>>>> >>>>>> I can't think of a way to do this without modifying the core cpf >>>>>> code. >>>>>> For some of the status-change-handling pipeline steps, there are >>>>>> pre-commit triggers, but those are for cpf bookkeeping mostly, I >>>>>> believe. >>>>>> And I would not recommend changing that code. >>>>>> >>>>>> But I would question why you would want to do this. CPF is >>>>>> designed to be resilient and to allow you to have multiple steps >>>>>> in your pipeline. >>>>>> Why not make a step that does the transform its own step? If you >>>>>> do not want the document to be visible until it is transformed, >>>>>> then put it in some collection initially that is not seen by your >>>>>> application, and then take it out of that collection when you are >>>>>> done with the transform (or something similar to that). >>>>>> >>>>>> Is it just because you want to have one less step in your pipeline >>>>>> that you want to do this? >>>>>> >>>>>> -Danny >>>>>> >>>>>> -----Original Message----- >>>>>> From: [email protected] >>>>>> [mailto:[email protected]] On Behalf Of Will >>>>>> Thompson >>>>>> Sent: Thursday, January 14, 2016 1:06 PM >>>>>> To: MarkLogic Developer Discussion >>>>>> Subject: [MarkLogic Dev General] CPF pre-commit action? >>>>>> >>>>>> Is it possible to configure a CPF pipeline such that when a >>>>>> document is inserted, transformations are first executed on the >>>>>> document in a pre-commit stage, and if those complete >>>>>> successfully, then the transformation result is what's finally >>>>>> committed at that URI? And any remaining pipeline could carry on >>>>>> without the pre-transformed data ever being visible to the >>>>>> database? >>>>>> >>>>>> -Will >>>>>> _______________________________________________ >>>>>> General mailing list >>>>>> [email protected] >>>>>> Manage your subscription at: >>>>>> http://developer.marklogic.com/mailman/listinfo/general >>>>>> _______________________________________________ >>>>>> General mailing list >>>>>> [email protected] >>>>>> Manage your subscription at: >>>>>> http://developer.marklogic.com/mailman/listinfo/general >>>>> >>>>> _______________________________________________ >>>>> General mailing list >>>>> [email protected] >>>>> Manage your subscription at: >>>>> http://developer.marklogic.com/mailman/listinfo/general >>>> >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> Manage your subscription at: >>>> http://developer.marklogic.com/mailman/listinfo/general >>> >>> _______________________________________________ >>> General mailing list >>> [email protected] >>> Manage your subscription at: >>> http://developer.marklogic.com/mailman/listinfo/general >> >> _______________________________________________ >> General mailing list >> [email protected] >> Manage your subscription at: >> http://developer.marklogic.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > [email protected] > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ > General mailing list > [email protected] > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
