And if your properties are only being used by cpf, then the last step can skip moving over the properties to the prod database, thus saving you those fragments in that database.
-Danny -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Geert Josten Sent: Monday, January 18, 2016 11:38 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] CPF pre-commit action? Hi Will, It doesn¹t have to be that much more complicated. Just add a ¹staging¹ database, put your CPF there, and insert docs into that. Add one extra state and pipeline action at the end of your chain that copies the final docs into a different database, duplicating any properties.. It would become slightly more complicated if there are linked docs that needs to be copied as well, but as long as you keep db uris the same, you don¹t need to touch the contents of any of the docs.. Kind regards, Geert On 1/18/16, 6:33 PM, "[email protected] on behalf of Will Thompson" <[email protected] on behalf of [email protected]> wrote: >Geert, > >I hadn't considered that, but it would definitely achieve the desired >behavior. Thanks for the suggestion. It would add a lot more complexity >to the pipeline, but its sounds feasible. > >-Will > >> On Jan 15, 2016, at 1:38 PM, Geert Josten >><[email protected]> >>wrote: >> >> Have you considered taking an approach similar to Info Studio: insert >>unprocessed docs in an separate database in which you have all the CPF >>pipelines you need. Not until finishing of processing it gets copied >>into the target database. Info Studio was using the Fab database for >>that purpose.. >> >> Cheers >> >> On 1/15/16, 5:06 PM, "[email protected] on >>behalf of Will Thompson" <[email protected] on >>behalf of [email protected]> wrote: >> >>> Geert, >>> >>> We're using CPF because some steps may necessitate human >>>intervention, in which case they go into a queue, get resolved, a >>>state change occurs, and the pipeline marches on. But are you >>>suggesting a pre-commit trigger to supplement the existing CPF >>>pipeline? >>> >>> Danny, >>> >>> The main issue for us is user experience. For example, in most cases >>>document X will already exist in the database and users can see it as >>>part of the application. We want to replace document X with an >>>updated version and not have it appear visible to users until it has >>>been transformed. Hiding it in a collection would hide it from users >>>as well. >>> I said "version" so maybe DLS seems like a good fit? But I'm still >>>unsure of how to begin the pipeline with the document in a >>>checked-out state. >>> >>> We do have a solution to this, which is essentially: the application >>> takes care of it. But I wanted to see if it were possible to push >>> that logic back into CPF and simplify our API such that other >>> applications could utilize the pipeline with basic insert and delete >>> operations. >>> >>> Thanks, >>> >>> Will >>> >>> >>>> On Jan 15, 2016, at 5:26 AM, Geert Josten >>>> <[email protected]> >>>> wrote: >>>> >>>> Or how about KISS: just use one pre-commit trigger that does all >>>> the transformation/processing in one blow.. >>>> >>>> Kind regards, >>>> Geert >>>> >>>> On 1/15/16, 12:42 AM, "[email protected] on >>>>behalf of Danny Sokolsky" <[email protected] >>>>on behalf of [email protected]> wrote: >>>> >>>>> Hi Will, >>>>> >>>>> I can't think of a way to do this without modifying the core cpf >>>>>code. >>>>> For some of the status-change-handling pipeline steps, there are >>>>>pre-commit triggers, but those are for cpf bookkeeping mostly, I >>>>>believe. >>>>> And I would not recommend changing that code. >>>>> >>>>> But I would question why you would want to do this. CPF is >>>>>designed to be resilient and to allow you to have multiple steps >>>>>in your pipeline. >>>>> Why not make a step that does the transform its own step? If you >>>>>do not want the document to be visible until it is transformed, >>>>>then put it in some collection initially that is not seen by your >>>>>application, and then take it out of that collection when you are >>>>>done with the transform (or something similar to that). >>>>> >>>>> Is it just because you want to have one less step in your pipeline >>>>>that you want to do this? >>>>> >>>>> -Danny >>>>> >>>>> -----Original Message----- >>>>> From: [email protected] >>>>> [mailto:[email protected]] On Behalf Of Will >>>>> Thompson >>>>> Sent: Thursday, January 14, 2016 1:06 PM >>>>> To: MarkLogic Developer Discussion >>>>> Subject: [MarkLogic Dev General] CPF pre-commit action? >>>>> >>>>> Is it possible to configure a CPF pipeline such that when a >>>>>document is inserted, transformations are first executed on the >>>>>document in a pre-commit stage, and if those complete >>>>>successfully, then the transformation result is what's finally >>>>>committed at that URI? And any remaining pipeline could carry on >>>>>without the pre-transformed data ever being visible to the >>>>>database? >>>>> >>>>> -Will >>>>> _______________________________________________ >>>>> General mailing list >>>>> [email protected] >>>>> Manage your subscription at: >>>>> http://developer.marklogic.com/mailman/listinfo/general >>>>> _______________________________________________ >>>>> General mailing list >>>>> [email protected] >>>>> Manage your subscription at: >>>>> http://developer.marklogic.com/mailman/listinfo/general >>>> >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> Manage your subscription at: >>>> http://developer.marklogic.com/mailman/listinfo/general >>> >>> _______________________________________________ >>> General mailing list >>> [email protected] >>> Manage your subscription at: >>> http://developer.marklogic.com/mailman/listinfo/general >> >> _______________________________________________ >> General mailing list >> [email protected] >> Manage your subscription at: >> http://developer.marklogic.com/mailman/listinfo/general > >_______________________________________________ >General mailing list >[email protected] >Manage your subscription at: >http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
