Geert, I hadn't considered that, but it would definitely achieve the desired behavior. Thanks for the suggestion. It would add a lot more complexity to the pipeline, but its sounds feasible.
-Will > On Jan 15, 2016, at 1:38 PM, Geert Josten <[email protected]> wrote: > > Have you considered taking an approach similar to Info Studio: insert > unprocessed docs in an separate database in which you have all the CPF > pipelines you need. Not until finishing of processing it gets copied into > the target database. Info Studio was using the Fab database for that > purpose.. > > Cheers > > On 1/15/16, 5:06 PM, "[email protected] on behalf of > Will Thompson" <[email protected] on behalf of > [email protected]> wrote: > >> Geert, >> >> We're using CPF because some steps may necessitate human intervention, in >> which case they go into a queue, get resolved, a state change occurs, and >> the pipeline marches on. But are you suggesting a pre-commit trigger to >> supplement the existing CPF pipeline? >> >> Danny, >> >> The main issue for us is user experience. For example, in most cases >> document X will already exist in the database and users can see it as >> part of the application. We want to replace document X with an updated >> version and not have it appear visible to users until it has been >> transformed. Hiding it in a collection would hide it from users as well. >> I said "version" so maybe DLS seems like a good fit? But I'm still unsure >> of how to begin the pipeline with the document in a checked-out state. >> >> We do have a solution to this, which is essentially: the application >> takes care of it. But I wanted to see if it were possible to push that >> logic back into CPF and simplify our API such that other applications >> could utilize the pipeline with basic insert and delete operations. >> >> Thanks, >> >> Will >> >> >>> On Jan 15, 2016, at 5:26 AM, Geert Josten <[email protected]> >>> wrote: >>> >>> Or how about KISS: just use one pre-commit trigger that does all the >>> transformation/processing in one blow.. >>> >>> Kind regards, >>> Geert >>> >>> On 1/15/16, 12:42 AM, "[email protected] on behalf >>> of Danny Sokolsky" <[email protected] on behalf of >>> [email protected]> wrote: >>> >>>> Hi Will, >>>> >>>> I can't think of a way to do this without modifying the core cpf code. >>>> For some of the status-change-handling pipeline steps, there are >>>> pre-commit triggers, but those are for cpf bookkeeping mostly, I >>>> believe. >>>> And I would not recommend changing that code. >>>> >>>> But I would question why you would want to do this. CPF is designed to >>>> be resilient and to allow you to have multiple steps in your pipeline. >>>> Why not make a step that does the transform its own step? If you do >>>> not >>>> want the document to be visible until it is transformed, then put it in >>>> some collection initially that is not seen by your application, and >>>> then >>>> take it out of that collection when you are done with the transform (or >>>> something similar to that). >>>> >>>> Is it just because you want to have one less step in your pipeline that >>>> you want to do this? >>>> >>>> -Danny >>>> >>>> -----Original Message----- >>>> From: [email protected] >>>> [mailto:[email protected]] On Behalf Of Will >>>> Thompson >>>> Sent: Thursday, January 14, 2016 1:06 PM >>>> To: MarkLogic Developer Discussion >>>> Subject: [MarkLogic Dev General] CPF pre-commit action? >>>> >>>> Is it possible to configure a CPF pipeline such that when a document is >>>> inserted, transformations are first executed on the document in a >>>> pre-commit stage, and if those complete successfully, then the >>>> transformation result is what's finally committed at that URI? And any >>>> remaining pipeline could carry on without the pre-transformed data ever >>>> being visible to the database? >>>> >>>> -Will >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> Manage your subscription at: >>>> http://developer.marklogic.com/mailman/listinfo/general >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> Manage your subscription at: >>>> http://developer.marklogic.com/mailman/listinfo/general >>> >>> _______________________________________________ >>> General mailing list >>> [email protected] >>> Manage your subscription at: >>> http://developer.marklogic.com/mailman/listinfo/general >> >> _______________________________________________ >> General mailing list >> [email protected] >> Manage your subscription at: >> http://developer.marklogic.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > [email protected] > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
