Ok, let me try to step back 1 sec, because I assumed we were in agreement that tracking which workflow definitions are actively deployed from the user point of view is a useful feature. So before going back to implementation, let's make sure we're talking about the same. So to be very specific, we offer a management console (we have different variations, but this screenshot is one for managing serverless workflows across several independent workflow runtimes), where workflows can be started. Being able to only show those workflows that are still active would be a useful feature, can we agree on that?
[image: image.png] Thx, Kris On Fri, Aug 9, 2024 at 6:06 PM Enrique Gonzalez Martinez < [email protected]> wrote: > Hi Kris, > > El vie, 9 ago 2024 a las 16:25, Kris Verlaenen > (<[email protected]>) escribió: > > > > Regarding the information required, the idea right now is to keep it very > > limited currently, a simple status for each ProcessDefinition would > already > > enable a lot. I can imagine though that, if we want to potentially log a > > bit more information for audit purposes - e.g. when a workflow was > deployed > > / undeployed or who requested it - we could look into more complex data > > structures to augment this, but I don't think this is currently required. > > Hi Kris, This idea is mixing concepts and responsibilities and keep > thinking that we act as a server when we are not anymore. Kogito does > not deploy anything anymore. We are not servers anymore. At the moment > we shift from being a server to being a microservice we don't offer a > platform, so it is outside our control who deploys anything. That is > the platform mission. That layer of abstraction is not our > responsibility anymore and trying to catch things. Activity logs, > when, where something is deployed is outside our control. The platform > does offer this sort of thing so we are going to try to recreate this > tool again. > > > Note that I agree we shouldn't be storing any kind of external > information, > > but this information is really key to the core platform behavior, so I > > think we should see to what degree the platform can help expose this. > But > > it would be an optional feature. > > We have already stored information in the data index that makes the > system very fragile (e.g: url from the runtime). This is going to > cause huge problems from the start once we go GA downstream. It makes > the system very complex to sustain because it stores information from > the platform and therefore ties the "devops" role to the intimate > knowledge of our runtime. This is a no go. The goal of any > microservice should strive to be able to deploy without any inner > knowledge just which are the other dependencies from microservices. > > > I agree that for retrieving data like this by users we typically have 2 > > options: > > - retrieve it from some runtime by looking it up from the single source > of > > truth > > - retrieve it from the data index that is caching the information > > The only source of truth we have is the platform, which is the one > containing all the information it is required. > > > I believe we have traditionally exposed data in the data index for > several > > reasons: > > 1) the information is potentially distributed or difficult to aggregate, > so > > the data index aggregates it and it can be more easily queried there > > This is just not true and it is ignoring the fact we mentioned before. > Aggregating data outside the platform is nearly impossible to do it > right. Load balancers, ip proxies, etc... and your counter will go > bananas easily. And we are talking about counting. If you lose events > in the stream your counting will be bad (and that is considering > somebody is not killing a container and so forth) > > > 2) the information is cached in the data index to avoid introducing load > on > > the runtime systems for common avoidable queries > > I do agree that different databases will unload the runtime database, > that is consistent (even if another proposal like transactions was > suggested to use a unique data source to avoid transactions). > It is not consistent though to suggest that this will be optional and > embedded this feature in the data index. (see your starting paragraph > here) > It is not very performant either. Keep in mind that the data index > already has a performance issue because it requires updating rows > causing row locks (or even table locks) > > > 3) the information in the runtime is limited and disappears at the end of > > the runtime life cycle, where it can exist for longer in the data index > > It is how it is supposed to work. But taking this as an argument to > embed this data in the data index has no logic as we agreed several > times already that the data index stores the last state of the process > data to make it easier to query the process status. So runtime > platform data does not fit that responsibility. > > > 4) data index abstracts the underlying complexity by offering an easy > > graphql view on the data > > You can create a small app to abstract the platform in a graphql manner. > > > Imho information about the status of a workflow definition satisfies a > lot > > of these reasons. We also have to take into account that retrieving the > > list of potential active definitions is a request that could be a lot > more > > common (everytime a user opens a view listing potential active workflows) > > vs changing the deployment status of a workflow, which could still be > > frequent but in general is typically a lot less frequent. Being able to > > update the status of the workflow definition in the data index so it can > > cache this information will avoid delegating this to the kubernetes > system > > every time instead. > > That is the mission of the platform. To control/monitor/query/audit > the deployments. > > > And while the information is limited for now (status > > only), I can imagine we might want to extend it in the future to include > > more information, information that is not going to be available in the > > runtime (especially once undeployed). It could also work as an > > abstraction, for example in architectures where you might not be able to > > query kubernetes for this, but I'll ignore this for now as this is really > > an edge case. > > Mentioning some data that we might like to have in the future makes it > really impossible for the discussion moving forward. It is not > possible to discuss that way. If you do plan to have additional data > that does not fit the idea of runtime platform data, please mention > it. Because as far as I can see it the platform is going to do this > job much better than any tool we can come up with. In the same way > regarding not being able to query the platform (in this case you > mentioned kubernetes). You should mention what are the use cases like > any other proposal. Mentioning about some possible future without any > specifics is not a valid argument because there is no way we can > discuss an unknown use case. > > I think the problem that we are trying to solve in this proposal is > how to monitor/query/audit which containers and process definitions > are being deployed. > Everything in there is provided by the platform but the process > definitions (but there is a mechanism in the containers to provide > such data) > If the problem is that the user cannot query that data to the platform > then I would suggest to create a small component to query directly > such data to the platform like openshift. Creating a client against > openshift is fairly easy and you don't need to know who the platform > counts or anything. > > Just to brief my position in this matter: > > 1. Let's try avoiding reinventing the wheel and reimplement things > that are provided already by the underlying platform. > 2. So far we are discussing runtime platform data (which process > definitions are being deployed, how many, audit of those operations). > All this is provided by the platform like it should. In the case of > process definition we can give such information to the platform > without > 3. There is no strong argument to change the responsibility of the > data index from storing the last state of the process state to storing > as well the runtime platform data. (Let's keep the single > responsibility principle here untouch) > 4. There is no strong argument to try to capture platform runtime data > (like who is deploying a container). I am strongly against starting to > build a driver to every potential platform a user can use in order to > retrieve such information. It is not our mission. > 5. The complexity of trying to figure out the underlying platform is > pretty big. > 6. If this is a heavy requirement, The only constrain I would set is > to do this in a separate component, avoiding mixing this data and any > other logic regarding the platform neither in runtime nor data index > (or any existing already well define component) > > > > > Thx, > > Kris > > > > On Thu, Jul 18, 2024 at 1:59 PM Enrique Gonzalez Martinez < > > [email protected]> wrote: > > > > > I would split this into 3 different problems > > > > > > 1. Information structure required to be there (Info and how to map > that in > > > LABEL for instance) > > > > > > 2. Check how to query the platform (docker inspect). > > > > > > 3. Wrap that into a tool that can be used. > > > > > > I will let jump Alex to jump into the 3 because i dont really have a > strong > > > opinion about it. I will ask how do you plan to do this. > > > > > > Are you trying to extend openshift or similar ? > > > Are we building small UI to query the platform ? > > > > > > El jue, 18 jul 2024, 13:09, Francisco Javier Tirado Sarti < > > > [email protected]> escribió: > > > > > > > Ok, good to know we share the same goal because I also think we > should go > > > > for 1. > > > > So, let's discuss how in a summarized way. The proposal Walter sent > was > > > > trying to achieve that by adding a new parameter, named > deploymentStatus, > > > > to the ProcessDefinition entity already existing in DataIndex. > > > > Another possibility that comes to my mind is a new deployment > component > > > (a > > > > deployment view of the process definition) different from DataIndex. > > > > Any other suggestions/ideas are welcome ;) > > > > > > > > > > > > On Thu, Jul 18, 2024 at 10:28 AM Enrique Gonzalez Martinez < > > > > [email protected]> wrote: > > > > > > > > > 1. Let the platform to do the work giving the information to the > > > platform > > > > > required and let use user to query that info. This is our preferred > > > way. > > > > We > > > > > just need to build a way to provide that information and the > platform > > > has > > > > > its own tools to query metadata from the deployment. We just need a > > > > common > > > > > ground to define the info structure. > > > > > > > > > > 2. Write our own tool capturing events from the deployments > whenever > > > > > deploys or not. This requires more effort as you will need to > clutter > > > the > > > > > data index and mix responsibilities in there > > > > > > > > > > The option that from my point of view is the first one. > > > > > > > > > > The advantages are clear imo > > > > > 1. Platform requires a different skill set and tools platform > > > dependent. > > > > > Moving this to our system will mean to try to solve a buch of > stuff in > > > an > > > > > independent way making things really hard and making devops needed > to > > > > learn > > > > > new stuff and not being able to use their tooling. That is and > > > additional > > > > > operation cost > > > > > 2. It is easier to provide a driver to set metadata in the > deployment > > > > than > > > > > creating custom drivers to solve properly events in a platform > > > constantly > > > > > changing > > > > > 3. Service url is a fragile approach in a cloud environment. It is > too > > > > > error prone and couple microservices to the deployment environment > > > > > > > > > > > > > > > El jue, 18 jul 2024, 10:18, Enrique Gonzalez Martinez < > > > > > [email protected]> > > > > > escribió: > > > > > > > > > > > Hi Francisco, > > > > > > > > > > > > Just to clarify. We are not against having some sort of way to > tell > > > > which > > > > > > process definitions are deployed or not. But you need to keep in > mind > > > > > that > > > > > > there are two different ways to do this: > > > > > > > > > > > > El jue, 18 jul 2024, 10:01, Francisco Javier Tirado Sarti < > > > > > > [email protected]> escribió: > > > > > > > > > > > >> Yes, I understand that your preference is to delegate deployment > > > > status > > > > > to > > > > > >> an external system. > > > > > >> However I would like to explore the consequences of that > approach > > > and > > > > > how > > > > > >> things are supposed to work. The first thing that come to mind > is > > > that > > > > > the > > > > > >> user of the platform will be forced to have two different > graphical > > > > > tools > > > > > >> for tracking: one for process definition deployment (the one > that > > > will > > > > > >> include the deployment status and will communicate with the user > > > > custom > > > > > >> db) and the one provided by the platform for process definition > > > > > instances > > > > > >> (current runtime tooling communicating with DI) > > > > > >> Are we sure we are fine with this architecture? > > > > > >> > > > > > >> On Wed, Jul 17, 2024 at 7:48 PM Alex Porcelli <[email protected] > > > > > > wrote: > > > > > >> > > > > > >> > Francisco, > > > > > >> > > > > > > >> > Seems that a few of us believe this would be better addressed > by > > > an > > > > > >> > external system. > > > > > >> > > > > > > >> > > > > > > >> > On Wed, Jul 17, 2024 at 7:14 AM Francisco Javier Tirado Sarti > < > > > > > >> > [email protected]> wrote: > > > > > >> > > > > > > >> > > Hi, > > > > > >> > > Yes, process definition is used because runtime tooling is > > > relying > > > > > on > > > > > >> > > process definition to be able to render a graphical > > > representation > > > > > of > > > > > >> the > > > > > >> > > runtime progress. > > > > > >> > > It can be argued that a runtime tool might want to know in > which > > > > > >> servers > > > > > >> > of > > > > > >> > > the cluster that process definition is deployed and the > status > > > of > > > > > that > > > > > >> > > deployment. So, the same serviceURL is there, a deployment > > > status > > > > > >> flag is > > > > > >> > > not a crazy idea. > > > > > >> > > > > > > > >> > > On Thu, Jul 11, 2024 at 6:45 PM Alex Porcelli < > [email protected] > > > > > > > > > >> wrote: > > > > > >> > > > > > > > >> > > > Francisco, > > > > > >> > > > > > > > > >> > > > You're spot on on the purpose of the current Process > > > Definition > > > > on > > > > > >> DI: > > > > > >> > > > it provides complementary information to process > instances. > > > > > >> > > > > > > > > >> > > > Example? Process definition (ie. diagram) for the current > > > > > instance. > > > > > >> > > > > > > > > >> > > > On Thu, Jul 11, 2024 at 12:48 PM Francisco Javier Tirado > Sarti > > > > > >> > > > <[email protected]> wrote: > > > > > >> > > > > > > > > > >> > > > > Hi, > > > > > >> > > > > Since DI already has a ProcessDefinition entity with a > > > service > > > > > >> URL, > > > > > >> > we > > > > > >> > > > feel > > > > > >> > > > > that adding deployment status was not breaking anything > > > that > > > > > was > > > > > >> not > > > > > >> > > > > already broken. > > > > > >> > > > > I guess it all depends on what we think should be the > > > purpose > > > > of > > > > > >> DI. > > > > > >> > > > > If it is just to offer information about process > instances, > > > > > >> Enrique > > > > > >> > is > > > > > >> > > > > right, this is not the place to monitor which process > > > > > definitions > > > > > >> are > > > > > >> > > > > available or not. > > > > > >> > > > > However, if the purpose of DI is just that, why is the > > > process > > > > > >> > > definition > > > > > >> > > > > entity there? as supporting information for the process > > > > instance > > > > > >> > only? > > > > > >> > > > > I feel we need to clearly define this before discarding > > > > Walters' > > > > > >> > > > proposal. > > > > > >> > > > > If we take a strict approach, then it means that any > > > software > > > > > >> > deploying > > > > > >> > > > > runtimes and data index will have to develop its own > > > > deployment > > > > > >> > > component > > > > > >> > > > > with its own db. That's probably fair, but should be > written > > > > > >> > somewhere. > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > On Tue, Jul 9, 2024 at 3:39 PM Alex Porcelli < > > > > [email protected]> > > > > > >> > wrote: > > > > > >> > > > > > > > > > >> > > > > > Walter, > > > > > >> > > > > > > > > > > >> > > > > > I have to agree with Enrique… although your ask is > minimal > > > > in > > > > > >> the > > > > > >> > > code > > > > > >> > > > > > changes and impact, the problem is more on the message > > > that > > > > it > > > > > >> > sends. > > > > > >> > > > > > > > > > > >> > > > > > As pointed out by last email from Enrique, we - as > Apache > > > > KIE > > > > > - > > > > > >> > don’t > > > > > >> > > > > > control or have visibility over the whole > environment, so > > > > it’s > > > > > >> > > > virtually > > > > > >> > > > > > impossible to guarantee that something is set to not > > > > available > > > > > >> is > > > > > >> > > > really > > > > > >> > > > > > not available, and vice versa. > > > > > >> > > > > > > > > > > >> > > > > > I understand that the serverless operator aims to > provide > > > an > > > > > >> > > > experience, > > > > > >> > > > > > but it also has limitation over the installation… so > > > > probably > > > > > >> this > > > > > >> > > > kind of > > > > > >> > > > > > information would be better stored outside the Apache > KIE > > > > > >> systems / > > > > > >> > > in > > > > > >> > > > your > > > > > >> > > > > > case, in the downstream product database. > > > > > >> > > > > > > > > > > >> > > > > > Regards, > > > > > >> > > > > > _____________ > > > > > >> > > > > > Alex Porcelli > > > > > >> > > > > > http://porcelli.me > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > On Tue, Jul 9, 2024 at 1:59 AM Enrique Gonzalez > Martinez < > > > > > >> > > > > > [email protected]> wrote: > > > > > >> > > > > > > > > > > >> > > > > > > Hi Walter, > > > > > >> > > > > > > > > > > > >> > > > > > > I see some problems aside from going on in mixing > the > > > data > > > > > >> index > > > > > >> > > with > > > > > >> > > > > > data > > > > > >> > > > > > > does not belong to it like deployment mgmt. E.g. The > > > > service > > > > > >> url > > > > > >> > is > > > > > >> > > > there > > > > > >> > > > > > > not because the DI but the gateway component > (mutations) > > > > > that > > > > > >> > makes > > > > > >> > > > > > changes > > > > > >> > > > > > > in runtimes. Something that needs to change in the > > > future. > > > > > >> But i > > > > > >> > > > digress. > > > > > >> > > > > > > > > > > > >> > > > > > > Back to this is deployment management. Basically the > > > > > >> requirement > > > > > >> > is > > > > > >> > > > to > > > > > >> > > > > > > compute the current catalog of process definitions > > > > deployed > > > > > in > > > > > >> > the > > > > > >> > > > > > > environment. > > > > > >> > > > > > > > > > > > >> > > > > > > I am not against but I am not comfortable mixing > with > > > the > > > > > data > > > > > >> > > index. > > > > > >> > > > > > > Because it is not part of the data index goal; > provide > > > the > > > > > >> last > > > > > >> > > state > > > > > >> > > > > > > information of the process instances. > > > > > >> > > > > > > > > > > > >> > > > > > > Regarding how this approach works and given you are > > > > capable > > > > > of > > > > > >> > > > counting > > > > > >> > > > > > > properly instances. > > > > > >> > > > > > > > > > > > >> > > > > > > 1. Let's say you bring down one workflow. The system > > > will > > > > be > > > > > >> > marked > > > > > >> > > > as > > > > > >> > > > > > > undeployed/deleted. You don't specify how alive > process > > > > > >> instance > > > > > >> > > > will be > > > > > >> > > > > > > processed or if you will perform some other action. > > > > > >> > > > > > > The other problem i see is trying to already solve > > > > platform > > > > > >> > > > problems. I > > > > > >> > > > > > > mean, if the platform already can compute which > > > containers > > > > > are > > > > > >> > > > deployed > > > > > >> > > > > > and > > > > > >> > > > > > > therefore which is the proper catalog why should we > > > polute > > > > > the > > > > > >> > > system > > > > > >> > > > > > with > > > > > >> > > > > > > this ? Would not make more sense in telling the > user how > > > > to > > > > > >> query > > > > > >> > > > that > > > > > >> > > > > > info > > > > > >> > > > > > > in the platform ? Making platform independent does > not > > > > give > > > > > a > > > > > >> > > proper > > > > > >> > > > > > > benefits of chasing multiple platform problems > regarding > > > > > >> > > deployment / > > > > > >> > > > > > > undeployment operations, url changes, counting > > > > instances... > > > > > >> Will > > > > > >> > > > never > > > > > >> > > > > > get > > > > > >> > > > > > > it right. It is an unsustainable approach. > > > > > >> > > > > > > > > > > > >> > > > > > > El lun, 8 jul 2024, 11:18, Walter Medvedeo < > > > > > >> [email protected] > > > > > >> > > > > > > > >> > > > > > > escribió: > > > > > >> > > > > > > > > > > > >> > > > > > > > Hi Enrique, the requirement we have from > SonataFlow > > > > users > > > > > is > > > > > >> > very > > > > > >> > > > > > simple. > > > > > >> > > > > > > > > > > > > >> > > > > > > > Imagine the ProcesssDefinitions in the DI as a > catalog > > > > of > > > > > >> all > > > > > >> > the > > > > > >> > > > > > > existing > > > > > >> > > > > > > > definitions. > > > > > >> > > > > > > > > > > > > >> > > > > > > > Similar to the sonataflow-console, the > jbpm-console, > > > > etc, > > > > > >> user > > > > > >> > > > > > > > applications query the ProcessDefintions to get > the > > > list > > > > > >> with > > > > > >> > > all > > > > > >> > > > the > > > > > >> > > > > > > > existing definitions, and present that list for > > > example > > > > > in a > > > > > >> > UI. > > > > > >> > > > > > > > > > > > > >> > > > > > > > So, imagine that with the ProcessDefinitions > query you > > > > get > > > > > >> the > > > > > >> > > > > > following > > > > > >> > > > > > > > list: > > > > > >> > > > > > > > > > > > > >> > > > > > > > purchase-order, serviceUrl_1 > > > > > >> > > > > > > > new-customer, serviceUrl_2 > > > > > >> > > > > > > > etc. > > > > > >> > > > > > > > > > > > > >> > > > > > > > Similar to what we do in our consoles, in their > UIs, > > > > users > > > > > >> can > > > > > >> > > > pick any > > > > > >> > > > > > > > process from that list to create new instances, > etc. > > > > > >> > > > > > > > > > > > > >> > > > > > > > All good. > > > > > >> > > > > > > > > > > > > >> > > > > > > > In a later point in time, an administrator, > decides > > > that > > > > > the > > > > > >> > > > > > > > "new-customer" process is out from the catalog. We > > > won't > > > > > >> work > > > > > >> > > with > > > > > >> > > > that > > > > > >> > > > > > > > ProcessDefinition anymore. > > > > > >> > > > > > > > > > > > > >> > > > > > > > Very simplified, In SonataFlow Operator driven > setups, > > > > you > > > > > >> can > > > > > >> > do > > > > > >> > > > it by > > > > > >> > > > > > > > telling the operator to "un-deploy" the > worfklow. As > > > > part > > > > > >> of > > > > > >> > > this > > > > > >> > > > > > > > procedure, the operator can send a basic event to > the > > > > data > > > > > >> > index > > > > > >> > > > > > telling > > > > > >> > > > > > > > that situation. > > > > > >> > > > > > > > > > > > > >> > > > > > > > So, at the data index, we need to mark the > > > > "new-customer" > > > > > >> > process > > > > > >> > > > > > > > definition as "deleted", or better said as > "logically > > > > > >> deleted". > > > > > >> > > > > > (Because, > > > > > >> > > > > > > > as I explained in previous comments, we don't > want, at > > > > > >> least by > > > > > >> > > > now, do > > > > > >> > > > > > > > physical deletions in the DI) > > > > > >> > > > > > > > > > > > > >> > > > > > > > To do that, we need to add a "new field". > (Simplest > > > > > >> approach, > > > > > >> > we > > > > > >> > > > don't > > > > > >> > > > > > > > want more elaborated modelings) > > > > > >> > > > > > > > > > > > > >> > > > > > > > I suggested to name that field as "status", to > keep it > > > > > >> > completely > > > > > >> > > > > > > > unrelated from the type of setup/deployment, > because, > > > > as I > > > > > >> > > > mentioned, > > > > > >> > > > > > > > kogito supports a plenty of setups, docker, docker > > > > > compose, > > > > > >> > > > Sonataflow > > > > > >> > > > > > > > operator driven, standalone JVM, etc. > > > > > >> > > > > > > > > > > > > >> > > > > > > > So, introduced that we could for example mark > > > "status" = > > > > > >> > > > "unavailable", > > > > > >> > > > > > > > and maybe this introduced confusion. If that is > the > > > > case, > > > > > >> > forget > > > > > >> > > > about > > > > > >> > > > > > > the > > > > > >> > > > > > > > world unavailable, and to follow the reasoning, > > > imagine > > > > we > > > > > >> mark > > > > > >> > > > that > > > > > >> > > > > > > field > > > > > >> > > > > > > > with "status" = "deleted" (logical deletion) > > > > > >> > > > > > > > > > > > > >> > > > > > > > Finally, by adding this simple field/concept, > totally > > > > > >> decoupled > > > > > >> > > > from > > > > > >> > > > > > the > > > > > >> > > > > > > > various setups that people can use, we can > facilitate > > > > > >> > SonataFlow > > > > > >> > > > users > > > > > >> > > > > > > > applications to query the DI, and only get the > > > > > >> > ProcessDefinitons > > > > > >> > > > that > > > > > >> > > > > > are > > > > > >> > > > > > > > not "deleted" (logically deleted) > > > > > >> > > > > > > > > > > > > >> > > > > > > > Select * from ProcessDefinitions where status != > > > deleted > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > Regards, > > > > > >> > > > > > > > Walter. > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > On 2024/07/04 11:35:08 Enrique Gonzalez Martinez > > > wrote: > > > > > >> > > > > > > > > Hi walter, > > > > > >> > > > > > > > > I am not certain about the req. You have few > things > > > in > > > > > >> motion > > > > > >> > > in > > > > > >> > > > here > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > 1. The req says the deployment is still > required : > > > > Then > > > > > >> you > > > > > >> > > have > > > > > >> > > > to > > > > > >> > > > > > do > > > > > >> > > > > > > > > nothing as this information is just available > in the > > > > > >> index. > > > > > >> > > > Query the > > > > > >> > > > > > > > > active process instance of a certain procces id > and > > > > > >> version. > > > > > >> > > You > > > > > >> > > > > > dont > > > > > >> > > > > > > > need > > > > > >> > > > > > > > > to access the process definition. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > The req says you need to know if the deployment > is > > > > > >> required > > > > > >> > but > > > > > >> > > > you > > > > > >> > > > > > did > > > > > >> > > > > > > > not > > > > > >> > > > > > > > > deploy data index. Then you need to check if > there > > > > > pricess > > > > > >> > > > instance > > > > > >> > > > > > > > > belonging to that deployment. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > The req says you need to know if a deployment is > > > > > >> > > > > > unavailable/available > > > > > >> > > > > > > i > > > > > >> > > > > > > > > would say you need to send events related to > that > > > > > >> deployment > > > > > >> > > and > > > > > >> > > > this > > > > > >> > > > > > > is > > > > > >> > > > > > > > > more tricky. Because you need to somehow define > > > first > > > > > >> what is > > > > > >> > > > > > > available. > > > > > >> > > > > > > > It > > > > > >> > > > > > > > > seems your definition is related to what is > > > deployed. > > > > In > > > > > >> that > > > > > >> > > > sense i > > > > > >> > > > > > > > would > > > > > >> > > > > > > > > rely into the platform. As if you dont want to > > > operate > > > > > >> with > > > > > >> > > > certain > > > > > >> > > > > > > > process > > > > > >> > > > > > > > > you just need to drop the container running it. > It > > > > will > > > > > be > > > > > >> > > shown > > > > > >> > > > in > > > > > >> > > > > > the > > > > > >> > > > > > > > > platform tool like dooker etc.... > > > > > >> > > > > > > > > But if we want to keep track of history i will > leave > > > > > >> things > > > > > >> > > > opening. > > > > > >> > > > > > > For > > > > > >> > > > > > > > > instance i am interested in we start issuing > these > > > > > events > > > > > >> to > > > > > >> > > > those > > > > > >> > > > > > > > > deployments that are still managing some process > > > > > instances > > > > > >> > but > > > > > >> > > > dont > > > > > >> > > > > > > > accept > > > > > >> > > > > > > > > new ones or deployments that have migrated the > new > > > > > process > > > > > >> > > > > > definitions > > > > > >> > > > > > > to > > > > > >> > > > > > > > > new deployments...so i would try to not mess too > > > much > > > > > with > > > > > >> > the > > > > > >> > > > > > process > > > > > >> > > > > > > > > definition but afd a new event in this case > open to > > > > new > > > > > >> > > > transition > > > > > >> > > > > > and > > > > > >> > > > > > > > > states.... And you can store that in any way you > > > fancy > > > > > in > > > > > >> the > > > > > >> > > > data > > > > > >> > > > > > > index. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > El jue, 4 jul 2024, 13:03, Walter Medvedeo < > > > > > >> > > [email protected] > > > > > >> > > > > > > > > > >> > > > > > > > escribió: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > Hi Enrique, > > > > > >> > > > > > > > > > maybe I couldn't express the concept well. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > But let's say that the DI has the catalog of > all > > > the > > > > > >> > existing > > > > > >> > > > > > > > > > ProcessDefinitions. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > To mark a definition as "unavailable" is > kind of > > > > > >> logically > > > > > >> > > > remove > > > > > >> > > > > > it > > > > > >> > > > > > > > from > > > > > >> > > > > > > > > > the Catalog. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > On 2024/07/04 09:59:00 Enrique Gonzalez > Martinez > > > > > wrote: > > > > > >> > > > > > > > > > > Hi walter, this won't give you the data you > are > > > > > aiming > > > > > >> > for. > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > If you have two instances and one goes down > the > > > > > state > > > > > >> > will > > > > > >> > > > be set > > > > > >> > > > > > > as > > > > > >> > > > > > > > not > > > > > >> > > > > > > > > > > available unless you compute something in > the > > > data > > > > > >> index > > > > > >> > so > > > > > >> > > > at > > > > > >> > > > > > > least > > > > > >> > > > > > > > you > > > > > >> > > > > > > > > > > will need more information or duplicate > lines. > > > > This > > > > > is > > > > > >> > > > unlikely > > > > > >> > > > > > to > > > > > >> > > > > > > > be the > > > > > >> > > > > > > > > > > responsibility of the data index. > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > El jue, 4 jul 2024, 11:28, Walter Medvedeo < > > > > > >> > > > [email protected] > > > > > >> > > > > > > > > > > > >> > > > > > > > > > escribió: > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > Hi, the idea is to introduce the > > > > > requirement/concept > > > > > >> > > > first, and > > > > > >> > > > > > > > > > summarize > > > > > >> > > > > > > > > > > > the impact on the information registered > in > > > the > > > > DI > > > > > >> as > > > > > >> > > much > > > > > >> > > > as > > > > > >> > > > > > > > possible. > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > And thus, very summarized, the proposal > is to > > > > > >> include > > > > > >> > one > > > > > >> > > > more > > > > > >> > > > > > > > filed in > > > > > >> > > > > > > > > > > > the ProcessDefinitions information > registered > > > by > > > > > the > > > > > >> > DI, > > > > > >> > > > that > > > > > >> > > > > > > will > > > > > >> > > > > > > > let > > > > > >> > > > > > > > > > us > > > > > >> > > > > > > > > > > > know if a given definition is still > > > "available" > > > > to > > > > > >> > create > > > > > >> > > > > > > > instances. > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > That is a requirement we have from > SonataFlow > > > > > users. > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > On 2024/07/03 20:50:20 Jason Porter wrote: > > > > > >> > > > > > > > > > > > > I am not exactly sure what the proposal > is > > > > > here. I > > > > > >> > see > > > > > >> > > a > > > > > >> > > > > > bunch > > > > > >> > > > > > > of > > > > > >> > > > > > > > > > > > background info for this new feature, but > I > > > > don't > > > > > >> see > > > > > >> > an > > > > > >> > > > actual > > > > > >> > > > > > > > > > proposal. > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > On 2024/07/03 15:06:21 Walter Medvedeo > > > wrote: > > > > > >> > > > > > > > > > > > > > Hello Guys, > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Let me share some requirement we have. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Right now, as part of the > > > ProcessDefinition > > > > > >> > > > information we > > > > > >> > > > > > > > store > > > > > >> > > > > > > > > > in the > > > > > >> > > > > > > > > > > > > > Data Index, we record for example the > > > > > processId > > > > > >> > and > > > > > >> > > > the > > > > > >> > > > > > > > > > serviceURL. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > That information is good for users to > > > know, > > > > > for > > > > > >> > > > instance, > > > > > >> > > > > > > which > > > > > >> > > > > > > > > > are the > > > > > >> > > > > > > > > > > > > > existing ProcessDefitnions, and, > based on > > > > that > > > > > >> > > > information, > > > > > >> > > > > > > > they > > > > > >> > > > > > > > > > can > > > > > >> > > > > > > > > > > > for > > > > > >> > > > > > > > > > > > > > example start a new process instance > by > > > > > >> > concatenating > > > > > >> > > > the > > > > > >> > > > > > > > > > serviceURL + > > > > > >> > > > > > > > > > > > “/” > > > > > >> > > > > > > > > > > > > > + processId. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > In the context of a SonataFlow > Operator > > > > driven > > > > > >> > > setups, > > > > > >> > > > > > users > > > > > >> > > > > > > > need > > > > > >> > > > > > > > > > to > > > > > >> > > > > > > > > > > > know > > > > > >> > > > > > > > > > > > > > if a ProcessDefinition is still > > > “available”. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > For example, users can deploy a WF, > use it > > > > for > > > > > >> some > > > > > >> > > > time, > > > > > >> > > > > > and > > > > > >> > > > > > > > then, > > > > > >> > > > > > > > > > > > decide > > > > > >> > > > > > > > > > > > > > to un-deploy it. From this point, no > more > > > > > >> instances > > > > > >> > > of > > > > > >> > > > that > > > > > >> > > > > > > > > > workflow > > > > > >> > > > > > > > > > > > can be > > > > > >> > > > > > > > > > > > > > created. The user has just decided to > > > remove > > > > > >> that > > > > > >> > > > > > deployment > > > > > >> > > > > > > > from > > > > > >> > > > > > > > > > the > > > > > >> > > > > > > > > > > > > > ecosystem. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > However, in the data-index, that > > > definition > > > > is > > > > > >> > still > > > > > >> > > > > > > recorded. > > > > > >> > > > > > > > And > > > > > >> > > > > > > > > > > > users > > > > > >> > > > > > > > > > > > > > want to keep it there, since they > want to > > > > keep > > > > > >> > > > consistent > > > > > >> > > > > > > > > > information > > > > > >> > > > > > > > > > > > about > > > > > >> > > > > > > > > > > > > > the already executed instances etc. > (The > > > > > >> intention > > > > > >> > is > > > > > >> > > > to > > > > > >> > > > > > not > > > > > >> > > > > > > > > > introduce > > > > > >> > > > > > > > > > > > any > > > > > >> > > > > > > > > > > > > > information deleting procedure at the > DI, > > > > > etc.) > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > On the other hand, users need a way to > > > > filter > > > > > >> which > > > > > >> > > > > > > > definitions are > > > > > >> > > > > > > > > > > > still > > > > > >> > > > > > > > > > > > > > “available” by using the standard > graphql > > > > api. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > What we are evaluating is to include a > > > > > “status” > > > > > >> > > field > > > > > >> > > > in > > > > > >> > > > > > the > > > > > >> > > > > > > > > > > > > > ProcessDefinitions, with the values > > > > > >> > > > available/unavailable, > > > > > >> > > > > > > and, > > > > > >> > > > > > > > > > that > > > > > >> > > > > > > > > > > > will > > > > > >> > > > > > > > > > > > > > be updated as usual via ClouldEvents. > In > > > the > > > > > >> case > > > > > >> > of > > > > > >> > > > > > > > SonataFlow, > > > > > >> > > > > > > > > > those > > > > > >> > > > > > > > > > > > > > events will be produced by the > operator, > > > > which > > > > > >> > > governs > > > > > >> > > > the > > > > > >> > > > > > > > > > > > > > deployment/undeployment, etc. Other > setups > > > > > might > > > > > >> > just > > > > > >> > > > > > ignore > > > > > >> > > > > > > > that > > > > > >> > > > > > > > > > > > field. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Note: we are looking for a very > > > minimalist > > > > > >> > approach > > > > > >> > > > that > > > > > >> > > > > > > lets > > > > > >> > > > > > > > us > > > > > >> > > > > > > > > > > > record > > > > > >> > > > > > > > > > > > > > the situation, and facilitate > SonataFlow > > > > users > > > > > >> to > > > > > >> > > query > > > > > >> > > > > > that > > > > > >> > > > > > > > > > situation > > > > > >> > > > > > > > > > > > by > > > > > >> > > > > > > > > > > > > > using the standard Graphql api. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > This is not any sort of more > elaborated > > > > > >> clustering > > > > > >> > > > > > > > > > > > > > management/monitoring/etc. solution. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Independently of fine grained details > > > about > > > > > the > > > > > >> > event > > > > > >> > > > to > > > > > >> > > > > > > > introduce, > > > > > >> > > > > > > > > > > > which > > > > > >> > > > > > > > > > > > > > carries more or less the information > we > > > are > > > > > >> already > > > > > >> > > > sending > > > > > >> > > > > > > in > > > > > >> > > > > > > > the > > > > > >> > > > > > > > > > > > other DI > > > > > >> > > > > > > > > > > > > > events. > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > From the point of view of let’s say > non > > > > > >> SonataFlow > > > > > >> > > > setups, > > > > > >> > > > > > do > > > > > >> > > > > > > > you > > > > > >> > > > > > > > > > guys > > > > > >> > > > > > > > > > > > see > > > > > >> > > > > > > > > > > > > > any objections? > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Regards, > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Walter. > > > > > >> > > > > > > > > > > > > > ResponderReenviar > > > > > >> > > > > > > > > > > > > > Añadir reacción > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > >> > > > --------------------------------------------------------------------- > > > > > >> > > > > > > > > > > > > To unsubscribe, e-mail: > > > > > >> > [email protected] > > > > > >> > > > > > > > > > > > > For additional commands, e-mail: > > > > > >> > > [email protected] > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > >> > > > --------------------------------------------------------------------- > > > > > >> > > > > > > > > > > > To unsubscribe, e-mail: > > > > > >> [email protected] > > > > > >> > > > > > > > > > > > For additional commands, e-mail: > > > > > >> > [email protected] > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > --------------------------------------------------------------------- > > > > > >> > > > > > > > > > To unsubscribe, e-mail: > > > > > [email protected] > > > > > >> > > > > > > > > > For additional commands, e-mail: > > > > > >> [email protected] > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > >> > > > --------------------------------------------------------------------- > > > > > >> > > > > > > > To unsubscribe, e-mail: > > > [email protected] > > > > > >> > > > > > > > For additional commands, e-mail: > > > > [email protected] > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > --------------------------------------------------------------------- > > > > > >> > > > To unsubscribe, e-mail: [email protected] > > > > > >> > > > For additional commands, e-mail: [email protected] > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
