This is an automated email from the ASF dual-hosted git repository. dhanak pushed a commit to branch 10.1.x in repository https://gitbox.apache.org/repos/asf/incubator-kie-kogito-docs.git
commit 52075bb18b21a9981650a85371eaa5734a0426ad Author: Walter Medvedeo <[email protected]> AuthorDate: Wed Apr 30 07:43:46 2025 +0200 kie-kogito-docs-716: [sonataflow-operator] Improvements on the DB migration documentation section (#717) * kie-kogito-docs-716: [sonataflow-operator] Improvements on the DB migration documentation section * Apply suggestions from code review Co-authored-by: Ricardo Zanini <[email protected]> --------- Co-authored-by: Ricardo Zanini <[email protected]> --- .../pages/cloud/operator/supporting-services.adoc | 162 +++++++++------------ 1 file changed, 71 insertions(+), 91 deletions(-) diff --git a/serverlessworkflow/modules/ROOT/pages/cloud/operator/supporting-services.adoc b/serverlessworkflow/modules/ROOT/pages/cloud/operator/supporting-services.adoc index 2f7510dab..1260bdf17 100644 --- a/serverlessworkflow/modules/ROOT/pages/cloud/operator/supporting-services.adoc +++ b/serverlessworkflow/modules/ROOT/pages/cloud/operator/supporting-services.adoc @@ -98,32 +98,44 @@ And thus, it's only recommended for development and testing purposes. The ephemeral deployment of a service requires no additional configurations than the shown, <<supporting-services-configuration, here>>. -[#postgresql-persistence-configuration] -=== PostgreSQL persistence configuration +[#database-migration-configuration] +=== Database migration configuration -The PostgreSQL persistence of a service is supported by a PostgreSQL server instance that you must previously install on the cluster. -The administration of that instance is totally independent of the {operator_name} scope, and to connect a supporting service with it, you must only configure the correct database connection parameters. +The term database migration refers to either initializing a given Data Index or Jobs Service database to the respective schema, or applying data or schema updates as new versions are released. + +The database migration strategy must be configured individually for each supporting service, by using the optional `dataIndex.persistence.dbMigrationStrategy` and `jobService.persistence.dbMigrationStrategy` fields respectively. -==== {product_name} Operator Database Migrator capabilities -The {product_name} Operator offers out of the box database migration features, which help in configuring the databases for use by Data Index and/or Jobs Service. +You can use any of the following database migration strategies: `job`, `service`, and `none`. When not configured, the default value is `service`. + +[NOTE] +==== +Currently, database migration is only supported when using the PostgreSQL persistence configuration. +==== -The term database migration refers to either initializing a given Data Index or Jobs Service database to the schema needed by them or applying data or schema updates as new versions are released. +==== Job based database migration strategy -{product_name} Operator offers jobs based database migration capabilities along with two other modes: service based database migration and none. +When you configure the `job` based strategy, the {product_name} Operator decouples the database migration by using a dedicated Kubernetes Job that manages the entire migration process. -The {product_name} Operator offered database migration modes are described as follows. +That Job executes before the supporting services deployment, and thus, ensures that a given service is started only if the corresponding migration was successful. -.Jobs based database migration -In this mode, the {product_name} Operator creates a Kubernetes Job and uses the database migrator component to migrate configured databases. You would typically use this mode when running in non-development scenarios such as Production environments, where there could be an existing database for use by Data Index and/or Jobs Service. +You would typically use this strategy in production environments. -.Service based database migration -In this mode, the {product_name} Operator does not create a Kubernetes Job and hence does not use the database migrator component to migrate configured databases. In this mode, thus the Data Index or Jobs Service are responsible to migrate the configured databases. You would typically use this mode when running in development scenarios such as local environments, where you are testing scenarios with clean state everytime you deploy and run. +==== Service based database migration strategy -.None mode -In this mode, the {product_name} Operator makes no migration attempt. +When you configure the `service` based strategy, the database migration is managed by each supporting service and is executed as part of the service's startup sequence. +In worst-case scenarios, a service could start with failures. +==== None migration strategy -=== Overview of PostgreSQL persistence configuration +When you configure the `none` strategy, no Operator nor service migration attempt is executed. + +You would typically use this strategy in environments where a database administrator executes all the database migrations. + +[#postgresql-persistence-configuration] +=== PostgreSQL persistence configuration + +The PostgreSQL persistence of a service is supported by a PostgreSQL server instance that you must have previously installed on the cluster. +The administration of that instance is totally independent of the {operator_name} scope, and to connect a supporting service with it, you must only configure the correct database connection parameters. The following `SonataFlowPlatform` CR fragment shows the configuration options that you must use: @@ -140,7 +152,6 @@ spec: dataIndex: enabled: true persistence: - # job=job based db migration, service=service based db migration, none=no db migration dbMigrationStrategy: job <1> postgresql: serviceRef: @@ -156,11 +167,12 @@ spec: jobService: enabled: true persistence: + dbMigrationStrategy: job postgresql: # Specific database configuration for the Job Service # might be included here. ---- -<1> (Optional) Database migration approach to be used by {product_name} Operator. Defaults to a value of `service`. +<1> (Optional) Database migration strategy to use, <<database-migration-configuration, see>>. Defaults to `service`. <2> Name of the Kubernetes Service to connect with the PostgreSQL database server. <3> (Optional) Kubernetes namespace containing the PostgreSQL Service. Defaults to the `SonataFlowPlatform's` local namespace. <4> Name of the PostgreSQL database to store the supporting service data. @@ -184,97 +196,65 @@ To create the secrets for the example above you can use a command like this: kubectl create secret generic postgres-secrets-example --from-literal=POSTGRESQL_USER=<user> --from-literal=POSTGRESQL_PASSWORD=<password> -n postgres-example-namespace ---- -[#common-persistence-scenarios] -=== Persistence examples with most common scenarios -==== Specify database information at persistence level instead of service level -You will use SonataFlowPlatform CR's `.spec.persistence.postgresql` to specify the database to be used by the Data Index and Jobs Service. Then, specify the database migration mode to use under `.spec.services.dataIndex.persistence.dbMigrationStrategy` or `.spec.services.jobService.persistence.dbMigrationStrategy` with a value such as job, service, or none. -In this mode the Data Index and Jobs Service will make use of one common database. +[#common-persistence-configuration] +=== Platform-scoped PostgreSQL persistence configuration + +To configure a common PostgreSQL service and database for the supporting services, you must use the field `spec.persistence.postgresql` in the `SonataFlowPlatform` CR. + +This information signals the {operator_name} to configure the supporting services to connect to that database. + +Additionally, workflows deployed in that namespace, with the `preview` or `gitops` profile, that don’t provide a custom persistence configuration, will connect to that database too, xref:cloud/operator/using-persistence.adoc#configuring-persistence-using-the-sonataflowplatform-cr[see]. + +And, similarly to the workflow's persistence, the following rules apply: + +* If a supporting service has a configured persistence, for example, the field `services.dataIndex.persistence.postgresql` is configured. That configuration will apply. + +* If a supporting service has no configured persistence, for example, if the `services.dataIndex.persistence.postgresql` field is not set, the persistence configuration will be taken from the current platform. + +* Additionally, if a supporting service requires a particular database migration strategy, you must use the respective `dataIndex.persistence.dbMigrationStrategy` and `jobService.persistence.dbMigrationStrategy` fields, <<#database-migration-configuration, see>>. -[source, yaml] +The following `SonataFlowPlatform` CR fragment shows the configuration options that you must use: + +.SonataFlowPlatform CR persistence configuration example +[source,yaml] ---- apiVersion: sonataflow.org/v1alpha08 kind: SonataFlowPlatform metadata: - name: sonataflow-platform + name: sonataflow-platform-example + namespace: example-namespace spec: - build: - config: - strategyOptions: - KanikoBuildCacheEnabled: "true" persistence: postgresql: - secretRef: - name: postgres-secrets - userKey: POSTGRES_USER - passwordKey: POSTGRES_PASSWORD serviceRef: - name: postgres - databaseName: sonataflow - services: - dataIndex: - enabled: true - persistence: - # job=job based db migration, service=service based db migration, none=no db migration - dbMigrationStrategy: job - jobService: - enabled: true - persistence: - # job=job based db migration, service=service based db migration, none=no db migration - dbMigrationStrategy: job ----- - -==== Specify database information at service level -You will use SonataFlowPlatform CR's `.spec.services.dataIndex` or `.spec.services.jobService.persistence` to specify database information as shown in the example. As earlier, the `dbMigrationStrategy` can be used with a value such as job, service, or none. - -[source, yaml] ----- -apiVersion: sonataflow.org/v1alpha08 -kind: SonataFlowPlatform -metadata: - name: sonataflow-platform -spec: - build: - config: - strategyOptions: - KanikoBuildCacheEnabled: "true" - services: + name: postgres-example <1> + namespace: postgres-example-namespace <2> + databaseName: example-database <3> + port: 1234 <4> + secretRef: + name: postgres-secrets-example <5> + userKey: POSTGRESQL_USER <6> + passwordKey: POSTGRESQL_PASSWORD <7> dataIndex: enabled: true persistence: - # job=job based db migration, service=service based db migration, none=no db migration - dbMigrationStrategy: job - postgresql: - jdbcUrl: jdbc:postgresql://postgres:5432/sonataflow?currentSchema=data-index-service - secretRef: - name: postgres-secrets - userKey: POSTGRES_USER - passwordKey: POSTGRES_PASSWORD + dbMigrationStrategy: job <8> jobService: enabled: true persistence: - # job=job based db migration, service=service based db migration, none=no db migration - dbMigrationStrategy: job - postgresql: - jdbcUrl: jdbc:postgresql://postgres:5432/sonataflow?currentSchema=jobs-service - secretRef: - name: postgres-secrets - userKey: POSTGRES_USER - passwordKey: POSTGRES_PASSWORD + dbMigrationStrategy: service <9> ---- - -[#common-persistence-configuration] -=== Common PostgreSQL persistence configuration - -To configure a common PostgreSQL service instance for all the supporting services you must read, xref:cloud/operator/using-persistence.adoc#configuring-persistence-using-the-sonataflowplatform-cr[Configuring the persistence using the SonataFlowPlatform CR]. - -In that case, the {operator_name} will automatically connect any of the supporting services with that common server configured in the field `spec.persistence`. And, similarly to the workflow's persistence, the following precedence rules apply: - -* If a supporting service has a configured persistence, for example, the field `services.dataIndex.persistence` is configured. That configuration will apply. - -* If a supporting service has no configured persistence, for example, the field `services.dataIndex.persistence` is not set at all, the persistence configuration will be taken from the current platform. - +<1> Name of the Kubernetes Service to connect with the PostgreSQL database server. +<2> (Optional) Kubernetes namespace containing the PostgreSQL Service. Defaults to the `SonataFlowPlatform's` local namespace. +<3> Name of the PostgreSQL database to store the supporting services and workflows data. +<4> (Optional) Port number to connect with the PostgreSQL Service. Defaults to 5432. +<5> Name of the link:{k8n_secrets_url}[Kubernetes Secret] containing the username and password to connect with the database. +<6> Name of the link:{k8n_secrets_url}[Kubernetes Secret] `key` containing the username to connect with the database. +<7> Name of the link:{k8n_secrets_url}[Kubernetes Secret] `key` containing the password to connect with the database. +<8> (Optional) Database migration strategy to use for the Data Index, <<database-migration-configuration, see>>. Defaults to `service`. +<9> (Optional) Database migration strategy to use for the Job Service Index, <<database-migration-configuration, see>>. Defaults to `service`. While no usual, the supporting services could use distinct database migration strategies. [NOTE] ==== --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
