This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-8032-cd4fd5a6d3845f0b7b478e1e86a31bb3151d4cd5 in repository https://gitbox.apache.org/repos/asf/texera.git
commit d09186855d1da1b3d188689d68ae93a57d87bc75 Author: Ryan Zhang <[email protected]> AuthorDate: Thu Sep 3 20:26:08 2026 +0000 feat(notebook-migration): resolve the Jupyter URL and token per user (#8032) ### What changes were proposed in this PR? `notebook-migration-service` read one Jupyter URL and token from configuration as process-wide values. That is safe only under the per-user-pod model, where each user runs their own pod. A single global instance would hand every user the same Jupyter and the same token. This resolves both per user instead. This is stage 2 of the staged roadmap in the architectural note on #5258. Stage 1 (#7390) and stage 4 (#7671) are merged. Stage 3 (#8006) depends on this one. **Registry table** A new `user_jupyter` table holds one row per user whose Jupyter has been provisioned, keyed on `uid`. Both the internal and public URLs are stored rather than derived at read time, matching how `workflow_computing_unit.uri` is the recorded source of truth for a computing unit's address. Storing them lets the deployment change its addressing scheme without a code change, which is what #8006 needs in order to choose between per-user hostnames and a path prefix. **Derived tokens** Each user's token is `HMAC-SHA256(secret, uid)`, computed from `storage.jupyter.token-secret`. No credential is stored at rest, any replica of the service derives the same value, and rotation is a secret change. The service refuses to start when per-user Jupyter is on with an empty secret: an empty HMAC key is publicly known, so tokens would still look distinct per user while anyone could derive anyone else's. **Fallback gated on a flag, not on row absence** With `kubernetes.jupyter-enabled` off, every user resolves to the statically configured Jupyter, which is how the single-node (#7932) and local-dev (#7934) deployments run one shared JupyterLab. With it on, a user with no row has nothing provisioned yet and gets an unavailable response. Falling back to the shared server there would hand an unprovisioned user somebody else's notebooks, and would let `delete-notebook` remove another user's file. The flag is deliberately separate from `kubernetes.enabled`, so a deployment can run computing units on Kubernetes without per-user Jupyter. **Provisioning** `JupyterKubernetesClient` mirrors the computing unit's `KubernetesClient`, including the hostname and subdomain pair that makes the pod FQDN resolve through a headless service. `get-jupyter-iframe-url` and `set-notebook` provision on demand; `get-jupyter-url` and `delete-notebook` resolve only, so reading a URL or deleting a file never starts a pod. A registered pod that stops answering is discarded and rebuilt, since the row would otherwise outlive the pod and point every later request at nothing. Two concurrent first requests are resolved by the primary key: the loser keeps the winner's row, which holds the same uid-derived addresses. The uid always comes from the authenticated session and never from a request body, so one user cannot address another's Jupyter. Two notes on the implementation. The Kubernetes client is built on demand rather than at object initialisation, unlike the computing unit's, because the single-node and local-dev deployments have no cluster to build one against and never provision. Adding `io.fabric8:kubernetes-client` also raises three transitive versions over Dropwizard's pins (`slf4j-api` to 2.0.13, `jackson-dataformat-yaml` and `jackson-datatype-jsr310` to 2.17.0); `LICENSE-binary` claims are updated to match. Out of scope, and left to #8006: the Helm templates, the NetworkPolicy, and the global Deployment. Also out of scope is reclaiming idle pods. A dead pod self-heals on next use, but nothing reaps one that is merely unused, and the row's `ON DELETE CASCADE` drops the registration without deleting the pod. Until a deployment supplies a browser-reachable address, `kubernetes.jupyter-public-url-template` defaults to empty and the public URL falls back to the in-cluster one. ### Any related issues, documentation, discussions? Closes #7665 Parent issue #4301 Stage 3 (#8006) is built directly on this branch, so every decision here has been exercised on a real cluster rather than only in unit tests. See the verification comment below. ### How was this PR tested? `NotebookMigrationService/test` passes (72) and `Config/test` passes (71). `Test/compile` across the whole repo is clean, as is `scalafmtCheckAll`. New coverage: token derivation is deterministic per uid, distinct across 50 uids, and changes with the secret; the start-up guard rejects an empty secret only when the feature is on. Resolution falls back with the feature off, returns the row with it on, returns nothing for an unregistered user, and never returns one user's Jupyter to another. Provisioning is exercised against a stubbed Kubernetes client across every branch: fresh provision, reuse of a live pod, rebuild of a pod that stopped answering, cleanup when readiness never arrives, public URL templating, and the concurrent-registration race. The existing `withFakeJupyter` suite is untouched and still passes on the stub bound to `localhost:9100`, which is the point of the flag-gated fallback: test users have no registry row and take the configured path exactly as before. Applied `sql/texera_ddl.sql` to a scratch database the way CI does, confirmed the table shape, and confirmed that deleting a user removes their registration row. `check_binary_deps.py` passes under both the PR-mode and the strict nightly invocations, and `NOTICE-binary` matches generator output byte for byte. Not tested on a live cluster. The chart has no notebook-migration or Jupyter templates yet, and the in-cluster pod address does not resolve from outside the cluster, so end to end Kubernetes validation belongs to #8006. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) --- common/config/src/main/resources/kubernetes.conf | 34 ++ common/config/src/main/resources/storage.conf | 9 +- .../texera/common/config/KubernetesConfig.scala | 14 + .../texera/common/config/StorageConfig.scala | 3 + .../common/config/KubernetesConfigSpec.scala | 29 +- .../texera/common/config/StorageConfigSpec.scala | 8 + notebook-migration-service/LICENSE-binary | 36 +- notebook-migration-service/build.sbt | 3 +- .../texera/service/NotebookMigrationService.scala | 4 + .../resource/NotebookMigrationResource.scala | 98 ++--- .../service/util/JupyterEndpointResolver.scala | 61 +++ .../texera/service/util/JupyterEndpoints.scala | 37 ++ .../service/util/JupyterKubernetesClient.scala | 113 ++++++ .../apache/texera/service/util/JupyterProbe.scala | 72 ++++ .../texera/service/util/JupyterProvisioner.scala | 209 ++++++++++ .../texera/service/util/JupyterTokenDeriver.scala | 59 +++ .../resource/NotebookMigrationResourceSpec.scala | 438 ++++++++++++++++++++- .../texera/service/util/JupyterEndpointsSpec.scala | 60 +++ .../service/util/JupyterKubernetesClientSpec.scala | 210 ++++++++++ .../texera/service/util/JupyterProbeSpec.scala | 109 +++++ .../service/util/JupyterTokenDeriverSpec.scala | 89 +++++ sql/changelog.xml | 5 + sql/texera_ddl.sql | 15 + sql/updates/48.sql | 40 ++ 24 files changed, 1698 insertions(+), 57 deletions(-) diff --git a/common/config/src/main/resources/kubernetes.conf b/common/config/src/main/resources/kubernetes.conf index e85924e570..19c9796288 100644 --- a/common/config/src/main/resources/kubernetes.conf +++ b/common/config/src/main/resources/kubernetes.conf @@ -54,4 +54,38 @@ kubernetes { # GPU resource key used in Kubernetes (vendor-specific) computing-unit-gpu-resource-key = "nvidia.com/gpu" computing-unit-gpu-resource-key = ${?KUBERNETES_COMPUTING_UNIT_GPU_RESOURCE_KEY} + + # Per-user JupyterLab pods. Separate from `enabled` above, so a deployment can run + # computing units on Kubernetes without per-user Jupyter. While this is off, the + # notebook migration service uses the single Jupyter from storage.jupyter. + jupyter-enabled = false + jupyter-enabled = ${?KUBERNETES_JUPYTER_ENABLED} + + jupyter-namespace = "texera-jupyter-pool" + jupyter-namespace = ${?KUBERNETES_JUPYTER_NAMESPACE} + + jupyter-service-name = "jupyter-svc" + jupyter-service-name = ${?KUBERNETES_JUPYTER_SERVICE_NAME} + + jupyter-image-name = "ghcr.io/apache/texera-jupyter:latest" + jupyter-image-name = ${?KUBERNETES_JUPYTER_IMAGE_NAME} + + jupyter-port-num = 8888 + + # Origin the pod names in its iframe CSP and its postMessage checks. The image defaults + # to the local dev origin when this is empty, which suits no real deployment. + jupyter-texera-origin = "" + jupyter-texera-origin = ${?KUBERNETES_JUPYTER_TEXERA_ORIGIN} + + jupyter-cpu-limit = "1" + jupyter-cpu-limit = ${?KUBERNETES_JUPYTER_CPU_LIMIT} + + jupyter-memory-limit = "2Gi" + jupyter-memory-limit = ${?KUBERNETES_JUPYTER_MEMORY_LIMIT} + + # Browser-facing address, with {uid} substituted. The in-network pod name does not + # resolve from the browser, so a deployment that publishes Jupyter sets this; empty + # falls back to the in-network address. + jupyter-public-url-template = "" + jupyter-public-url-template = ${?KUBERNETES_JUPYTER_PUBLIC_URL_TEMPLATE} } \ No newline at end of file diff --git a/common/config/src/main/resources/storage.conf b/common/config/src/main/resources/storage.conf index 9af2924901..b6b76d8ad3 100644 --- a/common/config/src/main/resources/storage.conf +++ b/common/config/src/main/resources/storage.conf @@ -177,7 +177,8 @@ storage { password = ${?STORAGE_JDBC_PASSWORD} } - # Configurations of the JupyterLab service + # The single JupyterLab used when per-user provisioning (kubernetes.jupyter-enabled) + # is off, which is how the single-node and local-dev deployments run. jupyter { internal-url = "http://localhost:9100"; internal-url = ${?STORAGE_JUPYTER_INTERNAL_URL} @@ -188,5 +189,11 @@ storage { # Read from the same JUPYTER_TOKEN env var as the Jupyter container token = "texera" token = ${?JUPYTER_TOKEN} + + # HMAC key each per-user Jupyter token is derived from, so no token is stored. + # Required when per-user Jupyter is on, and must stay stable across restarts and + # replicas or previously issued tokens stop matching. + token-secret = "" + token-secret = ${?JUPYTER_TOKEN_SECRET} } } diff --git a/common/config/src/main/scala/org/apache/texera/common/config/KubernetesConfig.scala b/common/config/src/main/scala/org/apache/texera/common/config/KubernetesConfig.scala index f629476736..0812eae7ad 100644 --- a/common/config/src/main/scala/org/apache/texera/common/config/KubernetesConfig.scala +++ b/common/config/src/main/scala/org/apache/texera/common/config/KubernetesConfig.scala @@ -64,4 +64,18 @@ object KubernetesConfig { // GPU resource key used directly in Kubernetes resource specifications val gpuResourceKey: String = conf.getString("kubernetes.computing-unit-gpu-resource-key") + + // Per-user JupyterLab pods, gated independently of computing units. + val jupyterEnabled: Boolean = conf.getBoolean("kubernetes.jupyter-enabled") + val jupyterNamespace: String = conf.getString("kubernetes.jupyter-namespace") + val jupyterServiceName: String = conf.getString("kubernetes.jupyter-service-name") + val jupyterImageName: String = conf.getString("kubernetes.jupyter-image-name") + val jupyterPortNumber: Int = conf.getInt("kubernetes.jupyter-port-num") + val jupyterCpuLimit: String = conf.getString("kubernetes.jupyter-cpu-limit") + val jupyterMemoryLimit: String = conf.getString("kubernetes.jupyter-memory-limit") + val jupyterTexeraOrigin: String = conf.getString("kubernetes.jupyter-texera-origin") + + // Browser-facing address with {uid} substituted; empty means use the in-network one. + val jupyterPublicUrlTemplate: String = + conf.getString("kubernetes.jupyter-public-url-template") } diff --git a/common/config/src/main/scala/org/apache/texera/common/config/StorageConfig.scala b/common/config/src/main/scala/org/apache/texera/common/config/StorageConfig.scala index e48fe4f84e..2627b12f5d 100644 --- a/common/config/src/main/scala/org/apache/texera/common/config/StorageConfig.scala +++ b/common/config/src/main/scala/org/apache/texera/common/config/StorageConfig.scala @@ -160,4 +160,7 @@ object StorageConfig { val jupyterInternalURL: String = conf.getString("storage.jupyter.internal-url") val jupyterPublicURL: String = conf.getString("storage.jupyter.public-url") val jupyterToken: String = conf.getString("storage.jupyter.token") + + // HMAC key for per-user token derivation; empty unless a deployment sets it. + val jupyterTokenSecret: String = conf.getString("storage.jupyter.token-secret") } diff --git a/common/config/src/test/scala/org/apache/texera/common/config/KubernetesConfigSpec.scala b/common/config/src/test/scala/org/apache/texera/common/config/KubernetesConfigSpec.scala index ba9a0acd7f..1c0d8da1b5 100644 --- a/common/config/src/test/scala/org/apache/texera/common/config/KubernetesConfigSpec.scala +++ b/common/config/src/test/scala/org/apache/texera/common/config/KubernetesConfigSpec.scala @@ -24,7 +24,7 @@ import org.scalatest.matchers.should.Matchers /** * Spec for [[KubernetesConfig]]. Reading each value forces resolution from kubernetes.conf, so a - * renamed or mistyped key surfaces here as a ConfigException. Every value except the port number + * renamed or mistyped key surfaces here as a ConfigException. Every value except the port numbers * carries a `${?ENV}` override, so exact-value assertions are guarded on the env var being unset. */ class KubernetesConfigSpec extends AnyFlatSpec with Matchers { @@ -70,6 +70,33 @@ class KubernetesConfigSpec extends AnyFlatSpec with Matchers { KubernetesConfig.maxNumOfRunningComputingUnitsPerUser should be >= 0 } + "KubernetesConfig jupyter settings" should "resolve to their kubernetes.conf defaults" in { + KubernetesConfig.jupyterPortNumber shouldBe 8888 + // Off by default and keyed separately from kubernetes.enabled, so enabling computing + // units on Kubernetes never silently enables per-user Jupyter. + ifUnset("KUBERNETES_JUPYTER_ENABLED")(KubernetesConfig.jupyterEnabled shouldBe false) + ifUnset("KUBERNETES_JUPYTER_NAMESPACE")( + KubernetesConfig.jupyterNamespace shouldBe "texera-jupyter-pool" + ) + ifUnset("KUBERNETES_JUPYTER_SERVICE_NAME")( + KubernetesConfig.jupyterServiceName shouldBe "jupyter-svc" + ) + ifUnset("KUBERNETES_JUPYTER_IMAGE_NAME")( + KubernetesConfig.jupyterImageName shouldBe "ghcr.io/apache/texera-jupyter:latest" + ) + // Empty by default: only a real deployment knows its own origin. + ifUnset("KUBERNETES_JUPYTER_TEXERA_ORIGIN")(KubernetesConfig.jupyterTexeraOrigin shouldBe "") + ifUnset("KUBERNETES_JUPYTER_CPU_LIMIT")(KubernetesConfig.jupyterCpuLimit shouldBe "1") + ifUnset("KUBERNETES_JUPYTER_MEMORY_LIMIT")( + KubernetesConfig.jupyterMemoryLimit shouldBe "2Gi" + ) + // Empty means the browser is handed the in-network address; a deployment that + // publishes Jupyter overrides it. + ifUnset("KUBERNETES_JUPYTER_PUBLIC_URL_TEMPLATE")( + KubernetesConfig.jupyterPublicUrlTemplate shouldBe "" + ) + } + "KubernetesConfig limit options" should "parse into trimmed, non-empty lists" in { ifUnset("KUBERNETES_COMPUTING_UNIT_CPU_LIMIT_OPTIONS")( KubernetesConfig.cpuLimitOptions shouldBe List("1", "2", "4") diff --git a/common/config/src/test/scala/org/apache/texera/common/config/StorageConfigSpec.scala b/common/config/src/test/scala/org/apache/texera/common/config/StorageConfigSpec.scala index ac34c46764..45b628a319 100644 --- a/common/config/src/test/scala/org/apache/texera/common/config/StorageConfigSpec.scala +++ b/common/config/src/test/scala/org/apache/texera/common/config/StorageConfigSpec.scala @@ -73,4 +73,12 @@ class StorageConfigSpec extends AnyFlatSpec with Matchers { StorageConfig.jupyterPublicURL shouldBe StorageConfig.jupyterInternalURL } } + + it should "default the token secret to empty so a deployment must set it deliberately" in { + // Per-user tokens are derived from this key, so it has no safe default: an empty + // value must be caught at start-up rather than silently deriving from nothing. + if (sys.env.get("JUPYTER_TOKEN_SECRET").isEmpty) { + StorageConfig.jupyterTokenSecret shouldBe "" + } + } } diff --git a/notebook-migration-service/LICENSE-binary b/notebook-migration-service/LICENSE-binary index 78f1df46a9..7b9b8f84b4 100644 --- a/notebook-migration-service/LICENSE-binary +++ b/notebook-migration-service/LICENSE-binary @@ -224,10 +224,10 @@ Scala/Java jars: - com.fasterxml.jackson.core.jackson-annotations-2.18.8.jar - com.fasterxml.jackson.core.jackson-core-2.18.8.jar - com.fasterxml.jackson.core.jackson-databind-2.18.8.jar - - com.fasterxml.jackson.dataformat.jackson-dataformat-yaml-2.16.1.jar + - com.fasterxml.jackson.dataformat.jackson-dataformat-yaml-2.17.0.jar - com.fasterxml.jackson.datatype.jackson-datatype-guava-2.16.1.jar - com.fasterxml.jackson.datatype.jackson-datatype-jdk8-2.16.1.jar - - com.fasterxml.jackson.datatype.jackson-datatype-jsr310-2.16.1.jar + - com.fasterxml.jackson.datatype.jackson-datatype-jsr310-2.17.0.jar - com.fasterxml.jackson.jakarta.rs.jackson-jakarta-rs-base-2.16.1.jar - com.fasterxml.jackson.jakarta.rs.jackson-jakarta-rs-json-provider-2.16.1.jar - com.fasterxml.jackson.module.jackson-module-blackbird-2.16.1.jar @@ -242,6 +242,9 @@ Scala/Java jars: - com.google.guava.listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar - com.google.j2objc.j2objc-annotations-2.8.jar - com.helger.profiler-1.1.1.jar + - com.squareup.okhttp3.logging-interceptor-3.12.12.jar + - com.squareup.okhttp3.okhttp-3.12.12.jar + - com.squareup.okio.okio-1.15.0.jar - com.thesamet.scalapb.lenses_2.13-0.11.20.jar - com.thesamet.scalapb.scalapb-json4s_2.13-0.12.0.jar - com.thesamet.scalapb.scalapb-runtime_2.13-0.11.20.jar @@ -274,6 +277,32 @@ Scala/Java jars: - io.dropwizard.metrics.metrics-json-4.2.25.jar - io.dropwizard.metrics.metrics-jvm-4.2.25.jar - io.dropwizard.metrics.metrics-logback-4.2.25.jar + - io.fabric8.kubernetes-client-6.12.1.jar + - io.fabric8.kubernetes-client-api-6.12.1.jar + - io.fabric8.kubernetes-httpclient-okhttp-6.12.1.jar + - io.fabric8.kubernetes-model-admissionregistration-6.12.1.jar + - io.fabric8.kubernetes-model-apiextensions-6.12.1.jar + - io.fabric8.kubernetes-model-apps-6.12.1.jar + - io.fabric8.kubernetes-model-autoscaling-6.12.1.jar + - io.fabric8.kubernetes-model-batch-6.12.1.jar + - io.fabric8.kubernetes-model-certificates-6.12.1.jar + - io.fabric8.kubernetes-model-common-6.12.1.jar + - io.fabric8.kubernetes-model-coordination-6.12.1.jar + - io.fabric8.kubernetes-model-core-6.12.1.jar + - io.fabric8.kubernetes-model-discovery-6.12.1.jar + - io.fabric8.kubernetes-model-events-6.12.1.jar + - io.fabric8.kubernetes-model-extensions-6.12.1.jar + - io.fabric8.kubernetes-model-flowcontrol-6.12.1.jar + - io.fabric8.kubernetes-model-gatewayapi-6.12.1.jar + - io.fabric8.kubernetes-model-metrics-6.12.1.jar + - io.fabric8.kubernetes-model-networking-6.12.1.jar + - io.fabric8.kubernetes-model-node-6.12.1.jar + - io.fabric8.kubernetes-model-policy-6.12.1.jar + - io.fabric8.kubernetes-model-rbac-6.12.1.jar + - io.fabric8.kubernetes-model-resource-6.12.1.jar + - io.fabric8.kubernetes-model-scheduling-6.12.1.jar + - io.fabric8.kubernetes-model-storageclass-6.12.1.jar + - io.fabric8.zjsonpatch-0.3.0.jar - io.r2dbc.r2dbc-spi-1.0.0.RELEASE.jar - jakarta.inject.jakarta.inject-api-2.0.1.jar - jakarta.validation.jakarta.validation-api-3.0.2.jar @@ -300,6 +329,7 @@ Scala/Java jars: - org.scala-lang.scala-reflect-2.13.18.jar - org.slf4j.jcl-over-slf4j-2.0.12.jar - org.slf4j.log4j-over-slf4j-2.0.12.jar + - org.snakeyaml.snakeyaml-engine-2.7.jar - org.yaml.snakeyaml-2.2.jar -------------------------------------------------------------------------------- @@ -327,7 +357,7 @@ Scala/Java jars: - net.sourceforge.argparse4j.argparse4j-0.9.0.jar - org.checkerframework.checker-qual-3.52.0.jar - org.slf4j.jul-to-slf4j-2.0.12.jar - - org.slf4j.slf4j-api-2.0.12.jar + - org.slf4j.slf4j-api-2.0.13.jar -------------------------------------------------------------------------------- Dependencies under the BSD 3-Clause License diff --git a/notebook-migration-service/build.sbt b/notebook-migration-service/build.sbt index e907d977f1..5714f570b8 100644 --- a/notebook-migration-service/build.sbt +++ b/notebook-migration-service/build.sbt @@ -82,5 +82,6 @@ libraryDependencies ++= Seq( libraryDependencies ++= Seq( "io.dropwizard" % "dropwizard-core" % dropwizardVersion, "io.dropwizard" % "dropwizard-auth" % dropwizardVersion, // Dropwizard Authentication module - "com.fasterxml.jackson.module" %% "jackson-module-scala" % "2.18.8" + "com.fasterxml.jackson.module" %% "jackson-module-scala" % "2.18.8", + "io.fabric8" % "kubernetes-client" % "6.12.1" // Provisions per-user JupyterLab pods ) \ No newline at end of file diff --git a/notebook-migration-service/src/main/scala/org/apache/texera/service/NotebookMigrationService.scala b/notebook-migration-service/src/main/scala/org/apache/texera/service/NotebookMigrationService.scala index fe9214b0d1..567cf2b4c6 100644 --- a/notebook-migration-service/src/main/scala/org/apache/texera/service/NotebookMigrationService.scala +++ b/notebook-migration-service/src/main/scala/org/apache/texera/service/NotebookMigrationService.scala @@ -35,6 +35,7 @@ import org.apache.texera.dao.SqlServer import org.glassfish.jersey.server.filter.RolesAllowedDynamicFeature import java.nio.file.Path import org.apache.texera.service.resource.{HealthCheckResource, NotebookMigrationResource} +import org.apache.texera.service.util.JupyterTokenDeriver class NotebookMigrationService extends Application[NotebookMigrationServiceConfiguration] @@ -61,6 +62,9 @@ class NotebookMigrationService configuration: NotebookMigrationServiceConfiguration, environment: Environment ): Unit = { + // Refuse to boot a misconfigured per-user Jupyter rather than failing per request. + JupyterTokenDeriver.validateConfiguration() + // Serve backend at /api environment.jersey.setUrlPattern("/api/*") diff --git a/notebook-migration-service/src/main/scala/org/apache/texera/service/resource/NotebookMigrationResource.scala b/notebook-migration-service/src/main/scala/org/apache/texera/service/resource/NotebookMigrationResource.scala index f048eda3f3..63f6e23888 100644 --- a/notebook-migration-service/src/main/scala/org/apache/texera/service/resource/NotebookMigrationResource.scala +++ b/notebook-migration-service/src/main/scala/org/apache/texera/service/resource/NotebookMigrationResource.scala @@ -35,7 +35,12 @@ import org.apache.texera.dao.jooq.generated.tables.WorkflowVersion import java.net.{HttpURLConnection, URL} import java.nio.charset.StandardCharsets import scala.util.control.NonFatal -import org.apache.texera.common.config.StorageConfig +import org.apache.texera.service.util.{ + JupyterEndpointResolver, + JupyterEndpoints, + JupyterProbe, + JupyterProvisioner +} object NotebookMigrationResource extends LazyLogging { @@ -58,6 +63,8 @@ object NotebookMigrationResource extends LazyLogging { mapper.createObjectNode().put("success", true).put("deleted", deleted) ) + // Also the answer when the user has no Jupyter provisioned: either way there is no + // server for them to reach. private def jupyterUnavailableResponse: Response = Response .status(500) @@ -106,50 +113,14 @@ object NotebookMigrationResource extends LazyLogging { } } - // The Jupyter server a request targets. internalUrl is what this service calls, publicUrl - // is what the browser loads; they differ once Jupyter is containerized, since the - // in-network name does not resolve from the browser. Passed per call so the two can be - // made distinct, and so per-user resolution (#7665) can build one of these per uid. - final case class JupyterEndpoints(internalUrl: String, publicUrl: String, token: String) - - // Configured default. Process-wide, so this service still targets one Jupyter per process - // (the per-user-pod model) and must not be deployed as a shared global instance yet: every - // user would get the same Jupyter and token. Per-user resolution is #7665. - private val configuredEndpoints = JupyterEndpoints( - StorageConfig.jupyterInternalURL, - StorageConfig.jupyterPublicURL, - StorageConfig.jupyterToken - ) - // Default notebook name used when a request does not specify one, so a param-less // getJupyterIframeURL call reproduces the URL from before this service became stateless. private val defaultNotebookName = "notebook.ipynb" - private def isJupyterAvailable(jupyterUrl: String): Boolean = { - var conn: java.net.HttpURLConnection = null - try { - conn = new java.net.URL(s"$jupyterUrl/api") - .openConnection() - .asInstanceOf[java.net.HttpURLConnection] - - conn.setRequestMethod("GET") - conn.setConnectTimeout(2000) - conn.setReadTimeout(2000) - - val status = conn.getResponseCode - - status == 200 || status == 403 - } catch { - case _: Exception => false - } finally { - if (conn != null) conn.disconnect() - } - } - // Returns the Jupyter iframe reference URL for the given notebook. def getJupyterIframeURL( notebookName: String, - jupyter: JupyterEndpoints = configuredEndpoints + jupyter: JupyterEndpoints = JupyterEndpoints.configured ): Response = { // notebookName flows into the returned URL, so validate it the same way setNotebook does: // block path traversal and keep it to a plain .ipynb filename. @@ -160,7 +131,7 @@ object NotebookMigrationResource extends LazyLogging { .build() } - if (!isJupyterAvailable(jupyter.internalUrl)) { + if (!JupyterProbe.isAvailable(jupyter.internalUrl)) { return jupyterUnavailableResponse } @@ -172,8 +143,8 @@ object NotebookMigrationResource extends LazyLogging { } // Returns the URL of Jupyter - def getJupyterURL(jupyter: JupyterEndpoints = configuredEndpoints): Response = { - if (!isJupyterAvailable(jupyter.internalUrl)) { + def getJupyterURL(jupyter: JupyterEndpoints = JupyterEndpoints.configured): Response = { + if (!JupyterProbe.isAvailable(jupyter.internalUrl)) { return jupyterUnavailableResponse } @@ -181,7 +152,10 @@ object NotebookMigrationResource extends LazyLogging { } // Set the notebook in Jupyter - def setNotebook(body: String, jupyter: JupyterEndpoints = configuredEndpoints): Response = { + def setNotebook( + body: String, + jupyter: JupyterEndpoints = JupyterEndpoints.configured + ): Response = { var conn: HttpURLConnection = null try { val json = parseBody(body) match { @@ -202,7 +176,7 @@ object NotebookMigrationResource extends LazyLogging { .build() } - if (!isJupyterAvailable(jupyter.internalUrl)) { + if (!JupyterProbe.isAvailable(jupyter.internalUrl)) { return jupyterUnavailableResponse } @@ -271,7 +245,10 @@ object NotebookMigrationResource extends LazyLogging { } // Delete the notebook file from Jupyter's work/ directory: - def deleteNotebook(body: String, jupyter: JupyterEndpoints = configuredEndpoints): Response = { + def deleteNotebook( + body: String, + jupyter: JupyterEndpoints = JupyterEndpoints.configured + ): Response = { var conn: HttpURLConnection = null try { val json = parseBody(body) match { @@ -290,7 +267,7 @@ object NotebookMigrationResource extends LazyLogging { .build() } - if (!isJupyterAvailable(jupyter.internalUrl)) { + if (!JupyterProbe.isAvailable(jupyter.internalUrl)) { return jupyterUnavailableResponse } @@ -566,6 +543,27 @@ object NotebookMigrationResource extends LazyLogging { @Consumes(Array(MediaType.APPLICATION_JSON)) class NotebookMigrationResource extends LazyLogging { + // Runs `call` against the caller's own Jupyter, starting one if they have none. The uid + // comes from the authenticated session, so a request cannot name another user's server. + private def withNewJupyter(user: SessionUser)(call: JupyterEndpoints => Response): Response = + respondWith(JupyterProvisioner.ensure(user.getUid), call) + + // As above, but never starts a pod: reading a URL or deleting a file should not bring a + // Jupyter into existence for a user who has none. + private def withJupyter(user: SessionUser)(call: JupyterEndpoints => Response): Response = + respondWith(JupyterEndpointResolver.resolve(user.getUid), call) + + // Visible to the spec so the no-Jupyter branch can be driven directly: the two callers + // above resolve against live configuration, which a test cannot flip. + private[resource] def respondWith( + jupyter: Option[JupyterEndpoints], + call: JupyterEndpoints => Response + ): Response = + jupyter match { + case Some(endpoints) => call(endpoints) + case None => NotebookMigrationResource.jupyterUnavailableResponse + } + @GET @Path("/get-jupyter-iframe-url") def getJupyterIframeURL( @@ -576,28 +574,30 @@ class NotebookMigrationResource extends LazyLogging { val name = Option(notebookName) .filter(_.nonEmpty) .getOrElse(NotebookMigrationResource.defaultNotebookName) - NotebookMigrationResource.getJupyterIframeURL(name) + withNewJupyter(user) { jupyter => + NotebookMigrationResource.getJupyterIframeURL(name, jupyter) + } } @GET @Path("/get-jupyter-url") def getJupyterURL(@Auth user: SessionUser): Response = { logger.info("Getting Jupyter API URL") - NotebookMigrationResource.getJupyterURL() + withJupyter(user)(NotebookMigrationResource.getJupyterURL) } @POST @Path("/set-notebook") def setNotebook(body: String, @Auth user: SessionUser): Response = { logger.info("Setting notebook") - NotebookMigrationResource.setNotebook(body) + withNewJupyter(user)(NotebookMigrationResource.setNotebook(body, _)) } @POST @Path("/delete-notebook") def deleteNotebook(body: String, @Auth user: SessionUser): Response = { logger.info("Deleting notebook from Jupyter") - NotebookMigrationResource.deleteNotebook(body) + withJupyter(user)(NotebookMigrationResource.deleteNotebook(body, _)) } @POST diff --git a/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterEndpointResolver.scala b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterEndpointResolver.scala new file mode 100644 index 0000000000..b4223a8c6e --- /dev/null +++ b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterEndpointResolver.scala @@ -0,0 +1,61 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import org.apache.texera.common.config.{KubernetesConfig, StorageConfig} +import org.apache.texera.dao.SqlServer +import org.apache.texera.dao.jooq.generated.tables.daos.UserJupyterDao + +/** + * Maps a user to the Jupyter their requests should reach. + * + * The uid always comes from the authenticated session, never from a request body, so one user + * can never address another's Jupyter. + */ +object JupyterEndpointResolver { + + /** + * Endpoints for the user's Jupyter, or None when they have none. + * + * With per-user Jupyter off, every user resolves to the statically configured server: that + * is how the single-node and local-dev deployments run one shared JupyterLab. With it on, a + * user with no registry row has nothing provisioned yet, and falling back to the shared + * server would hand them somebody else's notebooks. + */ + def resolve( + uid: Int, + jupyterEnabled: Boolean = KubernetesConfig.jupyterEnabled, + fallback: JupyterEndpoints = JupyterEndpoints.configured, + tokenSecret: String = StorageConfig.jupyterTokenSecret + ): Option[JupyterEndpoints] = + if (!jupyterEnabled) Some(fallback) + else + registrationOf(uid).map(row => + // The token is derived rather than stored, so it is rebuilt here from the uid. + JupyterEndpoints( + row.getInternalUrl, + row.getPublicUrl, + JupyterTokenDeriver.derive(uid, tokenSecret) + ) + ) + + private def registrationOf(uid: Int) = { + val dao = new UserJupyterDao(SqlServer.getInstance().createDSLContext().configuration()) + Option(dao.fetchOneByUid(uid)) + } +} diff --git a/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterEndpoints.scala b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterEndpoints.scala new file mode 100644 index 0000000000..5f5ea653b8 --- /dev/null +++ b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterEndpoints.scala @@ -0,0 +1,37 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import org.apache.texera.common.config.StorageConfig + +/** + * The Jupyter server a request targets. internalUrl is what the service calls, publicUrl is + * what the browser loads; they differ once Jupyter is containerized, since the in-network + * name does not resolve from the browser. + */ +final case class JupyterEndpoints(internalUrl: String, publicUrl: String, token: String) + +object JupyterEndpoints { + + // The single Jupyter from static config, used while per-user provisioning is off. + val configured: JupyterEndpoints = JupyterEndpoints( + StorageConfig.jupyterInternalURL, + StorageConfig.jupyterPublicURL, + StorageConfig.jupyterToken + ) +} diff --git a/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterKubernetesClient.scala b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterKubernetesClient.scala new file mode 100644 index 0000000000..426912516c --- /dev/null +++ b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterKubernetesClient.scala @@ -0,0 +1,113 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import io.fabric8.kubernetes.api.model.{ + EnvVarBuilder, + Pod, + PodBuilder, + Quantity, + ResourceRequirementsBuilder +} +import io.fabric8.kubernetes.client.KubernetesClientBuilder +import org.apache.texera.common.config.KubernetesConfig + +/** + * Thin wrapper over the fabric8 client for per-user JupyterLab pods, mirroring the computing + * unit's KubernetesClient. The fabric8 client is a constructor parameter rather than a global + * so tests can exercise the naming and addressing without a live cluster. + */ +class JupyterKubernetesClient(client: io.fabric8.kubernetes.client.KubernetesClient) { + + private val namespace: String = KubernetesConfig.jupyterNamespace + private val podNamePrefix = "jupyter" + + def generatePodName(uid: Int): String = s"$podNamePrefix-$uid" + + /** The in-cluster address of a user's pod, resolvable via the headless service. */ + def generatePodURI(uid: Int): String = + s"${generatePodName(uid)}.${KubernetesConfig.jupyterServiceName}.$namespace.svc.cluster.local:${KubernetesConfig.jupyterPortNumber}" + + def podExists(uid: Int): Boolean = getPodByName(generatePodName(uid)).isDefined + + def getPodByName(podName: String): Option[Pod] = + Option(client.pods().inNamespace(namespace).withName(podName).get()) + + /** + * Starts a user's JupyterLab. The token is passed as JUPYTER_TOKEN, which is what the image's + * start-texera-jupyter.sh reads, so each pod ends up with its owner's token and no other. + * Hostname and subdomain are what make generatePodURI resolve. + */ + def createPod(uid: Int, token: String): Pod = { + val podName = generatePodName(uid) + + val resources = new ResourceRequirementsBuilder() + .addToLimits("cpu", new Quantity(KubernetesConfig.jupyterCpuLimit)) + .addToLimits("memory", new Quantity(KubernetesConfig.jupyterMemoryLimit)) + .build() + + val pod = new PodBuilder() + .withNewMetadata() + .withName(podName) + .withNamespace(namespace) + .addToLabels("type", "jupyter") + .addToLabels("uid", uid.toString) + .addToLabels("name", podName) + .endMetadata() + .withNewSpec() + .addNewContainer() + .withName("jupyter") + .withImage(KubernetesConfig.jupyterImageName) + .withImagePullPolicy(KubernetesConfig.computingUnitImagePullPolicy) + .addNewPort() + .withContainerPort(KubernetesConfig.jupyterPortNumber) + .endPort() + .withEnv( + new EnvVarBuilder().withName("JUPYTER_TOKEN").withValue(token).build(), + // Drives the pod's iframe CSP frame-ancestors and its postMessage origin check. + // Empty is ignored by the image, which then keeps its local-development default. + new EnvVarBuilder() + .withName("TEXERA_ORIGIN") + .withValue(KubernetesConfig.jupyterTexeraOrigin) + .build() + ) + .withResources(resources) + .endContainer() + .withHostname(podName) + .withSubdomain(KubernetesConfig.jupyterServiceName) + .endSpec() + .build() + + client.resource(pod).inNamespace(namespace).create() + } + + // Grace period 0: a discarded pod holds nothing worth flushing, and a rebuild cannot claim + // the name until the old pod is really gone. + def deletePod(uid: Int): Unit = + client.pods().inNamespace(namespace).withName(generatePodName(uid)).withGracePeriod(0).delete() +} + +object JupyterKubernetesClient { + + /** + * Built on demand rather than at object initialisation: the single-node and local-dev + * deployments have no cluster to build a client against, and never provision. + */ + def inCluster: JupyterKubernetesClient = + new JupyterKubernetesClient(new KubernetesClientBuilder().build()) +} diff --git a/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterProbe.scala b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterProbe.scala new file mode 100644 index 0000000000..932e41509f --- /dev/null +++ b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterProbe.scala @@ -0,0 +1,72 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import java.net.{HttpURLConnection, URL} + +import scala.util.control.NonFatal + +/** Liveness check for a Jupyter server. */ +object JupyterProbe { + + private val timeoutMillis = 2000 + + /** + * Whether Jupyter on `internalUrl` accepts `token`. Distinct from isAvailable: a pod + * started with a superseded token is still perfectly alive, so liveness alone cannot + * tell a healthy pod from one whose token no longer matches the derived value. Probes + * an endpoint that requires authentication, so 403 means the token is wrong. + */ + def isAuthorized(internalUrl: String, token: String): Boolean = { + var conn: HttpURLConnection = null + try { + conn = new URL(s"$internalUrl/api/contents") + .openConnection() + .asInstanceOf[HttpURLConnection] + conn.setRequestMethod("GET") + conn.setRequestProperty("Authorization", s"token $token") + conn.setConnectTimeout(timeoutMillis) + conn.setReadTimeout(timeoutMillis) + conn.getResponseCode == 200 + } catch { + case NonFatal(_) => false + } finally { + if (conn != null) conn.disconnect() + } + } + + /** + * Whether Jupyter answers on `internalUrl`. /api returns the server version without a + * token, so 403 counts as reachable: the server is up and merely refusing the request. + */ + def isAvailable(internalUrl: String): Boolean = { + var conn: HttpURLConnection = null + try { + conn = new URL(s"$internalUrl/api").openConnection().asInstanceOf[HttpURLConnection] + conn.setRequestMethod("GET") + conn.setConnectTimeout(timeoutMillis) + conn.setReadTimeout(timeoutMillis) + val status = conn.getResponseCode + status == 200 || status == 403 + } catch { + case NonFatal(_) => false + } finally { + if (conn != null) conn.disconnect() + } + } +} diff --git a/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterProvisioner.scala b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterProvisioner.scala new file mode 100644 index 0000000000..2ba3ee0a18 --- /dev/null +++ b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterProvisioner.scala @@ -0,0 +1,209 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import com.typesafe.scalalogging.LazyLogging +import io.fabric8.kubernetes.client.KubernetesClientException +import org.apache.texera.common.config.{KubernetesConfig, StorageConfig} +import org.apache.texera.dao.SqlServer +import org.apache.texera.dao.jooq.generated.tables.daos.UserJupyterDao +import org.apache.texera.dao.jooq.generated.tables.pojos.UserJupyter +import org.jooq.exception.DataAccessException + +import java.util.concurrent.Semaphore +import scala.util.control.NonFatal + +/** + * Brings a user's JupyterLab into existence and registers where it lives. + * + * Dependencies are constructor parameters so the provisioning logic can be tested without a + * cluster; the companion object binds the production ones. + */ +class JupyterProvisioner( + kubernetesClient: => JupyterKubernetesClient, + accepts: (String, String) => Boolean, + publicUrlTemplate: String, + readinessTimeoutMillis: Long, + readinessPollMillis: Long, + maxConcurrentProvisions: Int +) extends LazyLogging { + + // By-name above, forced once here, so no client is built unless a provision happens. + private lazy val kubernetes = kubernetesClient + + // Provisioning parks the calling request thread for up to readinessTimeoutMillis, so a burst + // of first-time users could otherwise drain the HTTP worker pool. Requests past the cap are + // told to retry rather than queued behind a full readiness wait. + private val provisionPermits = new Semaphore(maxConcurrentProvisions) + + /** + * The user's Jupyter, starting one if they have none. None means it could not be made + * ready, which callers report the same as an unreachable server. + * + * A registered pod that no longer answers is discarded and rebuilt: the row would otherwise + * outlive the pod and point every later request at nothing. + */ + def ensure( + uid: Int, + jupyterEnabled: Boolean = KubernetesConfig.jupyterEnabled, + fallback: JupyterEndpoints = JupyterEndpoints.configured, + tokenSecret: String = StorageConfig.jupyterTokenSecret + ): Option[JupyterEndpoints] = { + if (!jupyterEnabled) return Some(fallback) + + val token = JupyterTokenDeriver.derive(uid, tokenSecret) + JupyterEndpointResolver.resolve(uid, jupyterEnabled = true, tokenSecret = tokenSecret) match { + case Some(endpoints) if accepts(endpoints.internalUrl, token) => Some(endpoints) + case Some(endpoints) => + // Either the pod is gone, or it predates a token-secret rotation and no longer + // accepts the derived token. Both are unusable and both are fixed by rebuilding. + logger.warn( + s"Jupyter for user $uid is registered at ${endpoints.internalUrl} but does not " + + "accept its current token; rebuilding it" + ) + withPermit(uid) { + if (discard(uid)) provision(uid, token) + else { + // The name is still taken, so a create would be skipped and the rebuild would poll + // the dying pod to the readiness deadline. Fail now and say why. + logger.error(s"Stale Jupyter pod for user $uid did not terminate; not rebuilding") + None + } + } + case None => withPermit(uid)(provision(uid, token)) + } + } + + // None means the cap is reached, which callers report the same as an unreachable server; the + // fast path never gets here, so a user with a working pod is unaffected by the limit. + private def withPermit(uid: Int)(work: => Option[JupyterEndpoints]): Option[JupyterEndpoints] = + if (!provisionPermits.tryAcquire()) { + logger.warn( + s"$maxConcurrentProvisions Jupyter provisions already in flight; " + + s"refusing user $uid for now" + ) + None + } else { + try work + finally provisionPermits.release() + } + + private def provision(uid: Int, token: String): Option[JupyterEndpoints] = { + val internalUrl = s"http://${kubernetes.generatePodURI(uid)}" + val endpoints = JupyterEndpoints(internalUrl, publicUrlFor(uid, internalUrl), token) + try { + createIfAbsent(uid, token) + if (!waitUntilAccepting(internalUrl, token)) { + logger.error(s"Jupyter for user $uid did not become ready; removing the pod") + kubernetes.deletePod(uid) + None + } else { + register(endpoints, uid) + Some(endpoints) + } + } catch { + case NonFatal(e) => + logger.error(s"Failed to provision Jupyter for user $uid", e) + None + } + } + + // Two concurrent first requests can both find the pod absent and both create it. The + // loser's create returns 409, and the winner's pod is the one it wanted anyway, since both + // derive the same token from the same uid, so the conflict is a success. Mirrors + // register()'s handling of a duplicate row. + private def createIfAbsent(uid: Int, token: String): Unit = { + if (kubernetes.podExists(uid)) return + try kubernetes.createPod(uid, token) + catch { + case e: KubernetesClientException if e.getCode == 409 => + logger.info(s"Jupyter pod for user $uid was created concurrently; using that one") + } + } + + /** Browser-facing address; the in-cluster name does not resolve from the browser. */ + private def publicUrlFor(uid: Int, internalUrl: String): String = + if (publicUrlTemplate.isEmpty) internalUrl + else publicUrlTemplate.replace("{uid}", uid.toString) + + // Waits for the pod to accept its own token rather than merely to answer, so a pod that + // starts with the wrong token never gets registered. + private def waitUntilAccepting(internalUrl: String, token: String): Boolean = { + val deadline = System.currentTimeMillis() + readinessTimeoutMillis + var ready = accepts(internalUrl, token) + while (!ready && System.currentTimeMillis() < deadline) { + Thread.sleep(readinessPollMillis) + ready = accepts(internalUrl, token) + } + ready + } + + private def register(endpoints: JupyterEndpoints, uid: Int): Unit = { + val row = new UserJupyter + row.setUid(uid) + row.setInternalUrl(endpoints.internalUrl) + row.setPublicUrl(endpoints.publicUrl) + try dao().insert(row) + catch { + // Two concurrent first requests can both provision. uid is the primary key, so the + // loser trips 23505; the winner's row holds the same uid-derived addresses, so leaving + // it in place is correct. + case e: DataAccessException if e.sqlState == "23505" => + logger.info(s"Jupyter for user $uid was registered concurrently; keeping that row") + } + } + + /** Removes the pod and its row. False means the pod outlived the wait, so no rebuild. */ + private def discard(uid: Int): Boolean = { + // A pod that is already gone throws here, which is the state the delete wanted; only the + // wait below decides whether the name is free. + try kubernetes.deletePod(uid) + catch { case NonFatal(e) => logger.warn(s"Could not delete stale Jupyter pod for $uid", e) } + val gone = waitUntilGone(uid) + dao().deleteById(uid) + gone + } + + // Deletion is asynchronous even at grace period 0, and podExists counts a Terminating pod as + // present, so returning early would make the rebuild skip its create. + private def waitUntilGone(uid: Int): Boolean = { + val deadline = System.currentTimeMillis() + readinessTimeoutMillis + var exists = kubernetes.podExists(uid) + while (exists && System.currentTimeMillis() < deadline) { + Thread.sleep(readinessPollMillis) + exists = kubernetes.podExists(uid) + } + !exists + } + + private def dao() = new UserJupyterDao(SqlServer.getInstance().createDSLContext().configuration()) +} + +// A pod is scheduled, pulled and started before Jupyter answers, so the first request after +// provisioning waits rather than failing. +object JupyterProvisioner + extends JupyterProvisioner( + JupyterKubernetesClient.inCluster, + JupyterProbe.isAuthorized, + KubernetesConfig.jupyterPublicUrlTemplate, + readinessTimeoutMillis = 60000, + readinessPollMillis = 1000, + // Far below Dropwizard's 1024 worker threads, so a provisioning burst cannot starve the + // rest of the service, while still admitting a classroom's worth of concurrent logins. + maxConcurrentProvisions = 16 + ) diff --git a/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterTokenDeriver.scala b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterTokenDeriver.scala new file mode 100644 index 0000000000..ea6b5135e7 --- /dev/null +++ b/notebook-migration-service/src/main/scala/org/apache/texera/service/util/JupyterTokenDeriver.scala @@ -0,0 +1,59 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import org.apache.texera.common.config.{KubernetesConfig, StorageConfig} + +import javax.crypto.Mac +import javax.crypto.spec.SecretKeySpec +import java.nio.charset.StandardCharsets.UTF_8 + +/** + * Derives each user's JupyterLab token from a server-held secret instead of storing one. + * The value is stable for a uid until the secret changes, so any replica of this service + * derives the same token and no credential is kept at rest. + */ +object JupyterTokenDeriver { + + private val algorithm = "HmacSHA256" + + // 128 bits of the digest, which is ample for a token and keeps the URL short. + private val tokenLength = 32 + + def derive(uid: Int, secret: String = StorageConfig.jupyterTokenSecret): String = { + require(secret.nonEmpty, "cannot derive a Jupyter token from an empty secret") + val mac = Mac.getInstance(algorithm) + mac.init(new SecretKeySpec(secret.getBytes(UTF_8), algorithm)) + mac.doFinal(uid.toString.getBytes(UTF_8)).map("%02x".format(_)).mkString.take(tokenLength) + } + + /** + * Refuses to start per-user Jupyter without a secret: an empty key is public, so anyone + * could derive another user's token. Only enforced when the feature is on, so the + * single-node and local-dev deployments are unaffected. + */ + def validateConfiguration( + jupyterEnabled: Boolean = KubernetesConfig.jupyterEnabled, + secret: String = StorageConfig.jupyterTokenSecret + ): Unit = + if (jupyterEnabled && secret.isEmpty) { + throw new IllegalStateException( + "kubernetes.jupyter-enabled requires a non-empty storage.jupyter.token-secret" + ) + } +} diff --git a/notebook-migration-service/src/test/scala/org/apache/texera/service/resource/NotebookMigrationResourceSpec.scala b/notebook-migration-service/src/test/scala/org/apache/texera/service/resource/NotebookMigrationResourceSpec.scala index 10d5829b87..66cbb4406a 100644 --- a/notebook-migration-service/src/test/scala/org/apache/texera/service/resource/NotebookMigrationResourceSpec.scala +++ b/notebook-migration-service/src/test/scala/org/apache/texera/service/resource/NotebookMigrationResourceSpec.scala @@ -21,22 +21,33 @@ package org.apache.texera.service.resource import jakarta.ws.rs.core.Response import org.apache.texera.auth.SessionUser +import io.fabric8.kubernetes.client.KubernetesClientException import org.apache.texera.dao.MockTexeraDB +import org.apache.texera.service.util.{ + JupyterEndpointResolver, + JupyterEndpoints, + JupyterKubernetesClient, + JupyterProvisioner, + JupyterTokenDeriver +} import org.apache.texera.dao.jooq.generated.enums.{PrivilegeEnum, UserRoleEnum} import org.apache.texera.dao.jooq.generated.tables.Notebook.NOTEBOOK import org.apache.texera.dao.jooq.generated.tables.User.USER +import org.apache.texera.dao.jooq.generated.tables.UserJupyter.USER_JUPYTER import org.apache.texera.dao.jooq.generated.tables.Workflow.WORKFLOW import org.apache.texera.dao.jooq.generated.tables.WorkflowNotebookMapping.WORKFLOW_NOTEBOOK_MAPPING import org.apache.texera.dao.jooq.generated.tables.WorkflowUserAccess.WORKFLOW_USER_ACCESS import org.apache.texera.dao.jooq.generated.tables.WorkflowVersion.WORKFLOW_VERSION import org.apache.texera.dao.jooq.generated.tables.daos.{ UserDao, + UserJupyterDao, WorkflowDao, WorkflowUserAccessDao, WorkflowVersionDao } import org.apache.texera.dao.jooq.generated.tables.pojos.{ User, + UserJupyter, Workflow, WorkflowUserAccess, WorkflowVersion @@ -49,6 +60,8 @@ import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach} import com.sun.net.httpserver.HttpServer import java.net.InetSocketAddress + +import scala.jdk.CollectionConverters._ import java.sql.Timestamp import java.util.UUID @@ -152,6 +165,7 @@ class NotebookMigrationResourceSpec .where(WORKFLOW_VERSION.WID.eq(testWid)) .execute() getDSLContext.deleteFrom(WORKFLOW).where(WORKFLOW.WID.eq(testWid)).execute() + getDSLContext.deleteFrom(USER_JUPYTER).execute() getDSLContext.deleteFrom(USER).where(USER.EMAIL.in(writerEmail, readerEmail)).execute() } @@ -543,12 +557,432 @@ class NotebookMigrationResourceSpec // gets. 192.0.2.0/24 is TEST-NET-1 (RFC 5737) and routes nowhere, so a call that wrongly // dials the public URL fails rather than silently passing. Numeric on purpose: a hostname // would go through the resolver, which setConnectTimeout does not bound. - private val splitEndpoints = NotebookMigrationResource.JupyterEndpoints( + private val splitEndpoints = JupyterEndpoints( internalUrl = "http://localhost:9100";, publicUrl = "http://192.0.2.1:1234";, token = "texera" ) + // -- per-user resolution ---------------------------------------------------- + + // Registers a Jupyter for `uid`, standing in for a provisioned pod. + private def registerJupyter( + uid: Integer, + internalUrl: String = "http://localhost:9100";, + publicUrl: String = "http://192.0.2.1:1234"; + ): Unit = { + val row = new UserJupyter + row.setUid(uid) + row.setInternalUrl(internalUrl) + row.setPublicUrl(publicUrl) + new UserJupyterDao(getDSLContext.configuration()).insert(row) + } + + private val specSecret = "resolver-spec-secret" + + "JupyterEndpointResolver" should "resolve every user to the configured Jupyter while the feature is off" in { + // How single-node and local dev run: one shared JupyterLab, no registry rows. + JupyterEndpointResolver.resolve(writerUid, jupyterEnabled = false) shouldBe Some( + JupyterEndpoints.configured + ) + } + + it should "resolve a registered user to their own Jupyter" in { + registerJupyter(writerUid, internalUrl = "http://jupyter-1:8888";) + val resolved = + JupyterEndpointResolver.resolve(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + resolved.map(_.internalUrl) shouldBe Some("http://jupyter-1:8888";) + resolved.map(_.publicUrl) shouldBe Some("http://192.0.2.1:1234";) + } + + it should "return None for an unregistered user rather than falling back to the shared Jupyter" in { + // The isolation property: falling back here would hand an unprovisioned user somebody + // else's notebooks, which is the whole point of resolving per user. + JupyterEndpointResolver.resolve( + writerUid, + jupyterEnabled = true, + tokenSecret = specSecret + ) shouldBe None + } + + it should "never return one user's Jupyter to another" in { + registerJupyter(writerUid, internalUrl = "http://jupyter-writer:8888";) + JupyterEndpointResolver + .resolve(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + .map(_.internalUrl) shouldBe Some("http://jupyter-writer:8888";) + JupyterEndpointResolver.resolve( + readerUid, + jupyterEnabled = true, + tokenSecret = specSecret + ) shouldBe None + } + + it should "derive the registered user's token rather than reading one from the row" in { + // No token column exists, so the resolver has to rebuild it from the uid. + registerJupyter(writerUid) + JupyterEndpointResolver + .resolve(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + .map(_.token) shouldBe Some(JupyterTokenDeriver.derive(writerUid, specSecret)) + } + + it should "give two registered users different tokens" in { + registerJupyter(writerUid) + registerJupyter(readerUid) + val writerToken = JupyterEndpointResolver + .resolve(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + .map(_.token) + val readerToken = JupyterEndpointResolver + .resolve(readerUid, jupyterEnabled = true, tokenSecret = specSecret) + .map(_.token) + writerToken should not be readerToken + } + + "the resource class" should "report Jupyter unavailable when the caller has none" in { + // What an unprovisioned user gets: never a fall back to somebody else's Jupyter. + val response = new NotebookMigrationResource() + .respondWith(None, _ => fail("must not call through without a Jupyter")) + response.getStatus shouldBe 500 + response.getEntity.toString should include("Cannot connect to Jupyter server") + } + + it should "call through to the endpoint when the caller has a Jupyter" in { + val response = new NotebookMigrationResource() + .respondWith(Some(splitEndpoints), jupyter => Response.ok(jupyter.internalUrl).build()) + response.getEntity.toString shouldBe "http://localhost:9100"; + } + + // -- provisioning ----------------------------------------------------------- + + // Records what would have been asked of Kubernetes, so the provisioning logic runs without + // a cluster. Subclassing rather than mocking keeps the real naming and addressing. + private class StubKubernetes extends JupyterKubernetesClient(null) { + var created: List[(Int, String)] = Nil + var deleted: List[Int] = Nil + var alreadyExists = false + var failCreate = false + var failDelete = false + // Distinct from failCreate: 409 means another request won the race, which is success. + var createConflicts = false + // Polls the pod must survive after deletePod before it reports gone, so the spec can model + // the Terminating window a real cluster has. 0 deletes synchronously. + var terminatingPolls = 0 + override def podExists(uid: Int): Boolean = + if (terminatingPolls > 0) { terminatingPolls -= 1; true } + else alreadyExists + override def createPod(uid: Int, token: String) = { + if (createConflicts) throw new KubernetesClientException("already exists", 409, null) + if (failCreate) throw new RuntimeException("cluster refused the pod") + created ::= ((uid, token)) + null + } + override def deletePod(uid: Int): Unit = { + deleted ::= uid + if (failDelete) throw new RuntimeException("pod already gone") + alreadyExists = false + } + } + + // Short windows so the "never ready" path does not sit in a real timeout. + private def provisionerFor( + kubernetes: JupyterKubernetesClient, + accepts: (String, String) => Boolean, + publicUrlTemplate: String = "", + maxConcurrentProvisions: Int = 4 + ) = + new JupyterProvisioner( + kubernetes, + accepts, + publicUrlTemplate, + readinessTimeoutMillis = 50, + readinessPollMillis = 10, + maxConcurrentProvisions + ) + + private def registeredUids(): List[Integer] = + getDSLContext + .select(USER_JUPYTER.UID) + .from(USER_JUPYTER) + .fetchInto(classOf[Integer]) + .asScala + .toList + + "JupyterProvisioner.ensure" should "return the configured Jupyter and start nothing while the feature is off" in { + val kubernetes = new StubKubernetes + val result = + provisionerFor(kubernetes, (_, _) => true).ensure(writerUid, jupyterEnabled = false) + result shouldBe Some(JupyterEndpoints.configured) + kubernetes.created shouldBe empty + registeredUids() shouldBe empty + } + + it should "start and register a Jupyter for a user who has none" in { + val kubernetes = new StubKubernetes + val result = provisionerFor(kubernetes, (_, _) => true) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + kubernetes.created.map(_._1) shouldBe List(writerUid.intValue()) + registeredUids() shouldBe List(writerUid) + result.map(_.internalUrl) shouldBe Some(s"http://${kubernetes.generatePodURI(writerUid)}") + } + + it should "give the pod the user's own derived token" in { + // What makes one user's token useless against another's Jupyter. + val kubernetes = new StubKubernetes + provisionerFor(kubernetes, (_, _) => true) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + kubernetes.created.map(_._2) shouldBe List(JupyterTokenDeriver.derive(writerUid, specSecret)) + } + + it should "reuse a registered Jupyter that still answers" in { + registerJupyter(writerUid) + val kubernetes = new StubKubernetes + val result = provisionerFor(kubernetes, (_, _) => true) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + kubernetes.created shouldBe empty + result.map(_.internalUrl) shouldBe Some("http://localhost:9100";) + } + + it should "rebuild a registered Jupyter whose pod is gone" in { + // The row would otherwise outlive the pod and point every later request at nothing. + registerJupyter(writerUid, internalUrl = "http://stale:8888";) + val kubernetes = new StubKubernetes + val result = provisionerFor(kubernetes, (url, _) => url != "http://stale:8888";) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + kubernetes.deleted shouldBe List(writerUid.intValue()) + kubernetes.created.map(_._1) shouldBe List(writerUid.intValue()) + result.map(_.internalUrl) shouldBe Some(s"http://${kubernetes.generatePodURI(writerUid)}") + } + + it should "register nothing and clean up when the pod never becomes ready" in { + val kubernetes = new StubKubernetes + val result = provisionerFor(kubernetes, (_, _) => false) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + result shouldBe None + kubernetes.deleted shouldBe List(writerUid.intValue()) + registeredUids() shouldBe empty + } + + it should "build the public URL from the configured template" in { + val kubernetes = new StubKubernetes + val result = + provisionerFor(kubernetes, (_, _) => true, "https://texera.example.com/jupyter/{uid}";) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + result.map(_.publicUrl) shouldBe Some(s"https://texera.example.com/jupyter/$writerUid";) + } + + it should "adopt an existing pod instead of creating a second one" in { + // A pod can outlive its row, so provisioning must be idempotent on the Kubernetes side. + val kubernetes = new StubKubernetes + kubernetes.alreadyExists = true + val result = provisionerFor(kubernetes, (_, _) => true) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + kubernetes.created shouldBe empty + result should not be empty + registeredUids() shouldBe List(writerUid) + } + + it should "reuse one Kubernetes client across calls" in { + // The client is built lazily and held, so a second request must not construct another. + val kubernetes = new StubKubernetes + val provisioner = provisionerFor(kubernetes, (_, _) => true) + provisioner.ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + provisioner.ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + // Second call finds the row it just wrote, so it provisions once in total. + kubernetes.created.map(_._1) shouldBe List(writerUid.intValue()) + registeredUids() shouldBe List(writerUid) + } + + it should "refuse to provision once the concurrency cap is reached" in { + // Every provision parks its request thread on the readiness wait, so the cap is what keeps + // a burst of first-time users from draining the HTTP worker pool. + val kubernetes = new StubKubernetes + val result = provisionerFor(kubernetes, (_, _) => true, maxConcurrentProvisions = 0) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + result shouldBe empty + kubernetes.created shouldBe empty + registeredUids() shouldBe empty + } + + it should "release its permit after provisioning succeeds" in { + // A permit leak would wedge the service after maxConcurrentProvisions successful starts. + // A second user, so this provisions twice rather than taking the fast path. + val kubernetes = new StubKubernetes + val provisioner = provisionerFor(kubernetes, (_, _) => true, maxConcurrentProvisions = 1) + provisioner.ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + provisioner + .ensure(readerUid, jupyterEnabled = true, tokenSecret = specSecret) should not be empty + + kubernetes.created.map(_._1) should contain allOf (writerUid.intValue(), readerUid.intValue()) + } + + it should "release its permit after provisioning fails" in { + // The failure path returns through the same permit, so a cluster refusing pods must not + // burn the cap down. + val kubernetes = new StubKubernetes + kubernetes.failCreate = true + val provisioner = provisionerFor(kubernetes, (_, _) => true, maxConcurrentProvisions = 1) + provisioner.ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) shouldBe empty + + kubernetes.failCreate = false + provisioner + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) should not be empty + } + + it should "not spend a permit on a user whose pod already works" in { + // The fast path never blocks, so the cap must not apply to it. + registerJupyter(writerUid, internalUrl = "http://live:8888";) + val kubernetes = new StubKubernetes + val result = provisionerFor(kubernetes, (_, _) => true, maxConcurrentProvisions = 0) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + result.map(_.internalUrl) shouldBe Some("http://live:8888";) + kubernetes.created shouldBe empty + } + + it should "rebuild a registered pod that no longer accepts its token" in { + // After a token-secret rotation the pod is still alive but holds the superseded token, + // so liveness alone would keep handing out a token the pod rejects. + registerJupyter(writerUid, internalUrl = "http://rotated:8888";) + val kubernetes = new StubKubernetes + val result = provisionerFor(kubernetes, (url, _) => url != "http://rotated:8888";) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + kubernetes.deleted shouldBe List(writerUid.intValue()) + kubernetes.created.map(_._1) shouldBe List(writerUid.intValue()) + result.map(_.internalUrl) shouldBe Some( + s"http://${kubernetes.generatePodURI(writerUid)}" + ) + } + + it should "wait out a discarded pod before creating its replacement" in { + // Deletion is asynchronous, and a Terminating pod still satisfies podExists. Returning + // from discard too early would skip the create and poll the dying pod to the deadline. + registerJupyter(writerUid, internalUrl = "http://rotated:8888";) + val kubernetes = new StubKubernetes + kubernetes.terminatingPolls = 2 + val result = provisionerFor(kubernetes, (url, _) => url != "http://rotated:8888";) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + kubernetes.created.map(_._1) shouldBe List(writerUid.intValue()) + result.map(_.internalUrl) shouldBe Some( + s"http://${kubernetes.generatePodURI(writerUid)}" + ) + } + + it should "not rebuild when the discarded pod never terminates" in { + // A pod stuck Terminating keeps its name, so a create would be skipped. Reporting + // unavailable beats polling a pod that cannot accept the new token. + registerJupyter(writerUid, internalUrl = "http://rotated:8888";) + val kubernetes = new StubKubernetes + kubernetes.terminatingPolls = Int.MaxValue + val result = provisionerFor(kubernetes, (url, _) => url != "http://rotated:8888";) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + kubernetes.deleted shouldBe List(writerUid.intValue()) + kubernetes.created shouldBe empty + result shouldBe empty + } + + it should "pass the derived token to the readiness check, not just the address" in { + // Guards the rotation fix: a probe that ignored the token would accept any live pod. + val kubernetes = new StubKubernetes + var seen: List[String] = Nil + provisionerFor(kubernetes, (_, tok) => { seen ::= tok; true }) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + seen.distinct shouldBe List(JupyterTokenDeriver.derive(writerUid, specSecret)) + } + + it should "succeed when another request created the pod first" in { + // Check-then-act on a uid-keyed pod: two concurrent first requests both find it absent + // and both create. The loser's 409 is the winner's pod, which is the one it wanted. + val kubernetes = new StubKubernetes + kubernetes.createConflicts = true + val result = provisionerFor(kubernetes, (_, _) => true) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + result.map(_.internalUrl) shouldBe Some( + s"http://${kubernetes.generatePodURI(writerUid)}" + ) + kubernetes.deleted shouldBe empty + registeredUids() shouldBe List(writerUid) + } + + it should "report unavailable when the cluster refuses to create the pod" in { + val kubernetes = new StubKubernetes + kubernetes.failCreate = true + val result = provisionerFor(kubernetes, (_, _) => true) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + result shouldBe None + registeredUids() shouldBe empty + } + + it should "still fail on a Kubernetes error that is not a conflict" in { + // Guards the narrowness of the 409 catch: only AlreadyExists means someone else won. + val kubernetes = new StubKubernetes { + override def createPod(uid: Int, token: String) = + throw new KubernetesClientException("quota exceeded", 403, null) + } + provisionerFor(kubernetes, (_, _) => true) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) shouldBe None + registeredUids() shouldBe empty + } + + it should "still rebuild when the stale pod cannot be deleted" in { + // The pod may already be gone, which is the state the delete was trying to reach. + registerJupyter(writerUid, internalUrl = "http://stale:8888";) + val kubernetes = new StubKubernetes + kubernetes.failDelete = true + val result = provisionerFor(kubernetes, (url, _) => url != "http://stale:8888";) + .ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + kubernetes.created.map(_._1) shouldBe List(writerUid.intValue()) + result.map(_.internalUrl) shouldBe Some(s"http://${kubernetes.generatePodURI(writerUid)}") + } + + it should "report unavailable when registration fails for a reason other than a race" in { + // A uid with no user row violates the foreign key. Only a duplicate primary key means + // "another request won"; anything else has to surface rather than be swallowed. + val orphanUid = 999999 + val kubernetes = new StubKubernetes + val result = provisionerFor(kubernetes, (_, _) => true) + .ensure(orphanUid, jupyterEnabled = true, tokenSecret = specSecret) + + result shouldBe None + registeredUids() shouldBe empty + } + + it should "keep the winning row when two requests provision at once" in { + // The readiness probe runs just before the insert, so registering there stands in for a + // concurrent request winning the race. + val kubernetes = new StubKubernetes + val racing = provisionerFor( + kubernetes, + (_, _) => { if (registeredUids().isEmpty) registerJupyter(writerUid); true } + ) + val result = racing.ensure(writerUid, jupyterEnabled = true, tokenSecret = specSecret) + + result should not be empty + registeredUids() shouldBe List(writerUid) + } + + it should "fall back to the configured endpoints when none are passed" in { + // The defaulted parameter is what keeps direct object calls working for callers that + // have no per-user endpoints to hand in. + withFakeJupyter(contentsStatus = 201) { + val response = NotebookMigrationResource.getJupyterURL() + response.getStatus shouldBe Response.Status.OK.getStatusCode + response.getEntity.toString should include(JupyterEndpoints.configured.publicUrl) + } + } + "the internal/public URL split" should "dial the internal URL and return only the public one" in { withFakeJupyter(contentsStatus = 201) { val urlResp = NotebookMigrationResource.getJupyterURL(splitEndpoints) @@ -593,7 +1027,7 @@ class NotebookMigrationResourceSpec // rescue an unreachable internal one. withFakeJupyter(contentsStatus = 201) { // Port 9 on loopback: refused immediately, so this fails fast and without DNS. - val swapped = NotebookMigrationResource.JupyterEndpoints( + val swapped = JupyterEndpoints( internalUrl = "http://127.0.0.1:9";, publicUrl = "http://localhost:9100";, token = "texera" diff --git a/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterEndpointsSpec.scala b/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterEndpointsSpec.scala new file mode 100644 index 0000000000..537c94dad3 --- /dev/null +++ b/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterEndpointsSpec.scala @@ -0,0 +1,60 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import org.apache.texera.common.config.StorageConfig +import org.scalatest.flatspec.AnyFlatSpec +import org.scalatest.matchers.should.Matchers + +class JupyterEndpointsSpec extends AnyFlatSpec with Matchers { + + private val endpoints = JupyterEndpoints("http://internal:8888";, "http://public";, "tok") + + "JupyterEndpoints.configured" should "mirror the static Jupyter configuration" in { + JupyterEndpoints.configured shouldBe JupyterEndpoints( + StorageConfig.jupyterInternalURL, + StorageConfig.jupyterPublicURL, + StorageConfig.jupyterToken + ) + } + + "JupyterEndpoints" should "compare by value" in { + endpoints shouldBe JupyterEndpoints("http://internal:8888";, "http://public";, "tok") + endpoints.hashCode shouldBe + JupyterEndpoints("http://internal:8888";, "http://public";, "tok").hashCode + } + + it should "differ when any field differs" in { + // The token is part of identity: two users share a URL in the fallback case but never a token. + endpoints should not be endpoints.copy(token = "other") + endpoints should not be endpoints.copy(internalUrl = "http://other:8888";) + endpoints should not be endpoints.copy(publicUrl = "http://other";) + } + + it should "not equal a value of another type" in { + endpoints should not be "http://internal:8888"; + endpoints.toString should include("http://internal:8888";) + } + + it should "destructure into its three parts" in { + val JupyterEndpoints(internal, public, token) = endpoints + internal shouldBe "http://internal:8888"; + public shouldBe "http://public"; + token shouldBe "tok" + } +} diff --git a/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterKubernetesClientSpec.scala b/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterKubernetesClientSpec.scala new file mode 100644 index 0000000000..624ff3ebc3 --- /dev/null +++ b/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterKubernetesClientSpec.scala @@ -0,0 +1,210 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import io.fabric8.kubernetes.api.model.{Pod, PodBuilder, PodList} +import io.fabric8.kubernetes.client.dsl.{ + Deletable, + MixedOperation, + NamespaceableResource, + NonNamespaceOperation, + PodResource, + Resource +} +import io.fabric8.kubernetes.client.{ + KubernetesClient => Fabric8Client, + PropagationPolicyConfigurable +} +import org.apache.texera.common.config.KubernetesConfig +import org.mockito.ArgumentCaptor +import org.mockito.Mockito.{doReturn, mock, verify, when} +import org.scalatest.flatspec.AnyFlatSpec +import org.scalatest.matchers.should.Matchers + +import scala.jdk.CollectionConverters._ + +/** + * Two layers: the pure naming and addressing, which is what the registry stores and the + * service dials, and the thin fabric8 wrappers driven through a Mockito-stubbed client so + * the pod spec can be asserted without a live cluster. + */ +class JupyterKubernetesClientSpec extends AnyFlatSpec with Matchers { + + private val namespace = KubernetesConfig.jupyterNamespace + + private val bare = new JupyterKubernetesClient(null) + + // fabric8's fluent API returns type variables, so RETURNS_DEEP_STUBS cannot be used and + // each step of the chain is mocked explicitly. Mirrors the computing unit's spec. + // withGracePeriod hands back a further step in the chain, so delete() lands on that object + // rather than on the PodResource; it is returned so the spec can verify it. + private def stubbedPods( + existing: Pod + ): (Fabric8Client, PodResource, PropagationPolicyConfigurable[Deletable]) = { + val client = mock(classOf[Fabric8Client]) + val mixed = mock(classOf[MixedOperation[_, _, _]]) + .asInstanceOf[MixedOperation[Pod, PodList, PodResource]] + val inNamespace = mock(classOf[NonNamespaceOperation[_, _, _]]) + .asInstanceOf[NonNamespaceOperation[Pod, PodList, PodResource]] + val podResource = mock(classOf[PodResource]) + when(client.pods()).thenReturn(mixed) + when(mixed.inNamespace(namespace)).thenReturn(inNamespace) + when(inNamespace.withName(org.mockito.ArgumentMatchers.anyString())).thenReturn(podResource) + val afterGracePeriod = mock(classOf[PropagationPolicyConfigurable[_]]) + .asInstanceOf[PropagationPolicyConfigurable[Deletable]] + when(podResource.get()).thenReturn(existing) + // doReturn, not when: withGracePeriod's return type is an existential that thenReturn + // cannot be given a name for. + doReturn(afterGracePeriod, Nil: _*).when(podResource).withGracePeriod(0L) + (client, podResource, afterGracePeriod) + } + + // -- naming and addressing -------------------------------------------------- + + "generatePodName" should "namespace the pod by uid" in { + bare.generatePodName(7) shouldBe "jupyter-7" + } + + it should "give every user a distinct pod name" in { + (1 to 50).map(bare.generatePodName).distinct.size shouldBe 50 + } + + "generatePodURI" should "address the pod through the headless service" in { + // Must match the pod's hostname.subdomain, or the name does not resolve in-cluster. + bare.generatePodURI(7) shouldBe + s"jupyter-7.${KubernetesConfig.jupyterServiceName}.$namespace" + + s".svc.cluster.local:${KubernetesConfig.jupyterPortNumber}" + } + + it should "carry the configured port" in { + bare.generatePodURI(7) should endWith(s":${KubernetesConfig.jupyterPortNumber}") + } + + // -- lookups --------------------------------------------------------------- + + "getPodByName" should "return the pod when one exists" in { + val pod = new PodBuilder().withNewMetadata().withName("jupyter-7").endMetadata().build() + val (client, _, _) = stubbedPods(pod) + new JupyterKubernetesClient(client).getPodByName("jupyter-7") shouldBe Some(pod) + } + + it should "return None when the pod is absent" in { + val (client, _, _) = stubbedPods(null) + new JupyterKubernetesClient(client).getPodByName("jupyter-7") shouldBe None + } + + "podExists" should "report true for a live pod and false for a missing one" in { + val pod = new PodBuilder().withNewMetadata().withName("jupyter-7").endMetadata().build() + new JupyterKubernetesClient(stubbedPods(pod)._1).podExists(7) shouldBe true + new JupyterKubernetesClient(stubbedPods(null)._1).podExists(7) shouldBe false + } + + "deletePod" should "delete the user's own pod by name" in { + val (client, podResource, afterGracePeriod) = stubbedPods(null) + new JupyterKubernetesClient(client).deletePod(7) + verify(client.pods().inNamespace(namespace)).withName("jupyter-7") + verify(podResource).withGracePeriod(0L) + verify(afterGracePeriod).delete() + } + + // -- pod spec -------------------------------------------------------------- + + // Captures the pod handed to fabric8, so every field the deployment depends on is asserted. + private def createdPod(uid: Int, token: String): Pod = { + val client = mock(classOf[Fabric8Client]) + val namespaceable = mock(classOf[NamespaceableResource[_]]) + .asInstanceOf[NamespaceableResource[Pod]] + val resource = mock(classOf[Resource[_]]).asInstanceOf[Resource[Pod]] + val captor = ArgumentCaptor.forClass(classOf[Pod]) + when(client.resource(captor.capture())).thenReturn(namespaceable) + when(namespaceable.inNamespace(namespace)).thenReturn(resource) + when(resource.create()).thenReturn(null) + new JupyterKubernetesClient(client).createPod(uid, token) + captor.getValue + } + + "createPod" should "name and namespace the pod for its owner" in { + val pod = createdPod(7, "tok") + pod.getMetadata.getName shouldBe "jupyter-7" + pod.getMetadata.getNamespace shouldBe namespace + } + + it should "label the pod so the headless service and the owner are identifiable" in { + val labels = createdPod(7, "tok").getMetadata.getLabels.asScala + labels("type") shouldBe "jupyter" + labels("uid") shouldBe "7" + labels("name") shouldBe "jupyter-7" + } + + it should "pass the owner's token as JUPYTER_TOKEN" in { + // The image's start-texera-jupyter.sh reads this, so it is what isolates one user's + // Jupyter from another's. + val env = createdPod(7, "derived-token").getSpec.getContainers.asScala.head.getEnv.asScala + env.map(_.getName) should contain("JUPYTER_TOKEN") + env.find(_.getName == "JUPYTER_TOKEN").map(_.getValue) shouldBe Some("derived-token") + } + + it should "tell the pod which Texera origin may frame it" in { + // Without this the pod keeps the image's local-development default, and its CSP + // frame-ancestors then blocks the real deployment from embedding it. + val env = createdPod(7, "tok").getSpec.getContainers.asScala.head.getEnv.asScala + env.find(_.getName == "TEXERA_ORIGIN").map(_.getValue) shouldBe + Some(KubernetesConfig.jupyterTexeraOrigin) + } + + it should "carry the configured image, pull policy and port" in { + val container = createdPod(7, "tok").getSpec.getContainers.asScala.head + container.getImage shouldBe KubernetesConfig.jupyterImageName + container.getImagePullPolicy shouldBe KubernetesConfig.computingUnitImagePullPolicy + container.getPorts.asScala.map(_.getContainerPort.intValue()) should contain( + KubernetesConfig.jupyterPortNumber + ) + } + + it should "carry the configured cpu and memory limits" in { + val limits = createdPod(7, "tok").getSpec.getContainers.asScala.head.getResources.getLimits + limits.get("cpu").toString shouldBe KubernetesConfig.jupyterCpuLimit + limits.get("memory").toString shouldBe KubernetesConfig.jupyterMemoryLimit + } + + it should "set hostname and subdomain so generatePodURI resolves" in { + // The pair is what makes <pod>.<service>.<namespace>.svc.cluster.local addressable. + val spec = createdPod(7, "tok").getSpec + spec.getHostname shouldBe "jupyter-7" + spec.getSubdomain shouldBe KubernetesConfig.jupyterServiceName + } + + "inCluster" should "build a client lazily without requiring a reachable cluster" in { + // The companion is only touched when a provision happens, but building the client must + // not itself need a cluster: single-node and local dev have none. + val client = JupyterKubernetesClient.inCluster + client.generatePodName(7) shouldBe "jupyter-7" + } + + it should "create the pod in the Jupyter namespace" in { + val client = mock(classOf[Fabric8Client]) + val namespaceable = mock(classOf[NamespaceableResource[_]]) + .asInstanceOf[NamespaceableResource[Pod]] + val resource = mock(classOf[Resource[_]]).asInstanceOf[Resource[Pod]] + when(client.resource(org.mockito.ArgumentMatchers.any(classOf[Pod]))).thenReturn(namespaceable) + when(namespaceable.inNamespace(namespace)).thenReturn(resource) + new JupyterKubernetesClient(client).createPod(7, "tok") + verify(namespaceable).inNamespace(namespace) + verify(resource).create() + } +} diff --git a/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterProbeSpec.scala b/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterProbeSpec.scala new file mode 100644 index 0000000000..b864000347 --- /dev/null +++ b/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterProbeSpec.scala @@ -0,0 +1,109 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import com.sun.net.httpserver.{HttpExchange, HttpServer} +import org.scalatest.flatspec.AnyFlatSpec +import org.scalatest.matchers.should.Matchers + +import java.net.InetSocketAddress + +class JupyterProbeSpec extends AnyFlatSpec with Matchers { + + // Binds an ephemeral port, so this never collides with the fixed-port stub in + // NotebookMigrationResourceSpec. + private def withServer(status: Int)(test: String => Unit): Unit = { + val server = HttpServer.create(new InetSocketAddress("localhost", 0), 0) + server.createContext( + "/api", + (exchange: HttpExchange) => { + exchange.getRequestBody.readAllBytes() + val body = """{"version":"2.7.0"}""".getBytes("UTF-8") + exchange.sendResponseHeaders(status, body.length) + val os = exchange.getResponseBody + os.write(body) + os.close() + } + ) + server.start() + try test(s"http://localhost:${server.getAddress.getPort}";) + finally server.stop(0) + } + + // Serves /api/contents only to the expected token, mimicking Jupyter's own behaviour. + private def withTokenServer(expected: String)(test: String => Unit): Unit = { + val server = HttpServer.create(new InetSocketAddress("localhost", 0), 0) + server.createContext( + "/api/contents", + (exchange: HttpExchange) => { + exchange.getRequestBody.readAllBytes() + val auth = Option(exchange.getRequestHeaders.getFirst("Authorization")).getOrElse("") + val status = if (auth == s"token $expected") 200 else 403 + val body = "{}".getBytes("UTF-8") + exchange.sendResponseHeaders(status, body.length) + val os = exchange.getResponseBody + os.write(body) + os.close() + } + ) + server.start() + try test(s"http://localhost:${server.getAddress.getPort}";) + finally server.stop(0) + } + + "isAuthorized" should "accept a pod that takes the given token" in { + withTokenServer("good")(JupyterProbe.isAuthorized(_, "good") shouldBe true) + } + + it should "reject a pod holding a superseded token" in { + // The rotation case: the pod is alive, so only an authenticated probe catches it. + withTokenServer("old")(JupyterProbe.isAuthorized(_, "new") shouldBe false) + } + + it should "report unauthorized when nothing is listening" in { + JupyterProbe.isAuthorized("http://localhost:1";, "any") shouldBe false + } + + it should "report unauthorized for a malformed URL without opening a connection" in { + // The URL itself throws, so the cleanup path runs with no connection to close. + JupyterProbe.isAuthorized("notaprotocol://host", "any") shouldBe false + } + + "isAvailable" should "treat 200 as reachable" in { + withServer(200)(JupyterProbe.isAvailable(_) shouldBe true) + } + + it should "treat 403 as reachable" in { + // /api needs no token, so a refusal still proves the server is up. + withServer(403)(JupyterProbe.isAvailable(_) shouldBe true) + } + + it should "treat any other status as unavailable" in { + withServer(500)(JupyterProbe.isAvailable(_) shouldBe false) + } + + it should "report unavailable when nothing is listening" in { + // Port 1 is reserved and unbound; the connect fails rather than hanging. + JupyterProbe.isAvailable("http://localhost:1";) shouldBe false + } + + it should "report unavailable for a malformed URL without opening a connection" in { + // The URL itself throws, so the cleanup path runs with no connection to close. + JupyterProbe.isAvailable("notaprotocol://host") shouldBe false + } +} diff --git a/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterTokenDeriverSpec.scala b/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterTokenDeriverSpec.scala new file mode 100644 index 0000000000..b1aa6b22b7 --- /dev/null +++ b/notebook-migration-service/src/test/scala/org/apache/texera/service/util/JupyterTokenDeriverSpec.scala @@ -0,0 +1,89 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.texera.service.util + +import org.scalatest.flatspec.AnyFlatSpec +import org.scalatest.matchers.should.Matchers + +class JupyterTokenDeriverSpec extends AnyFlatSpec with Matchers { + + private val secret = "test-secret" + + "JupyterTokenDeriver.derive" should "return the same token for a uid across calls" in { + // The token is never stored, so every call has to reproduce it or a running pod + // becomes unreachable. + JupyterTokenDeriver.derive(7, secret) shouldBe JupyterTokenDeriver.derive(7, secret) + } + + it should "return a different token for each uid" in { + // The isolation property: one user's token must not open another user's Jupyter. + val tokens = (1 to 50).map(JupyterTokenDeriver.derive(_, secret)) + tokens.distinct.size shouldBe 50 + } + + it should "return a different token when the secret changes" in { + // Rotation: changing the secret has to invalidate previously issued tokens. + JupyterTokenDeriver.derive(7, secret) should not be JupyterTokenDeriver.derive(7, "other") + } + + it should "return a fixed-length lowercase hex token" in { + JupyterTokenDeriver.derive(7, secret) should fullyMatch regex "[0-9a-f]{32}" + } + + it should "encode every digest byte as exactly two hex characters" in { + // Pinned against an independently computed HMAC-SHA256. This digest starts 0xa5, which is + // negative as a Byte, so a formatter that sign-extended would emit "ffffffa5" and shift + // the whole token. A length or charset check alone would not notice. + JupyterTokenDeriver.derive(7, "golden-secret") shouldBe "a5d36f59073ef59843384d5411d765e5" + } + + it should "reject an empty secret" in { + an[IllegalArgumentException] should be thrownBy JupyterTokenDeriver.derive(7, "") + } + + it should "fall back to the configured secret when none is passed" in { + // Exercises the default argument. The configured secret is empty unless a deployment + // sets one, so there is nothing to derive from and the require fires. + if (sys.env.get("JUPYTER_TOKEN_SECRET").isEmpty) { + an[IllegalArgumentException] should be thrownBy JupyterTokenDeriver.derive(7) + } + } + + "JupyterTokenDeriver.validateConfiguration" should "reject an empty secret when per-user Jupyter is on" in { + val thrown = the[IllegalStateException] thrownBy JupyterTokenDeriver.validateConfiguration( + jupyterEnabled = true, + secret = "" + ) + thrown.getMessage should include("storage.jupyter.token-secret") + } + + it should "allow an empty secret when per-user Jupyter is off" in { + // Single-node and local dev run one shared Jupyter from static config and set no secret. + noException should be thrownBy JupyterTokenDeriver.validateConfiguration( + jupyterEnabled = false, + secret = "" + ) + } + + it should "allow a configured secret when per-user Jupyter is on" in { + noException should be thrownBy JupyterTokenDeriver.validateConfiguration( + jupyterEnabled = true, + secret = secret + ) + } +} diff --git a/sql/changelog.xml b/sql/changelog.xml index b0c0f1354d..677af284f0 100644 --- a/sql/changelog.xml +++ b/sql/changelog.xml @@ -149,6 +149,11 @@ <sqlFile path="sql/updates/47.sql"/> </changeSet> + <!-- Add the per-user JupyterLab registry table --> + <changeSet id="48" author="zyratlo"> + <sqlFile path="sql/updates/48.sql"/> + </changeSet> + <!-- example changeSet <changeSet id="1" author="author"> <sqlFile path="sql/updates/1.sql"/> diff --git a/sql/texera_ddl.sql b/sql/texera_ddl.sql index 132080c18a..f1890e6f78 100644 --- a/sql/texera_ddl.sql +++ b/sql/texera_ddl.sql @@ -81,6 +81,7 @@ DROP TABLE IF EXISTS site_settings CASCADE; DROP TABLE IF EXISTS computing_unit_user_access CASCADE; DROP TABLE IF EXISTS notebook CASCADE; DROP TABLE IF EXISTS workflow_notebook_mapping CASCADE; +DROP TABLE IF EXISTS user_jupyter CASCADE; DROP TABLE IF EXISTS virtual_environments CASCADE; -- ============================================ @@ -675,6 +676,20 @@ CREATE TABLE IF NOT EXISTS workflow_notebook_mapping FOREIGN KEY (wid, nid) REFERENCES notebook(wid, nid) ON DELETE CASCADE ); +-- Per-user JupyterLab registration: one row per provisioned user, so the service resolves +-- endpoints per user instead of from process-wide config. Both URLs are stored rather than +-- derived, matching workflow_computing_unit.uri, so addressing can change without a code +-- change: internal_url is what the service calls, public_url is what the browser loads. +-- The token is derived from a server secret and the uid, so it is not stored. +CREATE TABLE IF NOT EXISTS user_jupyter +( + uid INT NOT NULL PRIMARY KEY, + internal_url VARCHAR(512) NOT NULL, + public_url VARCHAR(512) NOT NULL, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + FOREIGN KEY (uid) REFERENCES "user"(uid) ON DELETE CASCADE +); + -- START Fulltext search index creation (DO NOT EDIT THIS LINE) CREATE EXTENSION IF NOT EXISTS pgroonga; diff --git a/sql/updates/48.sql b/sql/updates/48.sql new file mode 100644 index 0000000000..c00d7d6d01 --- /dev/null +++ b/sql/updates/48.sql @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +\c texera_db + +SET search_path TO texera_db; + +BEGIN; + +-- Per-user JupyterLab registration: one row per provisioned user, so the service resolves +-- endpoints per user instead of from process-wide config. Both URLs are stored rather than +-- derived, matching workflow_computing_unit.uri, so addressing can change without a code +-- change: internal_url is what the service calls, public_url is what the browser loads. +-- The token is derived from a server secret and the uid, so it is not stored. +CREATE TABLE IF NOT EXISTS user_jupyter +( + uid INT NOT NULL PRIMARY KEY, + internal_url VARCHAR(512) NOT NULL, + public_url VARCHAR(512) NOT NULL, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + FOREIGN KEY (uid) REFERENCES "user"(uid) ON DELETE CASCADE +); + +COMMIT;
