This is an automated email from the ASF dual-hosted git repository.
jiadongb pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/texera.git
The following commit(s) were added to refs/heads/main by this push:
new ddad0157d0 fix: restructure bin/deployment and update Dockerfiles for
new repository structure (#3892)
ddad0157d0 is described below
commit ddad0157d037e84b3e818c07fbdbc1cebc789641
Author: Jiadong Bai <[email protected]>
AuthorDate: Tue Oct 14 18:33:42 2025 -0700
fix: restructure bin/deployment and update Dockerfiles for new repository
structure (#3892)
Closes #3891
## Summary
This PR restructures the `bin/deployment/` directory and updates all
Dockerfiles to be compatible with the recent repository changes where
`core` was renamed to `common` and services were moved to the root
level.
## Changes Made
### Directory Restructuring
- Moved all contents from `bin/deployment/` directly to `bin/`
- Moved `bin/deployment/k8s/texera-helmchart/` to `bin/k8s/` (renamed
from texera-helmchart)
### Dockerfile Updates
Updated all Dockerfiles to work with build context set at project root:
### Symlink Fixes
- All SQL file symlinks now resolve correctly
---------
Co-authored-by: ali risheh <[email protected]>
---
bin/{deployment => }/README.md | 18 +++++-----
.../access-control-service.dockerfile | 24 +++++++------
bin/{deployment/build-all.sh => build-images.sh} | 0
.../computing-unit-master.dockerfile | 34 +++++++++---------
.../computing-unit-worker.dockerfile | 28 ++++++++-------
bin/{deployment => }/config-service.dockerfile | 26 +++++++-------
.../files/iceberg_postgres_catalog.sql | 1 -
.../k8s/texera-helmchart/files/texera_ddl.sql | 1 -
.../k8s/texera-helmchart/files/texera_lakefs.sql | 1 -
bin/{deployment => }/file-service.dockerfile | 22 ++++++------
.../k8s/texera-helmchart => k8s}/Chart.yaml | 0
bin/{deployment => }/k8s/README.md | 0
bin/k8s/files/iceberg_postgres_catalog.sql | 1 +
bin/k8s/files/texera_ddl.sql | 1 +
bin/k8s/files/texera_lakefs.sql | 1 +
.../access-control-service-deployment.yaml | 0
.../templates/access-control-service-service.yaml | 0
.../templates/config-service-deployment.yaml | 0
.../templates/config-service-service.yaml | 0
.../templates/envoy-config.yaml | 0
.../templates/envoy-deployment.yaml | 0
.../templates/envoy-service.yaml | 0
.../templates/example-data-loader-job.yaml | 0
.../templates/external-names.yaml | 0
.../templates/file-service-deployment.yaml | 0
.../templates/file-service-service.yaml | 0
.../templates/ingress.yaml | 0
.../templates/lakefs-secret.yaml | 0
.../templates/lakefs-setup-job.yaml | 0
.../templates/minio-ingress.yaml | 0
.../templates/minio-persistence.yaml | 0
.../templates/postgresql-init-script-config.yaml | 0
.../templates/postgresql-persistence.yaml | 0
.../texera-helmchart => k8s}/templates/pylsp.yaml | 0
.../templates/shared-editing-server.yaml | 0
.../templates/webserver-deployment.yaml | 0
.../templates/webserver-service.yaml | 0
.../workflow-compiling-service-deployment.yaml | 0
.../workflow-compiling-service-service.yaml | 0
...workflow-computing-unit-manager-deployment.yaml | 0
...low-computing-unit-manager-service-account.yaml | 0
.../workflow-computing-unit-manager-service.yaml | 0
...ow-computing-unit-master-prepull-daemonset.yaml | 0
.../workflow-computing-unit-resource-quota.yaml | 0
.../workflow-computing-units-namespace.yaml | 0
.../workflow-computing-units-service.yaml | 0
.../k8s/texera-helmchart => k8s}/values.yaml | 0
bin/{deployment => }/merge-image-tags.sh | 0
.../postgres17-pgroonga.dockerfile | 0
bin/{deployment => }/pylsp/Dockerfile | 0
.../pylsp/python-language-server.yaml | 0
bin/{deployment => }/pylsp/run_pylsp.sh | 0
bin/{deployment => }/single-node/.env | 0
bin/{deployment => }/single-node/README.md | 0
.../single-node/docker-compose.yml | 0
bin/{deployment => }/single-node/nginx.conf | 0
.../texera-web-application.dockerfile | 42 +++++++++++-----------
.../workflow-compiling-service.dockerfile | 24 +++++++------
...flow-computing-unit-managing-service.dockerfile | 26 +++++++-------
bin/{deployment => }/y-websocket-server/Dockerfile | 0
.../y-websocket-server/package.json | 0
61 files changed, 132 insertions(+), 118 deletions(-)
diff --git a/bin/deployment/README.md b/bin/README.md
similarity index 55%
rename from bin/deployment/README.md
rename to bin/README.md
index 60ca74eea2..3b13961296 100644
--- a/bin/deployment/README.md
+++ b/bin/README.md
@@ -2,22 +2,20 @@
This directory contains Dockerfiles and configuration files for building and
deploying Texera's microservices.
-## Dockerfiles and Images
+## Dockerfiles
This directory includes several Dockerfiles, such as `file-service.dockerfile`
and `computing-unit-master.dockerfile`. Each Dockerfile builds a specific
Texera microservice. All Dockerfiles must be built from the `texera` project
root as the Docker build context.
-For example, to build the image using `texera-web-application.dockerfile`, run
the following command from the `deployment` directory:
+For example, to build the image using `texera-web-application.dockerfile`, run
the following command **from the project root**:
```bash
-docker build -f texera-web-application.dockerfile -t
texera/texera-web-application:test ..
-```
+docker build -f bin/texera-web-application.dockerfile -t
your-repo/texera-web-application:test .
+```
-You can also find prebuilt images published by the Texera team on the [Texera
DockerHub Repository](https://hub.docker.com/repositories/texera).
-
-## Deployment Based on Docker Images
+Two shell scripts, `build-images.sh` and `build-services.sh` are included for
building platform-dependent images conveniently.
-Subdirectories like `single-node` contain configuration files for deploying
Texera using the above Docker images.
+You can also find prebuilt images published by the Texera team on the [Texera
DockerHub Repository](https://hub.docker.com/repositories/texera).
-### `single-node` architecture
+## Deployment using images
-The `single-node` directory includes the necessary files for deploying Texera
on a single machine using Docker Compose.
+Subdirectories `single-node` and `k8s` contain configuration files for
deploying Texera using the above Docker images.
\ No newline at end of file
diff --git a/bin/deployment/access-control-service.dockerfile
b/bin/access-control-service.dockerfile
similarity index 68%
rename from bin/deployment/access-control-service.dockerfile
rename to bin/access-control-service.dockerfile
index 6121a2709d..41b328f78e 100644
--- a/bin/deployment/access-control-service.dockerfile
+++ b/bin/access-control-service.dockerfile
@@ -18,10 +18,13 @@
FROM sbtscala/scala-sbt:eclipse-temurin-jammy-11.0.17_8_1.9.3_2.13.11 AS build
# Set working directory
-WORKDIR /core
+WORKDIR /texera
-# Copy all projects under core to /core
-COPY core/ .
+# Copy modules for building the service
+COPY common/ common/
+COPY access-control-service/ access-control-service/
+COPY project/ project/
+COPY build.sbt build.sbt
# Update system and install dependencies
RUN apt-get update && apt-get install -y \
@@ -30,24 +33,23 @@ RUN apt-get update && apt-get install -y \
libpq-dev \
&& apt-get clean
-WORKDIR /core
# Add .git for runtime calls to jgit from OPversion
-COPY .git ../.git
+COPY .git .git
RUN sbt clean AccessControlService/dist
# Unzip the texera binary
-RUN unzip
access-control-service/target/universal/access-control-service-*.zip -d target/
+RUN unzip access-control-service/target/universal/access-control-service-*.zip
-d target/
FROM eclipse-temurin:11-jre-jammy AS runtime
-WORKDIR /core
+WORKDIR /texera
-COPY --from=build /.git /.git
+COPY --from=build /texera/.git /texera/.git
# Copy the built texera binary from the build phase
-COPY --from=build /core/target/access-control-service* /core/
-# Copy resources directories under /core from build phase
-COPY --from=build /core/access-control-service/src/main/resources
/core/access-control-service/src/main/resources
+COPY --from=build /texera/target/access-control-service* /texera/
+# Copy resources directories from build phase
+COPY --from=build /texera/access-control-service/src/main/resources
/texera/access-control-service/src/main/resources
CMD ["bin/access-control-service"]
diff --git a/bin/deployment/build-all.sh b/bin/build-images.sh
similarity index 100%
rename from bin/deployment/build-all.sh
rename to bin/build-images.sh
diff --git a/bin/deployment/computing-unit-master.dockerfile
b/bin/computing-unit-master.dockerfile
similarity index 77%
rename from bin/deployment/computing-unit-master.dockerfile
rename to bin/computing-unit-master.dockerfile
index d44fe9703d..e94c4158ff 100644
--- a/bin/deployment/computing-unit-master.dockerfile
+++ b/bin/computing-unit-master.dockerfile
@@ -18,10 +18,13 @@
FROM sbtscala/scala-sbt:eclipse-temurin-jammy-11.0.17_8_1.9.3_2.13.11 AS build
# Set working directory
-WORKDIR /core
+WORKDIR /texera
-# Copy all projects under core to /core
-COPY core/ .
+# Copy modules for building the service
+COPY common/ common/
+COPY amber/ amber/
+COPY project/ project/
+COPY build.sbt build.sbt
# Update system and install dependencies
RUN apt-get update && apt-get install -y \
@@ -30,22 +33,21 @@ RUN apt-get update && apt-get install -y \
libpq-dev \
&& apt-get clean
-WORKDIR /core
# Add .git for runtime calls to jgit from OPversion
-COPY .git ../.git
+COPY .git .git
RUN sbt clean WorkflowExecutionService/dist
# Unzip the texera binary
-RUN unzip amber/target/universal/texera-*.zip -d amber/target/
+RUN unzip amber/target/universal/amber-*.zip -d amber/target/
FROM eclipse-temurin:11-jdk-jammy AS runtime
-WORKDIR /amber
+WORKDIR /texera/amber
-COPY --from=build /amber/r-requirements.txt /tmp/r-requirements.txt
-COPY --from=build /amber/requirements.txt /tmp/requirements.txt
-COPY --from=build /amber/operator-requirements.txt
/tmp/operator-requirements.txt
+COPY --from=build /texera/amber/r-requirements.txt /tmp/r-requirements.txt
+COPY --from=build /texera/amber/requirements.txt /tmp/requirements.txt
+COPY --from=build /texera/amber/operator-requirements.txt
/tmp/operator-requirements.txt
# Install Python & R runtime dependencies
RUN apt-get update && apt-get install -y \
@@ -97,13 +99,13 @@ RUN Rscript -e "options(repos = c(CRAN =
'https://cran.r-project.org')); \
ENV LD_LIBRARY_PATH=/usr/local/lib/R/lib:$LD_LIBRARY_PATH
# Copy the built texera binary from the build phase
-COPY --from=build /.git /.git
-COPY --from=build /amber/target/texera-* /amber
-# Copy resources directories under /core from build phase
-COPY --from=build /core/config/src/main/resources
/core/config/src/main/resources
-COPY --from=build /amber/src/main/resources /amber/src/main/resources
+COPY --from=build /texera/.git /texera/amber/.git
+COPY --from=build /texera/amber/target/amber-* /texera/amber/
+# Copy resources directories from build phase
+COPY --from=build /texera/common/config/src/main/resources
/texera/amber/common/config/src/main/resources
+COPY --from=build /texera/amber/src/main/resources
/texera/amber/src/main/resources
# Copy code for python & R UDF
-COPY --from=build /amber/src/main/python /amber/src/main/python
+COPY --from=build /texera/amber/src/main/python /texera/amber/src/main/python
CMD ["bin/computing-unit-master"]
diff --git a/bin/deployment/computing-unit-worker.dockerfile
b/bin/computing-unit-worker.dockerfile
similarity index 71%
rename from bin/deployment/computing-unit-worker.dockerfile
rename to bin/computing-unit-worker.dockerfile
index 4d60b52497..9fd3867170 100644
--- a/bin/deployment/computing-unit-worker.dockerfile
+++ b/bin/computing-unit-worker.dockerfile
@@ -18,10 +18,13 @@
FROM sbtscala/scala-sbt:eclipse-temurin-jammy-11.0.17_8_1.9.3_2.13.11 AS build
# Set working directory
-WORKDIR /core
+WORKDIR /texera
-# Copy all projects under core to /core
-COPY core/ .
+# Copy modules for building the service
+COPY common/ common/
+COPY amber/ amber/
+COPY project/ project/
+COPY build.sbt build.sbt
# Update system and install dependencies
RUN apt-get update && apt-get install -y \
@@ -30,21 +33,20 @@ RUN apt-get update && apt-get install -y \
libpq-dev \
&& apt-get clean
-WORKDIR /core
# Add .git for runtime calls to jgit from OPversion
-COPY .git ../.git
+COPY .git .git
RUN sbt clean WorkflowExecutionService/dist
# Unzip the texera binary
-RUN unzip amber/target/universal/texera-*.zip -d amber/target/
+RUN unzip amber/target/universal/amber-*.zip -d amber/target/
FROM eclipse-temurin:11-jre-jammy AS runtime
-WORKDIR /amber
+WORKDIR /texera/amber
-COPY --from=build /amber/requirements.txt /tmp/requirements.txt
-COPY --from=build /amber/operator-requirements.txt
/tmp/operator-requirements.txt
+COPY --from=build /texera/amber/requirements.txt /tmp/requirements.txt
+COPY --from=build /texera/amber/operator-requirements.txt
/tmp/operator-requirements.txt
# Install Python runtime and dependencies
RUN apt-get update && apt-get install -y \
@@ -62,10 +64,10 @@ RUN pip3 install --no-cache-dir --find-links
https://pypi.org/simple/ -r /tmp/op
pip3 install --no-cache-dir wordcloud==1.9.2
# Copy the built texera binary from the build phase
-COPY --from=build /amber/target/texera-* /amber
-# Copy resources directories under /core from build phase
-COPY --from=build /amber/src/main/resources /amber/src/main/resources
-COPY --from=build /core/config/src/main/resources
/core/config/src/main/resources
+COPY --from=build /texera/amber/target/amber-* /texera/amber/
+# Copy resources directories from build phase
+COPY --from=build /texera/amber/src/main/resources
/texera/amber/src/main/resources
+COPY --from=build /texera/common/config/src/main/resources
/texera/amber/common/config/src/main/resources
CMD ["bin/computing-unit-worker"]
diff --git a/bin/deployment/config-service.dockerfile
b/bin/config-service.dockerfile
similarity index 67%
rename from bin/deployment/config-service.dockerfile
rename to bin/config-service.dockerfile
index bcabed144f..1a8bd34ec6 100644
--- a/bin/deployment/config-service.dockerfile
+++ b/bin/config-service.dockerfile
@@ -18,10 +18,13 @@
FROM sbtscala/scala-sbt:eclipse-temurin-jammy-11.0.17_8_1.9.3_2.13.11 AS build
# Set working directory
-WORKDIR /core
+WORKDIR /texera
-# Copy all projects under core to /core
-COPY core/ .
+# Copy modules for building the service
+COPY common/ common/
+COPY config-service/ config-service/
+COPY project/ project/
+COPY build.sbt build.sbt
# Update system and install dependencies
RUN apt-get update && apt-get install -y \
@@ -30,25 +33,24 @@ RUN apt-get update && apt-get install -y \
libpq-dev \
&& apt-get clean
-WORKDIR /core
# Add .git for runtime calls to jgit from OPversion
-COPY .git ../.git
+COPY .git .git
RUN sbt clean ConfigService/dist
# Unzip the texera binary
-RUN unzip config-service/target/universal/config-service-*.zip -d target/
+RUN unzip config-service/target/universal/config-service-*.zip -d target/
FROM eclipse-temurin:11-jre-jammy AS runtime
-WORKDIR /core
+WORKDIR /texera
-COPY --from=build /.git /.git
+COPY --from=build /texera/.git /texera/.git
# Copy the built texera binary from the build phase
-COPY --from=build /core/target/config-service-* /core/
-# Copy resources directories under /core from build phase
-COPY --from=build /core/config/src/main/resources
/core/config/src/main/resources
-COPY --from=build /core/config-service/src/main/resources
/core/config-service/src/main/resources
+COPY --from=build /texera/target/config-service-* /texera/
+# Copy resources directories from build phase
+COPY --from=build /texera/common/config/src/main/resources
/texera/common/config/src/main/resources
+COPY --from=build /texera/config-service/src/main/resources
/texera/config-service/src/main/resources
CMD ["bin/config-service"]
diff --git
a/bin/deployment/k8s/texera-helmchart/files/iceberg_postgres_catalog.sql
b/bin/deployment/k8s/texera-helmchart/files/iceberg_postgres_catalog.sql
deleted file mode 120000
index 684406cdac..0000000000
--- a/bin/deployment/k8s/texera-helmchart/files/iceberg_postgres_catalog.sql
+++ /dev/null
@@ -1 +0,0 @@
-../../../../core/scripts/sql/iceberg_postgres_catalog.sql
\ No newline at end of file
diff --git a/bin/deployment/k8s/texera-helmchart/files/texera_ddl.sql
b/bin/deployment/k8s/texera-helmchart/files/texera_ddl.sql
deleted file mode 120000
index 955a6bcd26..0000000000
--- a/bin/deployment/k8s/texera-helmchart/files/texera_ddl.sql
+++ /dev/null
@@ -1 +0,0 @@
-../../../../core/scripts/sql/texera_ddl.sql
\ No newline at end of file
diff --git a/bin/deployment/k8s/texera-helmchart/files/texera_lakefs.sql
b/bin/deployment/k8s/texera-helmchart/files/texera_lakefs.sql
deleted file mode 120000
index 42010984c8..0000000000
--- a/bin/deployment/k8s/texera-helmchart/files/texera_lakefs.sql
+++ /dev/null
@@ -1 +0,0 @@
-../../../../core/scripts/sql/texera_lakefs.sql
\ No newline at end of file
diff --git a/bin/deployment/file-service.dockerfile
b/bin/file-service.dockerfile
similarity index 72%
rename from bin/deployment/file-service.dockerfile
rename to bin/file-service.dockerfile
index 25fb9c945e..0fc8a433d7 100644
--- a/bin/deployment/file-service.dockerfile
+++ b/bin/file-service.dockerfile
@@ -18,10 +18,13 @@
FROM sbtscala/scala-sbt:eclipse-temurin-jammy-11.0.17_8_1.9.3_2.13.11 AS build
# Set working directory
-WORKDIR /core
+WORKDIR /texera
-# Copy all projects under core to /core
-COPY core/ .
+# Copy modules for building the service
+COPY common/ common/
+COPY file-service/ file-service/
+COPY project/ project/
+COPY build.sbt build.sbt
# Update system and install dependencies
RUN apt-get update && apt-get install -y \
@@ -30,9 +33,8 @@ RUN apt-get update && apt-get install -y \
libpq-dev \
&& apt-get clean
-WORKDIR /core
# Add .git for runtime calls to jgit from OPversion
-COPY .git ../.git
+COPY .git .git
RUN sbt clean FileService/dist
@@ -41,13 +43,13 @@ RUN unzip file-service/target/universal/file-service-*.zip
-d target/
FROM eclipse-temurin:11-jre-jammy AS runtime
-WORKDIR /core
+WORKDIR /texera
# Copy the built texera binary from the build phase
-COPY --from=build /core/target/file-service-* /core/
-# Copy resources directories under /core from build phase
-COPY --from=build /core/config/src/main/resources
/core/config/src/main/resources
-COPY --from=build /core/file-service/src/main/resources
/core/file-service/src/main/resources
+COPY --from=build /texera/target/file-service-* /texera/
+# Copy resources directories from build phase
+COPY --from=build /texera/common/config/src/main/resources
/texera/common/config/src/main/resources
+COPY --from=build /texera/file-service/src/main/resources
/texera/file-service/src/main/resources
CMD ["bin/file-service"]
diff --git a/bin/deployment/k8s/texera-helmchart/Chart.yaml b/bin/k8s/Chart.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/Chart.yaml
rename to bin/k8s/Chart.yaml
diff --git a/bin/deployment/k8s/README.md b/bin/k8s/README.md
similarity index 100%
rename from bin/deployment/k8s/README.md
rename to bin/k8s/README.md
diff --git a/bin/k8s/files/iceberg_postgres_catalog.sql
b/bin/k8s/files/iceberg_postgres_catalog.sql
new file mode 120000
index 0000000000..7def191e8e
--- /dev/null
+++ b/bin/k8s/files/iceberg_postgres_catalog.sql
@@ -0,0 +1 @@
+../../../sql/iceberg_postgres_catalog.sql
\ No newline at end of file
diff --git a/bin/k8s/files/texera_ddl.sql b/bin/k8s/files/texera_ddl.sql
new file mode 120000
index 0000000000..5653a9a01e
--- /dev/null
+++ b/bin/k8s/files/texera_ddl.sql
@@ -0,0 +1 @@
+../../../sql/texera_ddl.sql
\ No newline at end of file
diff --git a/bin/k8s/files/texera_lakefs.sql b/bin/k8s/files/texera_lakefs.sql
new file mode 120000
index 0000000000..6d58e92106
--- /dev/null
+++ b/bin/k8s/files/texera_lakefs.sql
@@ -0,0 +1 @@
+../../../sql/texera_lakefs.sql
\ No newline at end of file
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/access-control-service-deployment.yaml
b/bin/k8s/templates/access-control-service-deployment.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/access-control-service-deployment.yaml
rename to bin/k8s/templates/access-control-service-deployment.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/access-control-service-service.yaml
b/bin/k8s/templates/access-control-service-service.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/access-control-service-service.yaml
rename to bin/k8s/templates/access-control-service-service.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/config-service-deployment.yaml
b/bin/k8s/templates/config-service-deployment.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/config-service-deployment.yaml
rename to bin/k8s/templates/config-service-deployment.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/config-service-service.yaml
b/bin/k8s/templates/config-service-service.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/config-service-service.yaml
rename to bin/k8s/templates/config-service-service.yaml
diff --git a/bin/deployment/k8s/texera-helmchart/templates/envoy-config.yaml
b/bin/k8s/templates/envoy-config.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/envoy-config.yaml
rename to bin/k8s/templates/envoy-config.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/envoy-deployment.yaml
b/bin/k8s/templates/envoy-deployment.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/envoy-deployment.yaml
rename to bin/k8s/templates/envoy-deployment.yaml
diff --git a/bin/deployment/k8s/texera-helmchart/templates/envoy-service.yaml
b/bin/k8s/templates/envoy-service.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/envoy-service.yaml
rename to bin/k8s/templates/envoy-service.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/example-data-loader-job.yaml
b/bin/k8s/templates/example-data-loader-job.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/example-data-loader-job.yaml
rename to bin/k8s/templates/example-data-loader-job.yaml
diff --git a/bin/deployment/k8s/texera-helmchart/templates/external-names.yaml
b/bin/k8s/templates/external-names.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/external-names.yaml
rename to bin/k8s/templates/external-names.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/file-service-deployment.yaml
b/bin/k8s/templates/file-service-deployment.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/file-service-deployment.yaml
rename to bin/k8s/templates/file-service-deployment.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/file-service-service.yaml
b/bin/k8s/templates/file-service-service.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/file-service-service.yaml
rename to bin/k8s/templates/file-service-service.yaml
diff --git a/bin/deployment/k8s/texera-helmchart/templates/ingress.yaml
b/bin/k8s/templates/ingress.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/ingress.yaml
rename to bin/k8s/templates/ingress.yaml
diff --git a/bin/deployment/k8s/texera-helmchart/templates/lakefs-secret.yaml
b/bin/k8s/templates/lakefs-secret.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/lakefs-secret.yaml
rename to bin/k8s/templates/lakefs-secret.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/lakefs-setup-job.yaml
b/bin/k8s/templates/lakefs-setup-job.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/lakefs-setup-job.yaml
rename to bin/k8s/templates/lakefs-setup-job.yaml
diff --git a/bin/deployment/k8s/texera-helmchart/templates/minio-ingress.yaml
b/bin/k8s/templates/minio-ingress.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/minio-ingress.yaml
rename to bin/k8s/templates/minio-ingress.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/minio-persistence.yaml
b/bin/k8s/templates/minio-persistence.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/minio-persistence.yaml
rename to bin/k8s/templates/minio-persistence.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/postgresql-init-script-config.yaml
b/bin/k8s/templates/postgresql-init-script-config.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/postgresql-init-script-config.yaml
rename to bin/k8s/templates/postgresql-init-script-config.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/postgresql-persistence.yaml
b/bin/k8s/templates/postgresql-persistence.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/postgresql-persistence.yaml
rename to bin/k8s/templates/postgresql-persistence.yaml
diff --git a/bin/deployment/k8s/texera-helmchart/templates/pylsp.yaml
b/bin/k8s/templates/pylsp.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/pylsp.yaml
rename to bin/k8s/templates/pylsp.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/shared-editing-server.yaml
b/bin/k8s/templates/shared-editing-server.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/shared-editing-server.yaml
rename to bin/k8s/templates/shared-editing-server.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/webserver-deployment.yaml
b/bin/k8s/templates/webserver-deployment.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/webserver-deployment.yaml
rename to bin/k8s/templates/webserver-deployment.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/webserver-service.yaml
b/bin/k8s/templates/webserver-service.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/templates/webserver-service.yaml
rename to bin/k8s/templates/webserver-service.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/workflow-compiling-service-deployment.yaml
b/bin/k8s/templates/workflow-compiling-service-deployment.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/workflow-compiling-service-deployment.yaml
rename to bin/k8s/templates/workflow-compiling-service-deployment.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/workflow-compiling-service-service.yaml
b/bin/k8s/templates/workflow-compiling-service-service.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/workflow-compiling-service-service.yaml
rename to bin/k8s/templates/workflow-compiling-service-service.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-manager-deployment.yaml
b/bin/k8s/templates/workflow-computing-unit-manager-deployment.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-manager-deployment.yaml
rename to bin/k8s/templates/workflow-computing-unit-manager-deployment.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-manager-service-account.yaml
b/bin/k8s/templates/workflow-computing-unit-manager-service-account.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-manager-service-account.yaml
rename to bin/k8s/templates/workflow-computing-unit-manager-service-account.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-manager-service.yaml
b/bin/k8s/templates/workflow-computing-unit-manager-service.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-manager-service.yaml
rename to bin/k8s/templates/workflow-computing-unit-manager-service.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-master-prepull-daemonset.yaml
b/bin/k8s/templates/workflow-computing-unit-master-prepull-daemonset.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-master-prepull-daemonset.yaml
rename to
bin/k8s/templates/workflow-computing-unit-master-prepull-daemonset.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-resource-quota.yaml
b/bin/k8s/templates/workflow-computing-unit-resource-quota.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/workflow-computing-unit-resource-quota.yaml
rename to bin/k8s/templates/workflow-computing-unit-resource-quota.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/workflow-computing-units-namespace.yaml
b/bin/k8s/templates/workflow-computing-units-namespace.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/workflow-computing-units-namespace.yaml
rename to bin/k8s/templates/workflow-computing-units-namespace.yaml
diff --git
a/bin/deployment/k8s/texera-helmchart/templates/workflow-computing-units-service.yaml
b/bin/k8s/templates/workflow-computing-units-service.yaml
similarity index 100%
rename from
bin/deployment/k8s/texera-helmchart/templates/workflow-computing-units-service.yaml
rename to bin/k8s/templates/workflow-computing-units-service.yaml
diff --git a/bin/deployment/k8s/texera-helmchart/values.yaml
b/bin/k8s/values.yaml
similarity index 100%
rename from bin/deployment/k8s/texera-helmchart/values.yaml
rename to bin/k8s/values.yaml
diff --git a/bin/deployment/merge-image-tags.sh b/bin/merge-image-tags.sh
similarity index 100%
rename from bin/deployment/merge-image-tags.sh
rename to bin/merge-image-tags.sh
diff --git a/bin/deployment/postgres17-pgroonga.dockerfile
b/bin/postgres17-pgroonga.dockerfile
similarity index 100%
rename from bin/deployment/postgres17-pgroonga.dockerfile
rename to bin/postgres17-pgroonga.dockerfile
diff --git a/bin/deployment/pylsp/Dockerfile b/bin/pylsp/Dockerfile
similarity index 100%
rename from bin/deployment/pylsp/Dockerfile
rename to bin/pylsp/Dockerfile
diff --git a/bin/deployment/pylsp/python-language-server.yaml
b/bin/pylsp/python-language-server.yaml
similarity index 100%
rename from bin/deployment/pylsp/python-language-server.yaml
rename to bin/pylsp/python-language-server.yaml
diff --git a/bin/deployment/pylsp/run_pylsp.sh b/bin/pylsp/run_pylsp.sh
similarity index 100%
rename from bin/deployment/pylsp/run_pylsp.sh
rename to bin/pylsp/run_pylsp.sh
diff --git a/bin/deployment/single-node/.env b/bin/single-node/.env
similarity index 100%
rename from bin/deployment/single-node/.env
rename to bin/single-node/.env
diff --git a/bin/deployment/single-node/README.md b/bin/single-node/README.md
similarity index 100%
rename from bin/deployment/single-node/README.md
rename to bin/single-node/README.md
diff --git a/bin/deployment/single-node/docker-compose.yml
b/bin/single-node/docker-compose.yml
similarity index 100%
rename from bin/deployment/single-node/docker-compose.yml
rename to bin/single-node/docker-compose.yml
diff --git a/bin/deployment/single-node/nginx.conf b/bin/single-node/nginx.conf
similarity index 100%
rename from bin/deployment/single-node/nginx.conf
rename to bin/single-node/nginx.conf
diff --git a/bin/deployment/texera-web-application.dockerfile
b/bin/texera-web-application.dockerfile
similarity index 61%
rename from bin/deployment/texera-web-application.dockerfile
rename to bin/texera-web-application.dockerfile
index 609c0f40d6..0b33a12448 100644
--- a/bin/deployment/texera-web-application.dockerfile
+++ b/bin/texera-web-application.dockerfile
@@ -15,27 +15,30 @@
# specific language governing permissions and limitations
# under the License.
-FROM node:18.17 AS build-gui
+FROM node:18.17 AS build-frontend
RUN apt-get update && apt-get install -y --no-install-recommends \
python3 build-essential git ca-certificates
-WORKDIR /gui
-COPY core/gui /gui
-RUN rm -f /gui/.yarnrc.yml
+WORKDIR /frontend
+COPY frontend /frontend
+RUN rm -f /frontend/.yarnrc.yml
RUN corepack enable && corepack prepare [email protected] --activate && yarn set
version --yarn-path 4.5.1
-RUN echo "nodeLinker: node-modules" >> /gui/.yarnrc.yml
+RUN echo "nodeLinker: node-modules" >> /frontend/.yarnrc.yml
-WORKDIR /gui
+WORKDIR /frontend
RUN yarn install && yarn run build
FROM sbtscala/scala-sbt:eclipse-temurin-jammy-11.0.17_8_1.9.3_2.13.11 AS build
# Set working directory
-WORKDIR /core
+WORKDIR /texera
-# Copy all projects under core to /core
-COPY core/ .
+# Copy modules for building the service
+COPY common/ common/
+COPY amber/ amber/
+COPY project/ project/
+COPY build.sbt build.sbt
# Update system and install dependencies
RUN apt-get update && apt-get install -y \
@@ -44,26 +47,25 @@ RUN apt-get update && apt-get install -y \
libpq-dev \
&& apt-get clean
-WORKDIR /core
# Add .git for runtime calls to jgit from OPversion
-COPY .git ../.git
+COPY .git .git
RUN sbt clean WorkflowExecutionService/dist
# Unzip the texera binary
-RUN unzip amber/target/universal/texera-*.zip -d amber/target/
+RUN unzip amber/target/universal/amber-*.zip -d amber/target/
FROM eclipse-temurin:11-jre-jammy AS runtime
-WORKDIR /amber
-# Copy built GUI files from the build-gui stage
-COPY --from=build-gui /gui/dist /core/gui/dist
+WORKDIR /texera/amber
+# Copy built frontend files from the build-frontend stage to match
FileAssetsBundle path (../../frontend/dist from /texera/amber)
+COPY --from=build-frontend /frontend/dist /frontend/dist
# Copy the built texera binary from the build phase
-COPY --from=build /.git /.git
-COPY --from=build /amber/target/texera-* /amber
-# Copy resources directories under /core from build phase
-COPY --from=build /amber/src/main/resources /amber/src/main/resources
-COPY --from=build /core/config/src/main/resources
/core/config/src/main/resources
+COPY --from=build /texera/.git /texera/amber/.git
+COPY --from=build /texera/amber/target/amber-* /texera/amber/
+# Copy resources directories from build phase
+COPY --from=build /texera/amber/src/main/resources
/texera/amber/src/main/resources
+COPY --from=build /texera/common/config/src/main/resources
/texera/amber/common/config/src/main/resources
CMD ["bin/texera-web-application"]
diff --git a/bin/deployment/workflow-compiling-service.dockerfile
b/bin/workflow-compiling-service.dockerfile
similarity index 69%
rename from bin/deployment/workflow-compiling-service.dockerfile
rename to bin/workflow-compiling-service.dockerfile
index 9509752184..803cbe87de 100644
--- a/bin/deployment/workflow-compiling-service.dockerfile
+++ b/bin/workflow-compiling-service.dockerfile
@@ -18,10 +18,13 @@
FROM sbtscala/scala-sbt:eclipse-temurin-jammy-11.0.17_8_1.9.3_2.13.11 AS build
# Set working directory
-WORKDIR /core
+WORKDIR /texera
-# Copy all projects under core to /core
-COPY core/ .
+# Copy modules for building the service
+COPY common/ common/
+COPY workflow-compiling-service/ workflow-compiling-service/
+COPY project/ project/
+COPY build.sbt build.sbt
# Update system and install dependencies
RUN apt-get update && apt-get install -y \
@@ -30,9 +33,8 @@ RUN apt-get update && apt-get install -y \
libpq-dev \
&& apt-get clean
-WORKDIR /core
# Add .git for runtime calls to jgit from OPversion
-COPY .git ../.git
+COPY .git .git
RUN sbt clean WorkflowCompilingService/dist
@@ -41,14 +43,14 @@ RUN unzip
workflow-compiling-service/target/universal/workflow-compiling-service
FROM eclipse-temurin:11-jre-jammy AS runtime
-WORKDIR /core
+WORKDIR /texera
# Copy the built texera binary from the build phase
-COPY --from=build /.git /.git
-COPY --from=build /core/target/workflow-compiling-service-* /core/
-# Copy resources directories under /core from build phase
-COPY --from=build /core/config/src/main/resources
/core/config/src/main/resources
-COPY --from=build /core/workflow-compiling-service/src/main/resources
/core/workflow-compiling-service/src/main/resources
+COPY --from=build /texera/.git /texera/.git
+COPY --from=build /texera/target/workflow-compiling-service-* /texera/
+# Copy resources directories from build phase
+COPY --from=build /texera/common/config/src/main/resources
/texera/common/config/src/main/resources
+COPY --from=build /texera/workflow-compiling-service/src/main/resources
/texera/workflow-compiling-service/src/main/resources
CMD ["bin/workflow-compiling-service"]
diff --git a/bin/deployment/workflow-computing-unit-managing-service.dockerfile
b/bin/workflow-computing-unit-managing-service.dockerfile
similarity index 63%
rename from bin/deployment/workflow-computing-unit-managing-service.dockerfile
rename to bin/workflow-computing-unit-managing-service.dockerfile
index f98732d6bd..30924c8687 100644
--- a/bin/deployment/workflow-computing-unit-managing-service.dockerfile
+++ b/bin/workflow-computing-unit-managing-service.dockerfile
@@ -18,10 +18,13 @@
FROM sbtscala/scala-sbt:eclipse-temurin-jammy-11.0.17_8_1.9.3_2.13.11 AS build
# Set working directory
-WORKDIR /core
+WORKDIR /texera
-# Copy all projects under core to /core
-COPY core/ .
+# Copy modules for building the service
+COPY common/ common/
+COPY computing-unit-managing-service/ computing-unit-managing-service/
+COPY project/ project/
+COPY build.sbt build.sbt
# Update system and install dependencies
RUN apt-get update && apt-get install -y \
@@ -30,25 +33,24 @@ RUN apt-get update && apt-get install -y \
libpq-dev \
&& apt-get clean
-WORKDIR /core
# Add .git for runtime calls to jgit from OPversion
-COPY .git ../.git
+COPY .git .git
RUN sbt clean ComputingUnitManagingService/dist
# Unzip the texera binary
-RUN unzip
computing-unit-managing-service/target/universal/computing-unit-managing-service-*.zip
-d target/
+RUN unzip
computing-unit-managing-service/target/universal/computing-unit-managing-service-*.zip
-d target/
FROM eclipse-temurin:11-jre-jammy AS runtime
-WORKDIR /core
+WORKDIR /texera
-COPY --from=build /.git /.git
+COPY --from=build /texera/.git /texera/.git
# Copy the built texera binary from the build phase
-COPY --from=build /core/target/computing-unit-managing-service-* /core/
-# Copy resources directories under /core from build phase
-COPY --from=build /core/config/src/main/resources
/core/config/src/main/resources
-COPY --from=build /core/computing-unit-managing-service/src/main/resources
/core/computing-unit-managing-service/src/main/resources
+COPY --from=build /texera/target/computing-unit-managing-service-* /texera/
+# Copy resources directories from build phase
+COPY --from=build /texera/common/config/src/main/resources
/texera/common/config/src/main/resources
+COPY --from=build /texera/computing-unit-managing-service/src/main/resources
/texera/computing-unit-managing-service/src/main/resources
CMD ["bin/computing-unit-managing-service"]
diff --git a/bin/deployment/y-websocket-server/Dockerfile
b/bin/y-websocket-server/Dockerfile
similarity index 100%
rename from bin/deployment/y-websocket-server/Dockerfile
rename to bin/y-websocket-server/Dockerfile
diff --git a/bin/deployment/y-websocket-server/package.json
b/bin/y-websocket-server/package.json
similarity index 100%
rename from bin/deployment/y-websocket-server/package.json
rename to bin/y-websocket-server/package.json
