This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-6111-5c4a963f73c5d9e30eddde32b469b6c0069aed24 in repository https://gitbox.apache.org/repos/asf/texera.git
commit 6829fae77796d95a8e08a4641cd1019453a71cde Author: Yicong Huang <[email protected]> AuthorDate: Sat Jul 4 15:49:06 2026 -0700 docs: fix stale core/ directory paths in contribution docs (#6111) ### What changes were proposed in this PR? Docs still referenced the pre-ASF `core/` source tree. This PR updates every such path to the current layout: | Stale | Current | | --- | --- | | `core/workflow-operator/.../edu/uci/ics/amber/operator` | `common/workflow-operator/.../org/apache/texera/amber/operator` | | `core/gui`, `/core/new-gui` (operator images) | `frontend/src/assets/operator_images` | | `cd core` / `cd core/gui` (test instructions) | repo root / `cd frontend` | | `config/src/main/resources/...` | `common/config/src/main/resources/...` | | `core/amber/.../generator_operator.py` (renamed) | `amber/.../generator_operator_integer.py` | Also removes `docs/contribution-guidelines/micro-services-local-dev.md`: it documents the now-finished `core/micro-services` migration and build scripts that no longer exist; local development is covered by the developer guide and `bin/local-dev.sh`. No other doc links to that page. ### Any related issues, documentation, discussions? Closes #6105 ### How was this PR tested? Docs-only change. Every new path/link target was verified to exist in the repo: ``` ls common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/{regex,huggingFace,map,filter,flatmap,aggregate} ls frontend/src/assets/operator_images ls amber/src/main/python/pytexera/udf/examples/generator_operator_integer.py grep -rn "core/" docs/contribution-guidelines docs/tutorials # no stale hits remain ``` ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Fable 5) Co-authored-by: Claude Fable 5 <[email protected]> --- docs/contribution-guidelines/_index.md | 5 +- .../guide-for-developers.md | 6 +-- .../guide-to-implement-java-operator.md | 12 ++--- .../micro-services-local-dev.md | 58 ---------------------- docs/tutorials/guide-to-use-python-udf.md | 2 +- 5 files changed, 12 insertions(+), 71 deletions(-) diff --git a/docs/contribution-guidelines/_index.md b/docs/contribution-guidelines/_index.md index 72017aa635..be4a7eaad1 100644 --- a/docs/contribution-guidelines/_index.md +++ b/docs/contribution-guidelines/_index.md @@ -86,16 +86,15 @@ Avoid including: ``` 3. Execute tests: ```bash - cd core sbt test ``` -> For IntelliJ users: ensure the working directory matches the module (`amber` for engine tests, `core` for services). +> For IntelliJ users: ensure the working directory matches the module (`amber` for engine tests, the repo root for services). ### Frontend (Angular) 1. Run unit tests: ```bash - cd core/gui + cd frontend ng test --watch=false ``` 2. Format code: diff --git a/docs/contribution-guidelines/guide-for-developers.md b/docs/contribution-guidelines/guide-for-developers.md index 02e6a92423..ac73828e28 100644 --- a/docs/contribution-guidelines/guide-for-developers.md +++ b/docs/contribution-guidelines/guide-for-developers.md @@ -263,7 +263,7 @@ This command will optimize the frontend code to make it run faster. This step wi ## 3. Email Notification (Optional) </summary> -1. Set `smtp` in `config/src/main/resources/user-system.conf`. You need an App password if the account has 2FA. +1. Set `smtp` in `common/config/src/main/resources/user-system.conf`. You need an App password if the account has 2FA. 2. Log in to Texera with an admin account. 3. Open the Gmail dashboard under the admin tab. 5. Send a test email. @@ -286,10 +286,10 @@ This part is optional; you only need to do this if you are working on a specific Note: Jooq creates DAO for simple operations if the requested SQL query is complex, then the developer can use the generated Table classes to implement the operation ### Disable password login -Edit `config/src/main/resources/gui.conf`, change `local-login` to `false`. +Edit `common/config/src/main/resources/gui.conf`, change `local-login` to `false`. ### Enforce invite only -Edit `config/src/main/resources/user-system.conf`, change `invite-only` to `true`. +Edit `common/config/src/main/resources/user-system.conf`, change `invite-only` to `true`. ### Backend endpoints Role Annotation There are two types of permissions for the backend endpoints: diff --git a/docs/contribution-guidelines/guide-to-implement-java-operator.md b/docs/contribution-guidelines/guide-to-implement-java-operator.md index 15a4dc8c57..34ac4d0086 100644 --- a/docs/contribution-guidelines/guide-to-implement-java-operator.md +++ b/docs/contribution-guidelines/guide-to-implement-java-operator.md @@ -8,7 +8,7 @@ In this page, we'll explain the basic concepts in Texera and use examples to sho ### Code structure of every operator: -Every operator ideally has three classes that are found in each operator package in `core\workflow-operator\src\main\scala\edu\uci\ics\amber\operator` +Every operator ideally has three classes that are found in each operator package in `common/workflow-operator/src/main/scala/org/apache/texera/amber/operator` * LogicalOp * OperatorExecutor * OperatorExecutorConfig @@ -49,7 +49,7 @@ schema.getAttributes().get(1) // Attribute("tweet", AttributeType.String) A regular expression operator matches a regular expression (regex) on each input tuple. For example, if we search the regex "weather" on the `tweet` attribute, then only tuple 2 will be the result. In other words, the regular expression operator is a kind of `filter()` operation in many programming languages. -To implement a regular expression operator, you will first need to write an `LogicalOp`. The following code is part of class [`RegexOpDesc`](https://github.com/apache/texera/blob/main/core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/regex/RegexOpDesc.scala) . +To implement a regular expression operator, you will first need to write an `LogicalOp`. The following code is part of class [`RegexOpDesc`](https://github.com/apache/texera/blob/main/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/regex/RegexOpDesc.scala) . ```scala class RegexOpDesc extends FilterOpDesc { @@ -127,7 +127,7 @@ abstract class LogicalOp extends PortDescriptor with Serializable { Now this operator will be automatically available in the frontend. We can now start the system and test this operator. -To add an image for this operator, go to `core/gui/src/assets/operator_images`, then add an image with the _**SAME NAME**_ as what's specified in the operator registration. The image file should be in `png` format, with a transparent background, black and white, and should be square. +To add an image for this operator, go to `frontend/src/assets/operator_images`, then add an image with the _**SAME NAME**_ as what's specified in the operator registration. The image file should be in `png` format, with a transparent background, black and white, and should be square. For example, for the regex operator, the code `new Type(value = classOf[RegexOpDesc], name = "Regex")` specified a name `Regex`, then the image file name should be `Regex.png`. @@ -146,7 +146,7 @@ id tweet sentiment ``` -The following code is the implementation of class [`SentimentAnalysisOpDesc`](https://github.com/apache/texera/blob/main/core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/huggingFace/HuggingFaceSentimentAnalysisOpDesc.scala) in Java. +The following code is the implementation of class [`SentimentAnalysisOpDesc`](https://github.com/apache/texera/blob/main/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceSentimentAnalysisOpDesc.scala) in Java. ```java public class SentimentAnalysisOpDesc extends MapOpDesc { @@ -202,7 +202,7 @@ In Texera, currently we have 4 pre-defined operations you can extend. - `flatmap()`: for each input tuple, transforms it to a list of output tuples. - `aggregate()`: performs an aggregation, such as sum, count, average, etc. -To implement an operator, you can first check if your operator can be implemented using the 4 pre-defined operations. You can find these pre-defined operations under [`texera/workflow/common/operators`](https://github.com/Texera/texera/tree/master/core/amber/src/main/scala/edu/uci/ics/texera/workflow/common/operators). Your own operator implementation should be in [`texera/workflow/operators/youroperator`](https://github.com/Texera/texera/tree/master/core/amber/src/main/scala/edu/uci/ics [...] +To implement an operator, you can first check if your operator can be implemented using the 4 pre-defined operations. You can find these pre-defined operations under [`common/workflow-operator`](https://github.com/apache/texera/tree/main/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator) (e.g., the `map`, `filter`, `flatmap`, and `aggregate` packages). Your own operator implementation should be in its own package under [the same directory](https://github.com/apache [...] ### Low-level OperatorExecutor API For more complicated operators, if they cannot be implemented using these operations, then you need to implement `OperatorExecutor` using the following low-level interface. @@ -278,6 +278,6 @@ Texera's backend is responsible for determining the UI information to the fronte ``` ### Registration and icon -In the file `amber/src/main/scala/edu/uci/ics/texera/workflow/common/operators/LogicalOp.scala`, you will find a list of all registered operators, complete with their descriptor classes and names. After adding an operator's information, you can assign an icon to it. All operator icons are stored in the `/core/new-gui/src/assets/operator_images` directory. It's essential to ensure that the icon filename matches its respective operator descriptor name. +In the file `common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/LogicalOp.scala`, you will find a list of all registered operators, complete with their descriptor classes and names. After adding an operator's information, you can assign an icon to it. All operator icons are stored in the `frontend/src/assets/operator_images` directory. It's essential to ensure that the icon filename matches its respective operator descriptor name. diff --git a/docs/contribution-guidelines/micro-services-local-dev.md b/docs/contribution-guidelines/micro-services-local-dev.md deleted file mode 100644 index 7cb76b43f1..0000000000 --- a/docs/contribution-guidelines/micro-services-local-dev.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -title: "Build, Run and Configure micro‐services in local development environment" -weight: 60 ---- - -This Document is aim to provide a instruction on how to setup the local development environment for developing and deploying the `core/micro-services`. - -## Prerequisite - -This document requires you to finish all the setup of Texera local development environment described in `https://github.com/Texera/texera/wiki`. - -## What is `micro-services`? - -`core/micro-services` is a sbt-managed project added by the PR https://github.com/Texera/texera/pull/2922. The ongoing code separation effort will gradually migrate all the services in `core/amber` to `core/micro-services`. - -## How to directly build and run the micro-services directly - -If you just want to run some services under `micro-services`, you can use some provided shell scripts. - -### `WorkflowCompilingService` - -```shell -cd texera/core - -# make sure to give scripts the execution permission -chmod +x scripts/build-workflow-compiling-service.sh -chmod +x scripts/workflow-compiling-service.sh - -# Build the WorkflowCompilingService -scripts/build-workflow-compiling-service.sh - -# Run the WorkflowCompilingService -scripts/workflow-compiling-service.sh -``` - -## How to set up the development environment - -As there are many sub sbt projects under `micro-services`, Intellij is the most suitable IDE for setting up the whole environment - -### Use Intellij (Most Recommended) - -1. Open the folder `texera/core/micro-services` through `Open Project` in Intellij -<img width="716" alt="Screenshot 2024-11-19 at 6 00 08 PM" src="/images/github-assets/4e446332-7cfa-4974-b59b-2088a7a2d921.png"> - -Once you open it, Intellij will auto-detect the sbt setting and start to load the project. After loading you should see the sbt tab, which has the `micro-services` as the root project and several other services as the sub-projects: -<img width="200" alt="Screenshot 2024-11-19 at 6 05 15 PM" src="/images/github-assets/24ba1a31-1c82-4441-b525-7facc00c3ada.png"> - - -2. Run `sbt clean compile` command in folder `core/micro-services`. This command will compile everything under `micro-services` and generate proto-specified codes. - - - - - - - - - diff --git a/docs/tutorials/guide-to-use-python-udf.md b/docs/tutorials/guide-to-use-python-udf.md index 6cd5e536a3..e9de246c8c 100644 --- a/docs/tutorials/guide-to-use-python-udf.md +++ b/docs/tutorials/guide-to-use-python-udf.md @@ -137,7 +137,7 @@ class GenerateOperator(UDFSourceOperator): This `produce()` API returns an iterator of `TupleLike`, `TableLike`, or simply `None`. -See [Generator Operator](https://github.com/Texera/texera/blob/master/core/amber/src/main/python/pytexera/udf/examples/generator_operator.py) for an example of 1-out UDF. +See [Generator Operator](https://github.com/apache/texera/blob/main/amber/src/main/python/pytexera/udf/examples/generator_operator_integer.py) for an example of 1-out UDF. #### 2-in UDF
