hhvu0102 opened a new issue, #4074: URL: https://github.com/apache/texera/issues/4074
### What happened? <img width="671" height="645" alt="Image" src="https://github.com/user-attachments/assets/533395e1-3860-4f00-9b08-a4e46cfe71d8"; /> When there are multiple ports, each port is called multiple times. For example, using this code in the `multiport tests` node: ``` class ProcessTableOperator(UDFTableOperator): @overrides def process_table(self, table: Table, port: int) -> Iterator[Optional[TableLike]]: df0 = pd.DataFrame(table) print(f'port is {port}') df1 = pd.DataFrame(table) print(f'port is {port}') yield None ``` The `f'port is {port}` is printed twice <img width="774" height="311" alt="Image" src="https://github.com/user-attachments/assets/2915907d-6a3b-4e12-a7fc-edd7d0fc1598"; /> This signals to me that the function is called as many times as the number of ports, which should not be a normal behavior in my opinion. Additionally, `port 1` is called before `port 0` because `port 1` data is smaller and is generated upstream first, which Chris and Jiadong mentioned is what's intended. However, if a user is unaware, they may write a code to read in the data from `port 0` first because that's the intended order, which will be an error as data is not loaded. Currently I write my own wrapper so inputs from each port can be recognized correctly, but do you have any example of how you often handle this case? I can make a new issue for this question if that's easier. ### How to reproduce? Dataset: `samn16081314-downsampled-5k`; workflow: `PanKbase_example_5k_multi-port` - I have shared these via email to Meng, Jiadong, Chris and Chen. Please let me know if I should share them with any other team members! To run this workflow, you'll need to install some packages yourself. In Python: ``` pip install pysam pip install cellbender pip install scikit-image ``` In R: ``` if (!require("BiocManager", quietly = TRUE)) install.packages("BiocManager") BiocManager::install("DropletUtils") ``` ### Version 1.1.0-incubating (Pre-release/Master) ### Commit Hash (Optional) NA ### What browsers are you seeing the problem on? Chrome, using `https://texera.dknet-ai.org/` ### Relevant log output <img width="774" height="311" alt="Image" src="https://github.com/user-attachments/assets/2915907d-6a3b-4e12-a7fc-edd7d0fc1598"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
