bobbai00 commented on PR #3755: URL: https://github.com/apache/texera/pull/3755#issuecomment-3322654991
> > > Why don't we introduce another endpoint? What's the reason for `when installing texera in single-node mode, the multipart upload is unable to reach to the host network.` It's not a good practice that when A does not work with B, we introduce C. The codebase will become extremely large and unmanageable. > > > > > > I agree with the principle. > > In single node case, all containers are running in its own internal network and they cannot reach to host machine via `localhost`. Therefore, when multipart uploads the example dataset, the service failed to do `PUT http://localhost:8080/presigned-url`. > > Since current upload API forces the usage of multipart upload, I add a new parameter to toggle it. The reason why I didn't make it a separate endpoint is because at high level, either way (multipart or not) is logically doing file uploading. Therefore they should be handled by the same endpoint. > > I think we should manage functions based on their underlying logic, rather than just looking at the function name at a high level. I reviewed the code carefully, and with this flag in place, the function essentially skips 99% of its existing logic, making none of the code being reused. In effect, the function now contains two separate implementations: if the flag is true, it executes logic A; otherwise, it executes logic B. > > That aside, I still don’t fully understand the issue: why does the service fail on PUT http://localhost:8080/presigned-url, while LakeFSStorageClient is able to access the service without problems? The key difference lies in where the code executes and which network it uses: LakeFS Storage Client works because: - When it connects to LakeFS, since both LakeFS container and microservices are in the same internal network, communication can succeed using the correct internal service hostname, e.g. http://lakefs:8000, Multipart upload fails because: - The presigned URL is generated with prefix http://localhost:8000/ (the address of minio available on the host machine) - This URL is returned to the client, in this case, the example-data-loader container - When the client tries to PUT to localhost:8000, it's attempting to connect to port 8000 within the container, not the host machine -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
