CarlosJuncher03 commented on issue #7991: URL: https://github.com/apache/hop/issues/7991#issuecomment-5322877938
Yes, I agree with your point. On second thought, I realized my original idea was trying to solve too many things within Hop itself. A simpler approach would probably make more sense. Instead of Hop creating and managing the entire isolated execution environment, I believe the main requirement could simply be extending the existing remote execution mechanism so that the project and environment context are also sent to the remote runtime. The Remote Pipeline Engine already handles sending and executing the pipeline on a Hop Server. What I envision is allowing this remote execution to explicitly use the selected project and environment, similar to how this can already be specified when starting `hop-run` or `hop-server`. The remote runtime would remain responsible for accessing the project and its dependencies. How that project is made available—Git, PVC, shared filesystem, container image, CI/CD, etc.—would remain outside Hop's scope. Thus, the basic workflow would remain: select the remote destination; specify the project/environment context; execute a pipeline or workflow remotely; monitor status and logs via the Hop GUI. As a possible evolution of this idea, it would also be interesting to have a Run Configuration capable of launching an ephemeral runtime from a pre-defined container image. For example, a configuration could specify the Docker image and the project/environment parameters, and Hop could launch this runtime as a Docker container or as a Kubernetes Pod/Job. In this case, the infrastructure would still be responsible for making the project, configurations, secrets, drivers, and other dependencies available to that container. I would view this as a second layer of functionality, rather than a requirement for solving the simpler aspect of remote execution. I also believe this idea could be evaluated in conjunction with activity #7992. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
