robertwb commented on PR #21782: URL: https://github.com/apache/beam/pull/21782#issuecomment-1151756221
I don't think this is the right direction either. If an external transform needs a certain set of files, that transform should be responsible for making the declaration, not the callee (and especially, it seems, the callee from another language). I think the correct solution is to us resource hints (e.g. https://beam.apache.org/documentation/runtime/resource-hints/). There's also the issue of finding the resources at their new locations (on the worker). Presumably one would ask for a resource (e.g. file) to be staged against some kind of identifier, and on the worker one could use this identifier to look up the copied resource in its final destination. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
