I would also suggest looking at NFS client implementations in Java that would allow you to talk to the NFS server without needing to mount it within the OS. A quick search yielded https://github.com/raisercostin/yanfs or https://github.com/EMCECS/nfs-client-java
On Tue, Jan 31, 2023 at 3:31 PM Chad Dombrova <chad...@gmail.com> wrote: > Thanks for the info. We are going to test this further and we'll let you > know how it goes. > > -chad > > > On Mon, Jan 30, 2023 at 2:14 PM Valentyn Tymofieiev <valen...@google.com> > wrote: > >> It applies to custom containers as well. You can find the container >> manifest in the GCE VM metadata, and it should have an entry for privileged >> mode. The reason for this was to enable GPU accelerator support, but agree >> with Robert that it is not part of any contracts, so in theory this could >> change or perhaps be more strictly limited to accelerator support. In fact, >> originally, this was only enabled for pipelines using accelerators but for >> purely internal implementation details I believe it is currently enabled >> for all pipelines. >> >> So for prototyping purposes I think you could try it, but I can't make >> any guarantees in this thread that privileged mode will continue to work. >> >> cc: @Aaron Li <aaronle...@google.com> FYI >> >> >> On Mon, Jan 30, 2023 at 12:16 PM Robert Bradshaw <rober...@google.com> >> wrote: >> >>> I'm also not sure it's part of the contract that the containerization >>> technology we use will always have these capabilities. >>> >>> On Mon, Jan 30, 2023 at 10:53 AM Chad Dombrova <chad...@gmail.com> >>> wrote: >>> > >>> > Hi Valentyn, >>> > >>> >> >>> >> Beam SDK docker containers on Dataflow VMs are currently launched in >>> privileged mode. >>> > >>> > >>> > Does this only apply to stock sdk containers? I'm asking because we >>> use a custom sdk container that we build. We've tried various ways of >>> running mount from within our custom beam container in Dataflow and we >>> could not get it to work, while the same thing succeeds in local tests and >>> in our CI (gitlab). The assessment at the time (this was maybe a year ago) >>> was that the container was not running in privileged mode, but if you think >>> that's incorrect we can revisit this and report back with some error logs. >>> > >>> > -chad >>> > >>> >>
