On Mon, Mar 5, 2018 at 11:38 AM Reuven Lax <[email protected]> wrote: > What about a beam Filesystem impl on top of Vfs as an alternative > short-term solution? This would allow Vfs to be used with any IO. >
Yes, I think this is the VfsIO that was proposed. > On Mon, Mar 5, 2018 at 11:37 AM Robert Bradshaw <[email protected]> > wrote: > >> >> On Mon, Mar 5, 2018 at 11:23 AM Romain Manni-Bucau <[email protected]> >> wrote: >> >>> >>> 2018-03-05 20:04 GMT+01:00 Chamikara Jayalath <[email protected]>: >>> >>>> I assume you mean https://commons.apache.org/proper/commons-vfs/. >>>> >>>> I'm not sure if we considered this when we originally implemented our >>>> own file-system abstraction but based on a quick look seems like this is >>>> Java only. >>>> >>> >>> Yes, java only >>> >>> >>>> >>>> I think having a similar file-system abstraction for various languages >>>> is a plus point for Beam. May be we should consider a Java file-system >>>> implementation for VFS ? >>>> >>> >>> Can be an option but when I see the current complexity I'm not sure >>> mixing 2 abstractions would help, maybe just a VfsIO for java users would >>> be good enough - thinking out loud. >>> >>> What sounds clear to me is that each language will need its own >>> abstraction - which kind of join your proposal. However we can still make >>> it smooth and easy on the java side - which >>> will likely stay mainstream for still some years - using vfs as our java >>> impl instead of reimplementing the full abstraction? This way we keep our >>> *API* but we drop beam *impl* to just reuse VFS. >>> >>> PS: for gcs https://github.com/ltouati/vfs-gcs can be a good example on >>> how it can work. >>> >> >> I think a VfsIO makes a lot of sense in the short term, and will give use >> the experience needed to decide if we can move solely to VFS (for Java at >> least) for implementation, and possibly API in a future major release, in >> the long run. >> >
