I did a PR for that but it just beings connectivity to beam. To solve any issue the opposite is the only valid option.
Le 6 avr. 2018 22:31, "Reuven Lax" <re...@google.com> a écrit : In the other thread, we suggested writing a Beam FileSystem impl that wraps VFS. Is that a path forward here? Then you can build on top of VFS instead, and simply layer VfsFilesystem on top of it when running on Beam. On Fri, Apr 6, 2018 at 1:23 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Partially. Will run with beam in half of the cases or without in the > remaining 50% (and in this case the dependencies+api are currently > blocking). My constraint is that to activate any feature i must be able to > cover both cases. > > > > > Le 6 avr. 2018 22:14, "Reuven Lax" <re...@google.com> a écrit : > >> So is this project of yours also built on top of Beam, or is it unrelated? >> >> On Fri, Apr 6, 2018 at 1:12 PM Romain Manni-Bucau <rmannibu...@gmail.com> >> wrote: >> >>> Issues forking are: >>> >>> 1. I have to drop beam FileIO (in all its flavors) which means not >>> taking any benefit from beam in that area which is 50% of beam gain (the >>> other being the portability) >>> 2. I have to maintain a bridge for all filesystem impl - being said it >>> still misses some info ATM >>> 3. It is still in beam sdk so "here" which is misleading for dev (can be >>> fixed if beam becomes modular) >>> >>> As a side note - and to link with another thread topic: with vfs as an >>> abstraction i dont have that issue at all. >>> >>> Le 6 avr. 2018 20:35, "Reuven Lax" <re...@google.com> a écrit : >>> >>>> Personally, this is a case where I think forking might be a better >>>> option, even though I'm not generally a fan of duplicating code. >>>> >>>> In past projects, depending on internal modules of other projects never >>>> seems lead to good outcomes. FileSystem exists for Beam today, and Beam >>>> might make changes to it that cause problems for your other project. I >>>> would recommend starting by forking if it serves your needs. >>>> >>>> Reuven >>>> >>>> On Fri, Apr 6, 2018 at 8:17 AM Romain Manni-Bucau < >>>> rmannibu...@gmail.com> wrote: >>>> >>>>> Hi guys, >>>>> >>>>> I have a use case where I'd like to be able to expose to a user some >>>>> file system navigation and enable him to visualize the file system (as in >>>>> beam sense) >>>>> >>>>> Technically it is a matter of being able to use glob pattern to browse >>>>> the file system using match(specs). >>>>> >>>>> What is important in that use case is to align the visualization and >>>>> the potential runtime to have the same impl/view and not have to split it >>>>> in 2 code branches which can lead to inconsistency. >>>>> >>>>> Therefore i'd like to be able to reuse beam FileSystem but I have a >>>>> few blockers: >>>>> >>>>> 1. it is nested in sdk-java-core which brings 2 drawbacks >>>>> a. it brings the whole beam sdk which is not desired in that part of >>>>> the app (should not be visible in the classpath) >>>>> b. the dependency stack is just unpractiable (guava, jackson, >>>>> byte-buddy, avro, joda, at least, are not desired at all here) and a shade >>>>> makes it way too fat to be a valid dependency for that usage >>>>> 2. I don't know how to configure the FS from one of its instance (I'd >>>>> like to be able to get its options class like >>>>> FileSystem#getConfigurationType >>>>> returning a PipelineOptions) >>>>> >>>>> Do you think it is possible to extract the filesystem API in a >>>>> dependency free beam subproject (or at least submodule) and add the >>>>> configuration hint in the API? >>>>> >>>>> Romain Manni-Bucau >>>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>>>> <https://rmannibucau.metawerx.net/> | Old Blog >>>>> <http://rmannibucau.wordpress.com> | Github >>>>> <https://github.com/rmannibucau> | LinkedIn >>>>> <https://www.linkedin.com/in/rmannibucau> | Book >>>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >>>>> >>>>
