So is this project of yours also built on top of Beam, or is it unrelated? On Fri, Apr 6, 2018 at 1:12 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote:
> Issues forking are: > > 1. I have to drop beam FileIO (in all its flavors) which means not taking > any benefit from beam in that area which is 50% of beam gain (the other > being the portability) > 2. I have to maintain a bridge for all filesystem impl - being said it > still misses some info ATM > 3. It is still in beam sdk so "here" which is misleading for dev (can be > fixed if beam becomes modular) > > As a side note - and to link with another thread topic: with vfs as an > abstraction i dont have that issue at all. > > Le 6 avr. 2018 20:35, "Reuven Lax" <re...@google.com> a écrit : > >> Personally, this is a case where I think forking might be a better >> option, even though I'm not generally a fan of duplicating code. >> >> In past projects, depending on internal modules of other projects never >> seems lead to good outcomes. FileSystem exists for Beam today, and Beam >> might make changes to it that cause problems for your other project. I >> would recommend starting by forking if it serves your needs. >> >> Reuven >> >> On Fri, Apr 6, 2018 at 8:17 AM Romain Manni-Bucau <rmannibu...@gmail.com> >> wrote: >> >>> Hi guys, >>> >>> I have a use case where I'd like to be able to expose to a user some >>> file system navigation and enable him to visualize the file system (as in >>> beam sense) >>> >>> Technically it is a matter of being able to use glob pattern to browse >>> the file system using match(specs). >>> >>> What is important in that use case is to align the visualization and the >>> potential runtime to have the same impl/view and not have to split it in 2 >>> code branches which can lead to inconsistency. >>> >>> Therefore i'd like to be able to reuse beam FileSystem but I have a few >>> blockers: >>> >>> 1. it is nested in sdk-java-core which brings 2 drawbacks >>> a. it brings the whole beam sdk which is not desired in that part of the >>> app (should not be visible in the classpath) >>> b. the dependency stack is just unpractiable (guava, jackson, >>> byte-buddy, avro, joda, at least, are not desired at all here) and a shade >>> makes it way too fat to be a valid dependency for that usage >>> 2. I don't know how to configure the FS from one of its instance (I'd >>> like to be able to get its options class like >>> FileSystem#getConfigurationType returning a PipelineOptions) >>> >>> Do you think it is possible to extract the filesystem API in a >>> dependency free beam subproject (or at least submodule) and add the >>> configuration hint in the API? >>> >>> Romain Manni-Bucau >>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>> <https://rmannibucau.metawerx.net/> | Old Blog >>> <http://rmannibucau.wordpress.com> | Github >>> <https://github.com/rmannibucau> | LinkedIn >>> <https://www.linkedin.com/in/rmannibucau> | Book >>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >>> >>
