Partially. Will run with beam in half of the cases or without in the remaining 50% (and in this case the dependencies+api are currently blocking). My constraint is that to activate any feature i must be able to cover both cases.
Le 6 avr. 2018 22:14, "Reuven Lax" <re...@google.com> a écrit : > So is this project of yours also built on top of Beam, or is it unrelated? > > On Fri, Apr 6, 2018 at 1:12 PM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> Issues forking are: >> >> 1. I have to drop beam FileIO (in all its flavors) which means not taking >> any benefit from beam in that area which is 50% of beam gain (the other >> being the portability) >> 2. I have to maintain a bridge for all filesystem impl - being said it >> still misses some info ATM >> 3. It is still in beam sdk so "here" which is misleading for dev (can be >> fixed if beam becomes modular) >> >> As a side note - and to link with another thread topic: with vfs as an >> abstraction i dont have that issue at all. >> >> Le 6 avr. 2018 20:35, "Reuven Lax" <re...@google.com> a écrit : >> >>> Personally, this is a case where I think forking might be a better >>> option, even though I'm not generally a fan of duplicating code. >>> >>> In past projects, depending on internal modules of other projects never >>> seems lead to good outcomes. FileSystem exists for Beam today, and Beam >>> might make changes to it that cause problems for your other project. I >>> would recommend starting by forking if it serves your needs. >>> >>> Reuven >>> >>> On Fri, Apr 6, 2018 at 8:17 AM Romain Manni-Bucau <rmannibu...@gmail.com> >>> wrote: >>> >>>> Hi guys, >>>> >>>> I have a use case where I'd like to be able to expose to a user some >>>> file system navigation and enable him to visualize the file system (as in >>>> beam sense) >>>> >>>> Technically it is a matter of being able to use glob pattern to browse >>>> the file system using match(specs). >>>> >>>> What is important in that use case is to align the visualization and >>>> the potential runtime to have the same impl/view and not have to split it >>>> in 2 code branches which can lead to inconsistency. >>>> >>>> Therefore i'd like to be able to reuse beam FileSystem but I have a few >>>> blockers: >>>> >>>> 1. it is nested in sdk-java-core which brings 2 drawbacks >>>> a. it brings the whole beam sdk which is not desired in that part of >>>> the app (should not be visible in the classpath) >>>> b. the dependency stack is just unpractiable (guava, jackson, >>>> byte-buddy, avro, joda, at least, are not desired at all here) and a shade >>>> makes it way too fat to be a valid dependency for that usage >>>> 2. I don't know how to configure the FS from one of its instance (I'd >>>> like to be able to get its options class like >>>> FileSystem#getConfigurationType >>>> returning a PipelineOptions) >>>> >>>> Do you think it is possible to extract the filesystem API in a >>>> dependency free beam subproject (or at least submodule) and add the >>>> configuration hint in the API? >>>> >>>> Romain Manni-Bucau >>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>>> <https://rmannibucau.metawerx.net/> | Old Blog >>>> <http://rmannibucau.wordpress.com> | Github >>>> <https://github.com/rmannibucau> | LinkedIn >>>> <https://www.linkedin.com/in/rmannibucau> | Book >>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >>>> >>>
