Hi Romain,
it sounds good to me. I think any format should be packaged as an extension.
The only point is that some core transforms expect specific format, so, it means
that users will have to remember to add the avro extension to use some
transforms (or the transforms could be an extension as well). I have to check
the transforms working like this.
Regards
JB
On 12/19/2017 10:26 AM, Romain Manni-Bucau wrote:
Hi guys,
checking security issues of the project I'm responsible of (which integrates
beam) I realized the java sdk core module depends on avro. On security point of
view it is a blocker cause of the legacy avro brings (jackson from codehaus etc)
but all that can be fixed. However I would like to take this opportunity to open
the topic of avro in the core dependencies.
From my point of view it doesn't make much sense cause it is just one of the
serialization you can use with the file IO and it is highly not probable all the
potential formats are imported in the core. Since it is a very local usage and
not a core feature I think it should be extracted - we can discuss extracting
the actual transforms from the core in another thread, it would make a lot of
sense IMHO but not the current topic.
Therefore I'd like to propose to extract avro format - like others - in an
extension and remove it as a hard requirement of the core to bring more
consistency and modularity to beam.
Wdyt?
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau>
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
