Hi Ismaël,
1. We can always use license and version Maven plugin to check that. However,
it's under the responsibility of the PMC to enforce security/CVE/legal points.
2. Agree, CVE (via [email protected]) should be raised to Avro.
Take a look on https://www.apache.org/security/
Regards
JB
On 12/19/2017 11:28 AM, Ismaël Mejía wrote:
There are two important points raised here that we are missing in the
discussion:
1. We don’t have any kind of security validation for Beam and its
dependencies. This is important and it would be really nice if we
could get this improved and some sort of automation on this. However I
doubt there is a free service for this. Does the ASF has something for
this? or some of the companies involved in the project could help us
setup this?
Romain, can you share the security report with the Beam community so
we can be aware of this an other security issues at least for the
moment?
2. The issues raised on Avro security should also be raised into the
Avro community. Beam is using the latest Avro release so in part Beam
cannot be blamed of dependency negligence since we follow the latest
Avro release.
Are the issues about Avro itself or about the Jackson version because
if it is this the case I don’t think the things will move quickly:
https://issues.apache.org/jira/browse/AVRO-1126
On Tue, Dec 19, 2017 at 11:23 AM, Jean-Baptiste Onofré <[email protected]>
wrote:
Agree, it's what I meant by "core transforms".
Regards
JB
On 12/19/2017 11:18 AM, Reuven Lax wrote:
Keep in mind that today Avro is one of the most common coders used for
user data types, not just for file IO. The reason for this is that it's the
easiest way to get a coder for a users POJO - you simply annotate the POJO
with @DefaultCoder(AvroCoder.class), and it works. This is the coder used
for all internal shuffles (e.g. GroupByKey).
I would argue that most users don't really care about Avro for this use
case, what they really want is a way of saying "make this POJO work" and
Avro is the only way we give them. This was part of my argument in the
schema docs. However the status quo is that they use Avro here.
Reuven
On Tue, Dec 19, 2017 at 1:32 AM, Jean-Baptiste Onofré <[email protected]
<mailto:[email protected]>> wrote:
Hi Romain,
it sounds good to me. I think any format should be packaged as an
extension.
The only point is that some core transforms expect specific format,
so, it
means that users will have to remember to add the avro extension to
use some
transforms (or the transforms could be an extension as well). I have
to
check the transforms working like this.
Regards
JB
On 12/19/2017 10:26 AM, Romain Manni-Bucau wrote:
Hi guys,
checking security issues of the project I'm responsible of (which
integrates beam) I realized the java sdk core module depends on
avro. On
security point of view it is a blocker cause of the legacy avro
brings
(jackson from codehaus etc) but all that can be fixed. However I
would
like to take this opportunity to open the topic of avro in the
core
dependencies.
From my point of view it doesn't make much sense cause it is
just one
of the serialization you can use with the file IO and it is highly
not
probable all the potential formats are imported in the core. Since
it is
a very local usage and not a core feature I think it should be
extracted
- we can discuss extracting the actual transforms from the core in
another thread, it would make a lot of sense IMHO but not the
current topic.
Therefore I'd like to propose to extract avro format - like others
- in
an extension and remove it as a hard requirement of the core to
bring
more consistency and modularity to beam.
Wdyt?
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau
<https://twitter.com/rmannibucau>> | Blog
<https://rmannibucau.metawerx.net/
<https://rmannibucau.metawerx.net/>> | Old Blog
<http://rmannibucau.wordpress.com
<http://rmannibucau.wordpress.com>> |
Github <https://github.com/rmannibucau
<https://github.com/rmannibucau>> | LinkedIn
<https://www.linkedin.com/in/rmannibucau
<https://www.linkedin.com/in/rmannibucau>>
-- Jean-Baptiste Onofré
[email protected] <mailto:[email protected]>
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
