Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-11-05 Thread Peter Somogyi
+1 Definitely valuable for HBase users to be aware of Hadoop CVEs. Peter On Tue, Nov 6, 2018 at 6:16 AM Stack wrote: > On Mon, Nov 5, 2018 at 10:09 AM Sean Busbey wrote: > > > As of last week the Apache Hadoop project now keeps a public list of CVEs > > that are public. (although it's

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-11-05 Thread Stack
On Mon, Nov 5, 2018 at 10:09 AM Sean Busbey wrote: > As of last week the Apache Hadoop project now keeps a public list of CVEs > that are public. (although it's currently limited to things from the last > year) > > https://hadoop.apache.org/cve_list.html > > Folks okay with linking to that page

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-11-05 Thread Sean Busbey
As of last week the Apache Hadoop project now keeps a public list of CVEs that are public. (although it's currently limited to things from the last year) https://hadoop.apache.org/cve_list.html Folks okay with linking to that page and updating Hadoop requirements for the next minor releases in

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-25 Thread Josh Elser
I hear what you're saying, but I'd also be pissed if we waste our own time dealing with Hadoop issues that have already been addressed. Just my feeling, but I think I'm distracting from the original question at this point. On 10/24/18 5:01 PM, Sean Busbey wrote: Yes, because they deployed

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-24 Thread Sean Busbey
Yes, because they deployed when 2.6.5 wasn't the latest and they don't want to deal with the headaches of Hadoop upgrades. If we do only one, it should be the oldest IMHO. But if we're just talking about changing thing in new minor releases of HBase this is moot. We make 2.7.7 the minimum

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-24 Thread Josh Elser
IMO -- for the 2.6 line, let's just use 2.6.latest (2.6.5). Hadoop seems to have moved beyond 2.6, it doesn't seem likely that we're creating a lot of value for our users. Would someone deploying a Hadoop 2.6 release seriously try a release other than the latest? On 10/22/18 9:32 PM, 张铎(Duo

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-24 Thread Sean Busbey
FYI HADOOP-15878 is tracking publishing impacted versions for the CVE On Tue, Oct 23, 2018 at 11:37 AM Sean Busbey wrote: > > I can get behind more aggressively updating our dependencies to avoid > CVEs. I don't think we should do this in maintenance releases though. > Maintenance releases are

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-24 Thread Sean Busbey
The current patch includes documentation changes that impact all versions, so we'll have to update for that. Happy to deal with those details on the JIRA though, if we have lazy consensus here on the overall approach. On Tue, Oct 23, 2018 at 3:11 PM Stack wrote: > > On Tue, Oct 23, 2018 at 9:44

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-23 Thread Stack
On Tue, Oct 23, 2018 at 9:44 AM Sean Busbey wrote: > I can get behind more aggressively updating our dependencies to avoid > CVEs. I don't think we should do this in maintenance releases though. > Maintenance releases are meant to be extremely low risk for downstream > users. Despite our efforts

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-23 Thread Zach York
Thanks for the context Duo! Yes I agree, if we know about a CVE for a dependency it is our duty to either upgrade the dependency or disclose the vulnerability (ReleaseNotes?) so that users can make the necessary decision. However, this obviously does not apply for any CVE that hasn't been brought

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-23 Thread Sean Busbey
I can get behind more aggressively updating our dependencies to avoid CVEs. I don't think we should do this in maintenance releases though. Maintenance releases are meant to be extremely low risk for downstream users. Despite our efforts to date upgrading a dependency is still disruptive,

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-22 Thread Duo Zhang
I believe if there is a CVE for any of the dependencies we should try to upgrade it, and IIRC there is an issue about finding these dependencies out automatically. We haven't done this before does not mean ignoring a CVE is the correct way, it is just because no one takes care of it... And the

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-22 Thread Andrew Purtell
We should react to all CVEs if we’re going to. Fine to start now. > On Oct 22, 2018, at 6:50 PM, Sean Busbey wrote: > > Has the Hadoop PMC put out a public notice on the impact of that CVE yet? > Specifically have they stated what versions are vulnerable? Are we flagging > all versions

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-22 Thread Sean Busbey
Has the Hadoop PMC put out a public notice on the impact of that CVE yet? Specifically have they stated what versions are vulnerable? Are we flagging all versions impacted by it as "HBase says keep away"? Is there some reason this particular CVE especially impacts users of HBase? I presume not

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-22 Thread Duo Zhang
See here: https://access.redhat.com/security/cve/cve-2018-8009 All 2.7.x releases before 2.7.7 have the problem. And for 2.6.x, the hadoop team seems to drop the support as there is no release about two years, so either we keep the original support versions, or we just drop the support for the

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-22 Thread Sean Busbey
Zach you fine with conversation about this on the JIRA or you wanna do on the list? On Mon, Oct 22, 2018, 19:51 Zach York wrote: > What is the main reason for the change? Build time speedup? > > Any reason for testing all of the 2.6.x line, but not the 2.7.x line? We > don't check at all for

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-22 Thread Zach York
What is the main reason for the change? Build time speedup? Any reason for testing all of the 2.6.x line, but not the 2.7.x line? We don't check at all for 2.8.x? Can we be more consistent with how we test compatibility? (Do we only care about the latest patch release in a line?) Sorry If I'm

Re: [DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-22 Thread Sean Busbey
Please leave me time to review before it is committed. On Mon, Oct 22, 2018, 13:58 Stack wrote: > Duo has a patch up on HBASE-20970 that changes the Hadoop versions we check > at build time. Any objections to committing to branch-2.1+? > > It makes following changes: > > 2.6.1 2.6.2 2.6.3 2.6.4

[DISCUSS] Changing hadoop check versions in our hbase-personality?

2018-10-22 Thread Stack
Duo has a patch up on HBASE-20970 that changes the Hadoop versions we check at build time. Any objections to committing to branch-2.1+? It makes following changes: 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 becomes 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.7 And... 3.0.0 goes to 3.0.3