Re: [DISCUSS] tracing framework updates

2018-02-27 Thread Tony Kurc
Josh, It was exclusively the first - using the traces in the server-side code. The most common case is "I have a scan which is much slower than expected", and couldn't figure out why. I'm trying to think of alternative approaches to using the traces, and honestly, doing a bunch of log aggregation

Re: [DISCUSS] tracing framework updates

2018-02-27 Thread Josh Elser
Oh, that's a pleasant surprise to hear, actually. Anything you can share with the class, Tony? Would love to hear (even if brief) how it was used and benefited you. Specifically, I'm curious if... * You looked at traces from our server-side instrumented code * You instrumented your own code

Re: [DISCUSS] tracing framework updates

2018-02-27 Thread Christopher
I agree that it has value, and would also be disappointed ripping it out. However, unless the interest in keeping it is accompanied by interest in maintaining it, I don't think we have much choice. Eventually, we're going to have to make hard decisions about what things to keep, and what things to

Re: [DISCUSS] tracing framework updates

2018-02-27 Thread Tony Kurc
I'd personally be disappointed to see it removed. There is a bit of a learning curve and startup cost to use it now, but when diagnosing major challenges, it has been an invaluable capability. On Feb 27, 2018 3:15 PM, "Josh Elser" wrote: Wow... that's, erm, quite the paper.

Re: [DISCUSS] tracing framework updates

2018-02-27 Thread Christopher
I wonder if it would be better to just rip out tracing in 2.0. I know it's a big leap... but I really don't know anybody using it regularly, and given the potential compatibility problems, potential impact of migrating, and general lack of care from the HTrace project for API compatibility, as

RE: [DISCUSS] tracing framework updates

2018-02-27 Thread Ed Coleman
For general discussion - Facebook recently (Oct 28, 2017) published a paper on tracing: Canopy: An End-to-End Performance Tracing and Analysis System (https://research.fb.com/publications/canopy-end-to-end-performance-tracing-at-scale/) As a bonus, they referenced Accumulo and HTrace in section

Re: [DISCUSS] tracing framework updates

2018-02-27 Thread Tony Kurc
I have some experience with opentracing, and it definitely seems promising, however, potentially promising in the same way htrace was... That being said, I did a cursory thought exercise of what it would take to do a swap of the current tracing in accumulo to opentracing, and I didn't come across

Re: [DISCUSS] dropping hadoop 2 support

2018-02-27 Thread Keith Turner
+1 On Tue, Feb 27, 2018 at 10:42 AM, Sean Busbey wrote: > Let's get the discussion started early on when we'll drop hadoop 2 support. > > As of ACCUMULO-4826 we are poised to have Hadoop 2 and Hadoop 3 supported in > 1.y releases as of 1.9.0. That gives an upgrade path so

Re: [DISCUSS] tracing framework updates

2018-02-27 Thread Sean Busbey
On 2018/02/27 16:39:02, Christopher wrote: > I didn't realize HTrace was struggling in incubation. Maybe some of us can > start participating? The project did start within Accumulo, after all. What > does it need? I also wouldn't want to go back to maintaining cloudtrace.

[DISCUSS] tracing framework updates

2018-02-27 Thread Sean Busbey
More things we should get ahead of for Accumulo 2.0.0: distributed tracing. Right now we have an awkward situation wrt HTrace support. We're using and shipping htrace 3.1. It works okay for our internal uses, afaict. Hadoop 2.6 ships and uses HTrace 3.0. I believe this does not work with 3.1.

Re: [DISCUSS] dropping hadoop 2 support

2018-02-27 Thread Josh Elser
+1 AFAIK, this wouldn't have to be anything more than build changes. "Dropping hadoop2 support" wouldn't need to include any other changes (as adding H3 support didn't require any Java changes). Getting in front of the ball to help push people towards newer versions would be a welcome

Re: [DISCUSS] dropping hadoop 2 support

2018-02-27 Thread Christopher
+1 I'm in favor of this. On Tue, Feb 27, 2018 at 10:42 AM Sean Busbey wrote: > Let's get the discussion started early on when we'll drop hadoop 2 support. > > As of ACCUMULO-4826 we are poised to have Hadoop 2 and Hadoop 3 supported > in 1.y releases as of 1.9.0. That gives

[DISCUSS] dropping hadoop 2 support

2018-02-27 Thread Sean Busbey
Let's get the discussion started early on when we'll drop hadoop 2 support. As of ACCUMULO-4826 we are poised to have Hadoop 2 and Hadoop 3 supported in 1.y releases as of 1.9.0. That gives an upgrade path so that folks won't have to upgrade both Hadoop and Accumulo at the same time. How about

Re: [TEST][VOTE] Accumulo 1.7.4-rc0

2018-02-27 Thread Keith Turner
I ran mvn verify against fc75402, slightly newer than rc0 and saw the following failures. [INFO] Results: [INFO] [ERROR] Errors: [ERROR] RollWALPerformanceIT.testWalPerformance:126->testWalPerformanceOnce:116->getAverage:101->ingest:67 ยป TestTimedOut [ERROR]

Re[2]: [TEST][VOTE] Accumulo 1.7.4-rc0

2018-02-27 Thread J. Mark Owens
Mike, I had that failure as well when running mvn verify. But when I ran the IT test by itself it completed successfully. I had several IT tests fail (different ones each run) during the 'mvn verify' runs, but when run in isolation they succeeded. The exception to that was the