Re: [DISCUSS] Tracing in the Hadoop ecosystem

2018-08-21 Thread Andrew Purtell
And, sorry if not clear, only source level compatibility is needed. The goal wouldn't be drop in binary compatibility for user convenience, it would be source compatibility so we don't have to replace all of the HTrace instrumentation points cross stack in the short term, so, stack developer

Re: [DISCUSS] Tracing in the Hadoop ecosystem

2018-08-21 Thread Andrew Purtell
I was assuming taking the HTrace API implementation, removing all code from the methods, and reimplementing with Brave wouldn't face insurmountable challenges, especially given the result is only meant for near term use, but I can't say I've tried nor looked into it in details. Was thinking

Re: [DISCUSS] Tracing in the Hadoop ecosystem

2018-08-21 Thread Stack
On Tue, Aug 21, 2018 at 10:09 AM Andrew Purtell wrote: > What if someone built a HTrace facade for Zipkin / Brave? I like the idea but taking a look, HTrace does static dispatch. I was thinking that precludes our being able to do a facade. I would love to hear otherwise. Thanks, S > Hadoop,

Re: [DISCUSS] Tracing in the Hadoop ecosystem

2018-08-21 Thread Andrew Purtell
What if someone built a HTrace facade for Zipkin / Brave? Hadoop, HBase, Phoenix, and other HTrace API users would still need to move away from embedding HTrace instrumentation points to whatever is the normal API of the accepted replacement, but such a facade would give you a drop in replacement

Re: [DISCUSS] Tracing in the Hadoop ecosystem

2018-08-21 Thread Stack
On Tue, Aug 21, 2018 at 3:44 AM Tsuyoshi Ozawa wrote: > Thanks for starting discussion, Stack. > > The ZipKin seems to be coming to the Apache Incubator. As Andrew > Purtell said on HADOOP-15566, it would be good option since there is > no problem about licenses. >

Re: [DISCUSS] Tracing in the Hadoop ecosystem

2018-08-21 Thread Tsuyoshi Ozawa
Thanks for starting discussion, Stack. The ZipKin seems to be coming to the Apache Incubator. As Andrew Purtell said on HADOOP-15566, it would be good option since there is no problem about licenses. https://wiki.apache.org/incubator/ZipkinProposal Stack, do you have any knowledge about

Re: [DISCUSS] Tracing in the Hadoop ecosystem

2018-07-30 Thread Eric Yang
Most of code coverage tools can instrument java classes without make any source code changes, but tracing distributed system is more involved because code execution via network interactions are not easy to match up. All interactions between sender and receiver have some form of session id or

Re: [DISCUSS] Tracing in the Hadoop ecosystem

2018-07-30 Thread Duo Zhang
Anyway, for HBase, we'd better align our trace library with Hadoop, especially HDFS. A full trace from the hbase client down to the datanode will be really helpful for debugging and monitoring. 2018-07-31 6:06 GMT+08:00 Stack : > There is a healthy discussion going on over in HADOOP-15566 on

[DISCUSS] Tracing in the Hadoop ecosystem

2018-07-30 Thread Stack
There is a healthy discussion going on over in HADOOP-15566 on tracing in the Hadoop ecosystem. It would sit better on a mailing list than in comments up on JIRA so here's an attempt at porting the chat here. Background/Context: Bits of Hadoop and HBase had Apache HTrace trace points added.