Re: [VOTE] HTrace 4.0 Release Candidate 0

[Colin P. McCabe]

On Mon, Sep 7, 2015 at 8:47 AM, Lewis John Mcgibbney
 wrote:
> Hi Colin,
> Nice work in getting the release candidate prepared and available for
> review.
> Where is the KEYS file?
> I tried the one here
>

Good call.  I'll append my key to the KEYS file.  My public key is
here: http://pgp.mit.edu/pks/lookup?op=get=0xDE78987A9CD4D9D3

> It does not contain your sig so I cannot verify the release sigs.
> Moving on,
>
> Notes Binaries Archives Standards Apache Generated Unknown 0 0 0 116 92 0 15
> Unapproved licenses are as follows
>
> bootstrap-theme.css
> bootstrap-theme.min.css
> bootstrap.css
> bootstrap.min.css
> backbone-1.1.2.js
> bootstrap.js
> bootstrap.min.js
> d3.min.js
> jquery-2.1.4.js
> moment-2.10.3.js
> npm.js
> underscore-1.7.0.js
> SpanProtos.java
> dependency-reduced-pom.xml
>
> None of the above libraries are declared within NOTICE.txt

bootstrap-theme.css, bootstrap-theme.min.css, bootstrap.css,
bootstrap.min.css, bootstrap.js, bootstrap.min.js, and
bootstrap-3.3.1/js/npm.js are described in LICENSE.txt as:

> Bootstrap, an html, css, and javascript framework, is
> Copyright (c) 2011-2015 Twitter, Inc and MIT licensed:
> https://github.com/twbs/bootstrap/blob/master/LICENSE

backbone-1.1.2.js is described in LICENSE.txt as:

> backbone, is a javascript library, that is Copyright (c) 2010-2014
> Jeremy Ashkenas, DocumentCloud. It is MIT licensed:
> https://github.com/jashkenas/backbone/blob/master/LICENSE

d3.min.js is described in LICENSE.txt as:

> D3, a javascript library for manipulating data, used by htrace-hbase
> is Copyright 2010-2014, Michael Bostock and BSD licensed:
> https://github.com/mbostock/d3/blob/master/LICENSE

jquery-2.1.4.js is described in LICENSE.txt as:

> jquery, a javascript library, is Copyright jQuery Foundation and other
> contributors, https://jquery.org/. The software consists of
> voluntary contributions made by many individuals. For exact
> contribution history, see the revision history
> available at https://github.com/jquery/jquery
> It is MIT licensed:
> https://github.com/jquery/jquery/blob/master/LICENSE.txt

moment-2.10.3.js is described in LICENSE.txt as:

> moment.js is a front end time conversion project.
> It is (c) 2011-2014 Tim Wood, Iskren Chernev, Moment.js contributors
> and shared under the MIT license:
> https://github.com/moment/moment/blob/develop/LICENSE

underscore-1.7.0.js is described in LICENSE.txt as:

> underscore, a javascript library of functional programming helpers, is
> (c) 2009-2014 Jeremy Ashkenas, DocumentCloud and Investigative Reporters
> & Editors and an MIT license:
> https://github.com/jashkenas/underscore/blob/master/LICENSE

According to http://www.apache.org/dev/licensing-howto.html:

> Bundling a dependency which is issued under one of the following
> licenses is straightforward, assuming that said license applies
> uniformly to all files within the dependency:
> 1. BSD (without advertising clause)
> 2. MIT/X11
> In LICENSE, add a pointer to the dependency's license within the source tree
> and a short note summarizing its licensing...
> Under normal circumstances, there is no need to modify NOTICE.

Since these are all BSD or MIT licensed, I would interpret this to
mean there is no need for us to modify NOTICE.  Does that make sense?

SpanProtos.java and dependency-reduced-pom.xml are generated files.
According to 
http://incubator.apache.org/guides/releasemanagement.html#notes-license-headers
:

> The issue of licenses on generated documentation is a little controversial.
> Copyright may not subsist in a document which is generated by an 
> transformation
> from an original. In which case, the license header may be unnecessary. 
> License
> headers should always be present in the original. Where it is reasonable to 
> do so,
> the templates should also add the license header to the generated documents.

I looked at how Apache Hadoop is handling this, and they do not have
license headers on their protobuf-generated files.  So I think this is
fine.  (From a technical point of view, I think the PB compiler also
provides no way to do this, as far as I know.)  The situation for
dependency-reduced-pom.xml is the same-- it is a file generated by
Maven.  This is similar to jar files, which are also generated by the
build process, and do not contain a license header.

Also, I believe all these library and generated  files were in the
last release we did (although they moved around a bit).

>
> I navigate to the decompressed RC directory and try mvn clean install... it
> enters into a loop!

As Masatake commented, this is HTRACE-236.  There is a note about it
in the README.md.  For a workaround, you can use the solution
described in HTRACE-234 to get the build to run on your version of
Maven (which I assume is 3.3)...

Thanks for trying out the release-- I will take a look at updating
KEYS tomorrow.  So far I haven't seen anything that would need a
respin (please let me know if I missed 

HTrace on Jira

[Lewis John Mcgibbney]

Hi Folks,
What is happening with versioning on Jira?
Master branch runs off of 4.0.0-incubating-SNAPSHOT, the only version which
exists within Jira is 4.1.
Most issues are not assigned to any release at all.
Can someone sort it out and we could potentially set a roadmap for
releasing HTrace 4.0/4.0.0-incubating?
Thanks folks
lewis

-- 
*Lewis*

Re: HTrace on Jira

[Colin P. McCabe]

Hi Lewis,

At this point, nothing else is going into 4.0 unless it's a blocker
for the release.  The master branch version is now at 4.1.  Pretty
much all open JIRAs should be targeted at 4.1.

You can see the JIRAs fixed in 4.0 here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315924=12333022

I'm not sure what you mean when you say that "most issues are not
assigned to any release at all."  Pretty much every issue has a "fix
version" which is the release it was fixed in, and an "affects
version", which is the first release it appeared in.  Do you see any
issues without those fields set?  If so, let's fill them in.

One thing that frustrates me is that there is no "Target version:"
field in our JIRA, like there is on Hadoop's JIRA.  I'm not sure what
we have to configure to get a field like that.

best,
Colin


On Tue, Sep 8, 2015 at 4:46 PM, Lewis John Mcgibbney
 wrote:
> Hi Folks,
> What is happening with versioning on Jira?
> Master branch runs off of 4.0.0-incubating-SNAPSHOT, the only version which
> exists within Jira is 4.1.
> Most issues are not assigned to any release at all.
> Can someone sort it out and we could potentially set a roadmap for
> releasing HTrace 4.0/4.0.0-incubating?
> Thanks folks
> lewis
>
> --
> *Lewis*

Re: [VOTE] HTrace 4.0 Release Candidate 0

[Masatake Iwasaki]

Thanks for putting this up, Colin.

- verified mds and signature
- ran "mvn install" without test failure
- ran "mvn package -Pnative" without test failure in htrace-c
- built src tarball by running "mvn clean install -DskipTests 
assembly:single -Pdist"
- launched htraced and sent test tracing spans by HTracedRESTReceiver 
and checked the spans by Web-UI


I'm +1 if the issue about NOTICE.txt Lewis pointed out is not critical 
for release.


Masatake Iwasaki

On 9/5/15 08:02, Colin P. McCabe wrote:

I've posted the first release candidate here:

http://people.apache.org/~cmccabe/htrace/releases/4.0.0/rc0

The jars have been staged here:

https://repository.apache.org/content/repositories/orgapachehtrace-1017

There's a lot of great stuff in this release, including a new web UI,
many bug fixes, API improvements and enlargement of span IDs to 128
bits to avoid conflicts.

The vote will run for 5 days.

cheers,
Colin

Release Notes - HTrace - Version 4.0

** Sub-task
 * [HTRACE-208] - Remove deprecated addKVAnnotation(byte[], byte[]) method
 * [HTRACE-209] - Make span ID 128 bit to avoid collisions
 * [HTRACE-210] - Remove TrueIfTracingSampler
 * [HTRACE-211] - Move htrace-core classes to the
org.apache.htrace.core namespace
 * [HTRACE-212] - Change version to 4.0
 * [HTRACE-214] - De-globalize Tracer.java
 * [HTRACE-215] - Simplify the Sampler type
 * [HTRACE-216] - SpanReceivers should not fill in ProcessId
 * [HTRACE-217] - Rename ProcessId to TracerId
 * [HTRACE-222] - Add SpanReceiverPool
 * [HTRACE-228] - Fix subprojects to refer to new
org.apache.htrace.core namespace
 * [HTRACE-229] - htrace-webapp needs to be updated to refer to
"tracerid" not "processid"
** Bug
 * [HTRACE-159] - libhtrace.so: use HRPC endpoint of htraced
 * [HTRACE-164] - htrace hrpc: use msgpack for serialization
 * [HTRACE-166] - Add tabbed view
 * [HTRACE-167] - Update go build instructions in BUILDING.txt
 * [HTRACE-171] - htraced godeps should use
github.com/ugorji/go/codec rather than github.com/ugorji/go
 * [HTRACE-174] - Refactor GUI
 * [HTRACE-177] - htrace-zipkin: shade all dependencies
 * [HTRACE-182] - htraced: add rpm build via -Prpm
 * [HTRACE-189] - gui: fix error handling in a few places
 * [HTRACE-190] - htraced: allow querying by process ID
 * [HTRACE-191] - gui: add "duration" to span details, filter out "selected"
 * [HTRACE-192] - gui: when expanding parents or children, sort the
spans by begin time
 * [HTRACE-193] - gui: avoid doing multiple redraws when
spanResults is updated
 * [HTRACE-196] - gui: add scrolling for spans view
 * [HTRACE-201] - htrace-web: URL-Encode query JSON
 * [HTRACE-202] - htrace-web: fix "converting circular object to
JSON" error when pressing "clear" button
 * [HTRACE-218] - Fix issues with finding json-c includes and librt
in the native library
 * [HTRACE-219] - Add -Dleveldb.prefix and -Djsonc.prefix build options
 * [HTRACE-220] - htraced: should be able to set log.path to the
empty string via "-Dlog.path=" on the command line
 * [HTRACE-223] - gobuild.sh: fix issue where maven succeeds if go
build fails
 * [HTRACE-224] - htrace C client: htrace_conf_get_u64,
htrace_conf_get_double can't handle spaces at the end of strings
 * [HTRACE-230] - Make TracerBuilder like all other Builders; an
internal rather than adjacent class
 * [HTRACE-233] - htrace-zipkin should explicitly include slf4j-api
to avoid ClassNotFoundException
 * [HTRACE-234] - Add workaround to prevent htrace-hbase from
getting in an infinite loop while creating the dependency-reduced pom
** Improvement
 * [HTRACE-29] - add javascript web UI for htraced
 * [HTRACE-160] - htraced: support continuing a query from where
the client left it off by sending a previous span
 * [HTRACE-162] - htraced hrpc: some logging improvements
 * [HTRACE-170] - Optimize use of Random in htrace-core by using
ThreadLocalRandom
 * [HTRACE-172] - Move minJdk to 1.7 (JDK 7)
 * [HTRACE-175] - Add Trace#addKVAnnotation convenience method
 * [HTRACE-176] - Expose ZipkinSpanReceiver configuration keys externally
 * [HTRACE-180] - Move the GUI to a top-level subproject
 * [HTRACE-184] - Expose PROCESS_ID_KEY configuration key
 * [HTRACE-186] - gui: support finding the parents and children of
spans, add owl
 * [HTRACE-194] - gui: support multiple selections, zooming to fit
a group of spans, deleting a group of spans
 * [HTRACE-197] - htraced build: set RUNPATH if possible
 * [HTRACE-199] - gui: Double clicking on spans should bring up span details
 * [HTRACE-203] - htrace-web: pressing enter should dismiss the
modal dialog box
 * [HTRACE-204] - htrace-web: add draggable bar which allows more
or less visual space for process name in search view
 * [HTRACE-205] - htrace-web: Width of SearchResultsView should be
uppdated along with resizing of 

Re: HTrace on Jira

[Colin P. McCabe]

Good idea... I created some components:

build: The Maven build system, pom.xml, Makefile, CMake, Docker, etc.
flume: The htrace-flume span receiver
zipkin: The htrace-zipkin span receiver
hbase: The htrace-hbase span receiver
core: The htrace-core subproject which contains the main htrace
library used by applications to initiate tracing
htraced: The htraced daemon and span receiver
ui: htrace-web graphical user interface

I think it probably makes sense to combine docs with the website since
we generate our website from the docs via Maven.  Feel free to add
more stuff, of course.

cheers,
Colin

On Tue, Sep 8, 2015 at 6:40 PM, Lewis John Mcgibbney
 wrote:
> Hi Folks,
> Further to this, is it possible for someone to go through the components
> and create components for each module as well as one for website?
> Thanks
> Lewis
>
> On Tue, Sep 8, 2015 at 4:46 PM, Lewis John Mcgibbney <
> lewis.mcgibb...@gmail.com> wrote:
>
>> Hi Folks,
>> What is happening with versioning on Jira?
>> Master branch runs off of 4.0.0-incubating-SNAPSHOT, the only version
>> which exists within Jira is 4.1.
>> Most issues are not assigned to any release at all.
>> Can someone sort it out and we could potentially set a roadmap for
>> releasing HTrace 4.0/4.0.0-incubating?
>> Thanks folks
>> lewis
>>
>> --
>> *Lewis*
>>
>
>
>
> --
> *Lewis*

Re: [VOTE] HTrace 4.0 Release Candidate 0

[Colin P. McCabe]

Thanks for looking at this, Masatake and Lewis!  Lewis, is my
explanation of NOTICE.txt correct, or did I miss something?

Remember to vote... the vote will be open until tomorrow.

Here is my +1. ;)

cheers,
Colin

On Tue, Sep 8, 2015 at 6:32 PM, Masatake Iwasaki
 wrote:
> Thanks for putting this up, Colin.
>
> - verified mds and signature
> - ran "mvn install" without test failure
> - ran "mvn package -Pnative" without test failure in htrace-c
> - built src tarball by running "mvn clean install -DskipTests
> assembly:single -Pdist"
> - launched htraced and sent test tracing spans by HTracedRESTReceiver and
> checked the spans by Web-UI
>
> I'm +1 if the issue about NOTICE.txt Lewis pointed out is not critical for
> release.
>
> Masatake Iwasaki
>
>
> On 9/5/15 08:02, Colin P. McCabe wrote:
>>
>> I've posted the first release candidate here:
>>
>> http://people.apache.org/~cmccabe/htrace/releases/4.0.0/rc0
>>
>> The jars have been staged here:
>>
>> https://repository.apache.org/content/repositories/orgapachehtrace-1017
>>
>> There's a lot of great stuff in this release, including a new web UI,
>> many bug fixes, API improvements and enlargement of span IDs to 128
>> bits to avoid conflicts.
>>
>> The vote will run for 5 days.
>>
>> cheers,
>> Colin
>>
>> Release Notes - HTrace - Version 4.0
>>
>> ** Sub-task
>>  * [HTRACE-208] - Remove deprecated addKVAnnotation(byte[], byte[])
>> method
>>  * [HTRACE-209] - Make span ID 128 bit to avoid collisions
>>  * [HTRACE-210] - Remove TrueIfTracingSampler
>>  * [HTRACE-211] - Move htrace-core classes to the
>> org.apache.htrace.core namespace
>>  * [HTRACE-212] - Change version to 4.0
>>  * [HTRACE-214] - De-globalize Tracer.java
>>  * [HTRACE-215] - Simplify the Sampler type
>>  * [HTRACE-216] - SpanReceivers should not fill in ProcessId
>>  * [HTRACE-217] - Rename ProcessId to TracerId
>>  * [HTRACE-222] - Add SpanReceiverPool
>>  * [HTRACE-228] - Fix subprojects to refer to new
>> org.apache.htrace.core namespace
>>  * [HTRACE-229] - htrace-webapp needs to be updated to refer to
>> "tracerid" not "processid"
>> ** Bug
>>  * [HTRACE-159] - libhtrace.so: use HRPC endpoint of htraced
>>  * [HTRACE-164] - htrace hrpc: use msgpack for serialization
>>  * [HTRACE-166] - Add tabbed view
>>  * [HTRACE-167] - Update go build instructions in BUILDING.txt
>>  * [HTRACE-171] - htraced godeps should use
>> github.com/ugorji/go/codec rather than github.com/ugorji/go
>>  * [HTRACE-174] - Refactor GUI
>>  * [HTRACE-177] - htrace-zipkin: shade all dependencies
>>  * [HTRACE-182] - htraced: add rpm build via -Prpm
>>  * [HTRACE-189] - gui: fix error handling in a few places
>>  * [HTRACE-190] - htraced: allow querying by process ID
>>  * [HTRACE-191] - gui: add "duration" to span details, filter out
>> "selected"
>>  * [HTRACE-192] - gui: when expanding parents or children, sort the
>> spans by begin time
>>  * [HTRACE-193] - gui: avoid doing multiple redraws when
>> spanResults is updated
>>  * [HTRACE-196] - gui: add scrolling for spans view
>>  * [HTRACE-201] - htrace-web: URL-Encode query JSON
>>  * [HTRACE-202] - htrace-web: fix "converting circular object to
>> JSON" error when pressing "clear" button
>>  * [HTRACE-218] - Fix issues with finding json-c includes and librt
>> in the native library
>>  * [HTRACE-219] - Add -Dleveldb.prefix and -Djsonc.prefix build
>> options
>>  * [HTRACE-220] - htraced: should be able to set log.path to the
>> empty string via "-Dlog.path=" on the command line
>>  * [HTRACE-223] - gobuild.sh: fix issue where maven succeeds if go
>> build fails
>>  * [HTRACE-224] - htrace C client: htrace_conf_get_u64,
>> htrace_conf_get_double can't handle spaces at the end of strings
>>  * [HTRACE-230] - Make TracerBuilder like all other Builders; an
>> internal rather than adjacent class
>>  * [HTRACE-233] - htrace-zipkin should explicitly include slf4j-api
>> to avoid ClassNotFoundException
>>  * [HTRACE-234] - Add workaround to prevent htrace-hbase from
>> getting in an infinite loop while creating the dependency-reduced pom
>> ** Improvement
>>  * [HTRACE-29] - add javascript web UI for htraced
>>  * [HTRACE-160] - htraced: support continuing a query from where
>> the client left it off by sending a previous span
>>  * [HTRACE-162] - htraced hrpc: some logging improvements
>>  * [HTRACE-170] - Optimize use of Random in htrace-core by using
>> ThreadLocalRandom
>>  * [HTRACE-172] - Move minJdk to 1.7 (JDK 7)
>>  * [HTRACE-175] - Add Trace#addKVAnnotation convenience method
>>  * [HTRACE-176] - Expose ZipkinSpanReceiver configuration keys
>> externally
>>  * [HTRACE-180] - Move the GUI to a top-level subproject
>>  * [HTRACE-184] - Expose PROCESS_ID_KEY configuration key
>>  * [HTRACE-186] - gui: support finding the parents and children of
>> 

Regarding Htrace webUI : Tool for visualizing tracing info with Phoenix Tracing Web App

[Dhruv Gohil]

Hi,
Just want to check with anybody who's working with Htrace Web 
UI(not able to find any docs about it on apache site, just found in JIRA 
), How's much of phoenix's trace UI overlapping with what we are doing? 
Can we reuse their code, or the other way around?


--
Dhruv

 Forwarded Message 
Subject: 	Re: Tool for visualizing tracing info with Phoenix Tracing Web 
App [GSoC]

Date:   Mon, 24 Aug 2015 13:23:16 +0530
From:   Ayola Jayamaha 
Reply-To:   d...@phoenix.apache.org, raphaelan...@gmail.com
To: 	d...@phoenix.apache.org , Nick Dimiduk 
, James , Mujtaba Chohan 

CC: 	Jesse Yates , michael stack , 
Rajeshbabu Chintaguntla 




Hi All,

Future work: -

   - Search trace list by SQL Query
   - Turn Tracing on, run query, turn tracing off and visualize the trace
   dataset

Welcome any feedback.

Thanks.

On Fri, Aug 21, 2015 at 12:18 PM, Ayola Jayamaha 
wrote:


Hi All,

You can find all the tasks and milestones on GSOC 2015 - Phoenix in the 2
PRs[1,2]. It contains a launch script to run the jetty server. It creates a
web application running on user preferred port. The back-end services
contains querying the SYSTEM.TRACING_STATS table and projecting the JSON
output of it.

You can try them out as described in the blog post[3].

The respective jiras are given[4,5]. The phoenix tracing page
https://phoenix.apache.org/tracing.html will  be updated and the patch is
attached with the screenshots. [6]

All the blog posts I have written regard to GSoC 2015 can be found here[7].

Thanks.

On Fri, Aug 21, 2015 at 12:11 AM, Ayola Jayamaha 
wrote:


Hi All,

The Phoenix Tracing Web App main features and setup information can be
found on the following blog posts[1,2].

[1]
http://ayolajayamaha.blogspot.com/2015/08/apache-phoenix-web-application.html

[2]
http://ayolajayamaha.blogspot.com/2015/08/apache-phoenix-web-application-part-02.html

--
Best Regards,
Nishani Jayamaha
http://ayolajayamaha.blogspot.com/




[1] https://github.com/apache/phoenix/pull/111
[2] https://github.com/apache/phoenix/pull/112
[3]
http://ayolajayamaha.blogspot.com/2015/08/apache-phoenix-web-application.html
[4] https://issues.apache.org/jira/browse/PHOENIX-2186
[5] https://issues.apache.org/jira/browse/PHOENIX-2187
[6] https://issues.apache.org/jira/browse/PHOENIX-2190
[7] http://ayolajayamaha.blogspot.com/search?q=phoenix

--
Best Regards,
Nishani Jayamaha
http://ayolajayamaha.blogspot.com/






--
Best Regards,
Nishani Jayamaha
http://ayolajayamaha.blogspot.com/

I think independent war was already available but for phoenix merged 
into server itself, as phoenix itself is Java App.


Embed Tracing Web Application as a service
---

Key: PHOENIX-2212
URL:https://issues.apache.org/jira/browse/PHOENIX-2212


Embed Tracing Web Application as a service
---

Key: PHOENIX-2212
URL:https://issues.apache.org/jira/browse/PHOENIX-2212