Re: [VOTE] htrace-3.1.0, the this is it for sure!, tenth release candidate

[Masatake Iwasaki]

+1

verified signature
checked sha1 checksum
ran mvn clean install and passed all tests
checked that no .orig files in tarball

Masatake Iwasaki

On 1/16/15 04:41, Stack wrote:

I've posted the tenth htrace-3.1.0 release candidate here:

   http://people.apache.org/~stack/htrace-3.1.0-incubatingRC9/

(Our RCs are zero based so RC9 is the tenth RC)

Maven artifacts are here:

   https://repository.apache.org/content/repositories/orgapachehtrace-1014

The only changes from RC8 are changes to address hawkeye Billies comments.

The release is a src-only tarball for now. Later we can add binary bundles
to our release after we have better sense of what we as a project would
like to deliver.

This release is mainly a change of packaging from org.htrace to
org.apache.htrace to suit our new home here in Apache Incubator but it does
also includes 44 resolved issues [1] including the beginnings of an htrace
daemon whose intent is to make it so there is a low barrier collecting
cluster traces as well as a new flume receiver.

Beware that Apache Incubator, org.apache.htrace is not compatible with its
former self, org.htrace: the package name has changed but so has the JSON
serialization format. See README.md for a few notes on how to migrate.

Shall we make this release candidate our first incubator release?

Lets keep the vote period short (We'll have to run another vote over in
incubator general after this one if I understand the process properly).

Please vote +1/0/-1 by Friday, January 15th, 2015.

Thanks,
St.Ack

1.
https://issues.apache.org/jira/issues/?jql=project%20%3D%20HTRACE%20AND%20status%20%3D%20resolved%20AND%20fixVersion%20%3D%203.1.0%20ORDER%20BY%20issuetype%20DESC

Re: Trace HBase/HDFS with HTrace

[Masatake Iwasaki]

Hi,

Thanks for trying this. I am sorry for late reply.

I tried this today
by hbase-1.0.1-SANPSHOT built with {{-Dhadoop-two.version=2.7.0-SNAPSHOT}}
in pseudo distributed cluster
but failed to get end-to-end trace.

I checked that
* tracing works for both of hbase and hdfs,
* hbase runs with 2.7.0-SNAPSHOT jar of hadoop.

When I did do put with tracing on,
I saw span named FSHLog.sync with annotations such as
syncing writer and writer synced.
The code for tracing in FSHLog worked at least.

I'm still looking into this.
If it turned out that tracing spans are not reached to
actual HDFS writer thread in HBase, I will file a JIRA.

# We need hadoop-2.6.0 or higher in order to trace HDFS.
# Building hbase from source with {{-Dhadoop-two.version=2.6.0}}
# is straight forward way to do this
# because the binary release of hbase-1.0.0 bundles hadoop-2.5.1 jars.

Masatake

On 2/11/15 08:56, Nick Dimiduk wrote:

Hi Joshua,

In theory there's nothing special for you to do. Just issue your query to
HBase with tracing enabled. The active span will go through HBase, down
into HDFS, and back again. You'll need both systems collecting spans into
the same place so that you can report on the complete trace tree.

I've not recently tested the end-to-end, but I believe it's all there. If
not, it's a bug -- this is an intended use case. Can you give it a try
and let us know how it goes?

FYI, 0.99.x are preview releases of HBase and not for production use. Just
so you know :)

-n

On Wednesday, February 11, 2015, Chunxu Tang chunxut...@gmail.com wrote:


Hi all,

Now I’m exploiting HTrace to trace request level data flows in HBase and
HDFS. I have successfully traced HBase and HDFS by using HTrace,
respectively.

After that, I combine HBase and HDFS together and I want to just send a
PUT/GET request to HBase, but to trace the whole data flow in both HBase
and HDFS. In my opinion, when I send a request such as Get to HBase, it
will at last try to read the blocks on HDFS, so I can construct a whole
data flow tracing through HBase and HDFS. While, the fact is that I can
only get tracing data of HBase, with no data of HDFS.

Could you give me any suggestions on how to trace the data flow in both
HBase and HDFS? Does anyone have similar experience? Do I need to modify
the source code? And maybe which part(s) should I touch? If I need to
modify the code, I will try to create a patch for that.

Thank you.

My Configurations:
Hadoop version: 2.6.0
HBase version: 0.99.2
HTrace version: htrace-master
OS: Ubuntu 12.04


Joshua

Re: Trace HBase/HDFS with HTrace

[Masatake Iwasaki]

 perhaps a double detach.

It turned out to be double detach of NullScope singleton and I filed 
HTRACE-119.
I got the error on meta assignments on regionserver startup without 
starting trace spans..


Thanks,
Masatake

On 2/26/15 09:58, Colin P. McCabe wrote:

Hmm.  Looking at that error, my guess would be that there is an
incorrect usage of TraceScope#detach going on somewhere in hbase...
perhaps a double detach.  But I could be wrong.  We added some code
recently to catch issues like this.

best,
Colin

On Wed, Feb 25, 2015 at 12:28 AM, Masatake Iwasaki
iwasak...@oss.nttdata.co.jp wrote:

I tried hbase-1 built against today's htrace-3.2.0-SNAPSHOT (with quick fix
to TestHTraceHooks).
I got the error below in regionserver log.
I will dig this tomorrow.::

   2015-02-25 00:18:29,270 ERROR [RS_OPEN_META-centos7:16201-0]
htrace.Tracer: Tried to detach trace span null but it has already been
detached.
   2015-02-25 00:18:29,271 ERROR [RS_OPEN_META-centos7:16201-0]
handler.OpenRegionHandler: Failed open of region=hbase:meta,,1.1588230740,
starting to roll back the global memstore size.
   java.lang.RuntimeException: Tried to detach trace span null but it has
already been detached.
   at org.apache.htrace.Tracer.clientError(Tracer.java:61)
   at org.apache.htrace.TraceScope.detach(TraceScope.java:57)
   at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.sync(FSHLog.java:1559)
   at
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:94)
   at
org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:910)
   at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4911)
   at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4874)
   at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4845)
   at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4801)
   at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4752)
   at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:356)
   at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:126)
   at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
   at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)



On 2/24/15 18:27, Colin P. McCabe wrote:

Thanks for trying this, Mastake.  I've got HDFS working on my cluster
with tracing and LocalFileSpanReceiver.  Did you try using HBase +
HDFS with LocalFileSpanReceiver?  Be sure to use a build including
HTRACE-112 since LFSR was kind of busted prior to that.

I'm going to do a longer writeup about getting HDFS + HBase working
with other span receivers just as soon as I finish stomping a few more
bugs.

best,
Colin

On Tue, Feb 24, 2015 at 12:04 PM, Masatake Iwasaki
iwasak...@oss.nttdata.co.jp wrote:

Hi,

Thanks for trying this. I am sorry for late reply.

I tried this today
by hbase-1.0.1-SANPSHOT built with
{{-Dhadoop-two.version=2.7.0-SNAPSHOT}}
in pseudo distributed cluster
but failed to get end-to-end trace.

I checked that
* tracing works for both of hbase and hdfs,
* hbase runs with 2.7.0-SNAPSHOT jar of hadoop.

When I did do put with tracing on,
I saw span named FSHLog.sync with annotations such as
syncing writer and writer synced.
The code for tracing in FSHLog worked at least.

I'm still looking into this.
If it turned out that tracing spans are not reached to
actual HDFS writer thread in HBase, I will file a JIRA.

# We need hadoop-2.6.0 or higher in order to trace HDFS.
# Building hbase from source with {{-Dhadoop-two.version=2.6.0}}
# is straight forward way to do this
# because the binary release of hbase-1.0.0 bundles hadoop-2.5.1 jars.

Masatake


On 2/11/15 08:56, Nick Dimiduk wrote:

Hi Joshua,

In theory there's nothing special for you to do. Just issue your query
to
HBase with tracing enabled. The active span will go through HBase, down
into HDFS, and back again. You'll need both systems collecting spans
into
the same place so that you can report on the complete trace tree.

I've not recently tested the end-to-end, but I believe it's all there.
If
not, it's a bug -- this is an intended use case. Can you give it a try
and let us know how it goes?

FYI, 0.99.x are preview releases of HBase and not for production use.
Just
so you know :)

-n

On Wednesday, February 11, 2015, Chunxu Tang chunxut...@gmail.com
wrote:


Hi all,

Now I’m exploiting HTrace to trace request level data flows in HBase
and
HDFS. I have successfully traced HBase and HDFS by using HTrace,
respectively.

After that, I combine HBase and HDFS together and I want to just send a
PUT/GET request

Re: Trace HBase/HDFS with HTrace

[Masatake Iwasaki]

I tried hbase-1 built against today's htrace-3.2.0-SNAPSHOT (with quick 
fix to TestHTraceHooks).

I got the error below in regionserver log.
I will dig this tomorrow.::

  2015-02-25 00:18:29,270 ERROR [RS_OPEN_META-centos7:16201-0] 
htrace.Tracer: Tried to detach trace span null but it has already been 
detached.
  2015-02-25 00:18:29,271 ERROR [RS_OPEN_META-centos7:16201-0] 
handler.OpenRegionHandler: Failed open of 
region=hbase:meta,,1.1588230740, starting to roll back the global 
memstore size.
  java.lang.RuntimeException: Tried to detach trace span null but it 
has already been detached.

  at org.apache.htrace.Tracer.clientError(Tracer.java:61)
  at org.apache.htrace.TraceScope.detach(TraceScope.java:57)
  at 
org.apache.hadoop.hbase.regionserver.wal.FSHLog.sync(FSHLog.java:1559)
  at 
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:94)
  at 
org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:910)
  at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4911)
  at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4874)
  at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4845)
  at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4801)
  at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4752)
  at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:356)
  at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:126)
  at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  at java.lang.Thread.run(Thread.java:745)


On 2/24/15 18:27, Colin P. McCabe wrote:

Thanks for trying this, Mastake.  I've got HDFS working on my cluster
with tracing and LocalFileSpanReceiver.  Did you try using HBase +
HDFS with LocalFileSpanReceiver?  Be sure to use a build including
HTRACE-112 since LFSR was kind of busted prior to that.

I'm going to do a longer writeup about getting HDFS + HBase working
with other span receivers just as soon as I finish stomping a few more
bugs.

best,
Colin

On Tue, Feb 24, 2015 at 12:04 PM, Masatake Iwasaki
iwasak...@oss.nttdata.co.jp wrote:

Hi,

Thanks for trying this. I am sorry for late reply.

I tried this today
by hbase-1.0.1-SANPSHOT built with {{-Dhadoop-two.version=2.7.0-SNAPSHOT}}
in pseudo distributed cluster
but failed to get end-to-end trace.

I checked that
* tracing works for both of hbase and hdfs,
* hbase runs with 2.7.0-SNAPSHOT jar of hadoop.

When I did do put with tracing on,
I saw span named FSHLog.sync with annotations such as
syncing writer and writer synced.
The code for tracing in FSHLog worked at least.

I'm still looking into this.
If it turned out that tracing spans are not reached to
actual HDFS writer thread in HBase, I will file a JIRA.

# We need hadoop-2.6.0 or higher in order to trace HDFS.
# Building hbase from source with {{-Dhadoop-two.version=2.6.0}}
# is straight forward way to do this
# because the binary release of hbase-1.0.0 bundles hadoop-2.5.1 jars.

Masatake


On 2/11/15 08:56, Nick Dimiduk wrote:

Hi Joshua,

In theory there's nothing special for you to do. Just issue your query to
HBase with tracing enabled. The active span will go through HBase, down
into HDFS, and back again. You'll need both systems collecting spans into
the same place so that you can report on the complete trace tree.

I've not recently tested the end-to-end, but I believe it's all there. If
not, it's a bug -- this is an intended use case. Can you give it a try
and let us know how it goes?

FYI, 0.99.x are preview releases of HBase and not for production use. Just
so you know :)

-n

On Wednesday, February 11, 2015, Chunxu Tang chunxut...@gmail.com wrote:


Hi all,

Now I’m exploiting HTrace to trace request level data flows in HBase and
HDFS. I have successfully traced HBase and HDFS by using HTrace,
respectively.

After that, I combine HBase and HDFS together and I want to just send a
PUT/GET request to HBase, but to trace the whole data flow in both HBase
and HDFS. In my opinion, when I send a request such as Get to HBase, it
will at last try to read the blocks on HDFS, so I can construct a whole
data flow tracing through HBase and HDFS. While, the fact is that I can
only get tracing data of HBase, with no data of HDFS.

Could you give me any suggestions on how to trace the data flow in both
HBase and HDFS? Does anyone have similar experience? Do I need to modify
the source code? And maybe which part(s) should I touch? If I need to
modify the code, I will try to create a patch

Re: [VOTE] HTrace 3.2.0 - Release Candidate 0

[Masatake Iwasaki]

+1

* Verified md5 and sha1 sums
* Ran mvn package -Pnative successfully
* Did mvn install of HTrace 3.2.0 and built hadoop-trunk against it
* Verified that HTracedRESTReceiver worked with hadoop

Thanks,
Masatake Iwasaki


On 4/29/15 06:46, Abraham Elmahrek wrote:

I've posted our first release candidate here:

   http://people.apache.org/~abe/htrace/releases/3.2.0/

The jars have been staged here:

   https://repository.apache.org/content/repositories/orgapachehtrace-1015

This release improves the Java client with better error checking and
setting parents in a span after the span is created. Also, some issues in
the local file span receiver are fixed and work has started on a new C/C++
native client.

Please vote +1/0/-1 by Friday, May 1st, 2015.

Thanks,
Abe

1.
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%203.2.0%20AND%20project%20%3D%20HTRACE%20AND%20status%20%3D%20Resolved%20ORDER%20BY%20priority%20DESC


Release Notes - HTrace - Version 3.2.0

** Sub-task
 * [HTRACE-44] - Add htraced web UI search page skeleton
 * [HTRACE-45] - Add Span details page skeleton
 * [HTRACE-75] - UI should query htraced
 * [HTRACE-76] - Search page: Add +/- filters to search control
 * [HTRACE-77] - htraced gui: add pagination to the search page.
 * [HTRACE-78] - Search page: Enhance begin time and end time selection
widget
 * [HTRACE-79] - Search page: Make list more tabular with sorting on
front end
 * [HTRACE-108] - Search Page: Use server side pagination
 * [HTRACE-121] - Details page: Graph parents and children
 * [HTRACE-131] - Port spans.js in htrace-hbase to htraced
 * [HTRACE-134] - Port front end to marionette JS
 * [HTRACE-142] - Details page: Update graph
 * [HTRACE-146] - Search page: Remove client-side sorting and improve
pagination

** Bug
 * [HTRACE-32] - Change span timeline annotations map to be a
mapstring, string
 * [HTRACE-37] - htraced: serve index.html if the root URL was given
 * [HTRACE-47] - Add Content-Type header in REST server
 * [HTRACE-48] - use -L 1 in format.sh to make it work on macosx
 * [HTRACE-52] - FIgure out content-type handling in JSON API
 * [HTRACE-55] - Add Clean command to htrace go
 * [HTRACE-57] - Fix LocalFileSpanReceiver to avoid adding irrelevant
wrapper object
 * [HTRACE-68] - Clean up LocalFileSpanReceiver and SpanReceiverBuilder
a bit
 * [HTRACE-71] - format.sh: only format src/ directory
 * [HTRACE-90] - Remove Guava and shade other depdendencies in HTrace
subprojects
 * [HTRACE-93] - Add null check to SamplerBuilder
 * [HTRACE-97] - Support both -D and --D when specifying conf vars on
the command-line
 * [HTRACE-98] - Web Server should use PrefixHandler
 * [HTRACE-99] - log.go fails to create new log files
 * [HTRACE-100] - htraced: put query parameters in the URL, not in the
GET request body
 * [HTRACE-101] - Add better error-checking to Java HTrace client
 * [HTRACE-103] - Reset unclosed spans after TestBadClient
 * [HTRACE-107] - Support for Greater Than operator in htraced
 * [HTRACE-109] - fix TestHTracedRESTReceiver unit test failures
 * [HTRACE-110] - Fix rat check failure due to
dependency-reduced-pom.xml in htrace-hbase
 * [HTRACE-111] - HTrace Java client API fixes for 3.2
 * [HTRACE-112] - Fix LocalFileSpanReceiver to avoid BG thread and
problems around close()
 * [HTRACE-115] - The htraced datastore should use uint64 for span ids
rather than int64
 * [HTRACE-116] - htraced's data.store.clear option does not work
 * [HTRACE-117] - graph.go: sort children array to get deterministic
output
 * [HTRACE-118] - Java API: support setting the parents of a span after
the span is created
 * [HTRACE-119] - detach of NullScope singleton should not fail
 * [HTRACE-123] - fix graphviz functionality in the htrace command
 * [HTRACE-126] - BUILDING.txt should not say that godep is required,
because we auto-install godep
 * [HTRACE-127] - fix ./build/htrace dumpAll
 * [HTRACE-128] - RAT check hangs on files under
htrace-core/src/web/lib/rome-2.1.0
 * [HTRACE-132] - Trace#startSpan(String, TraceInfo) must not create
spans with spanId == 0
 * [HTRACE-133] - HTracedRESTReceiver drops spans when close() is called
 * [HTRACE-140] - Web UI of htraced causes high CPU usage if search
result is empty
 * [HTRACE-141] - Fix nits of swimlane view of htraced Web UI
 * [HTRACE-148] - Add deserialization utility method to MilliSpan for
testing outside of htrace-core
 * [HTRACE-151] - htrace-core, htrace-htraced: Use shaded import paths
 * [HTRACE-152] - Fix TestHTracedRESTReceiver
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException

** Improvement
 * [HTRACE-30] - Add writeSpan REST verb to htraced
 * [HTRACE-33] - Add getUniqueLocalTraceFileName to LocalFileSpanReceiver
 * [HTRACE-50] - Clean up build warnings
 * [HTRACE-53] - Move client

Re: [VOTE] HTrace 3.2.0 - Release Candidate 2

[Masatake Iwasaki]

+1

* Verified md5 and sha1 sums
* Ran mvn package -Pnative successfully
* Did mvn install of HTrace 3.2.0 and built hadoop-trunk against it
* Verified that HTracedRESTReceiver worked with hadoop

Thanks,
Masatake Iwasaki

On 5/1/15 13:16, Abraham Elmahrek wrote:

I've the second release candidate here:

   *http://people.apache.org/~abe/htrace/releases/3.2.0/rc1/
http://people.apache.org/~abe/htrace/releases/3.2.0/rc1/*

The jars have been staged here:

   *https://repository.apache.org/content/repositories/orgapachehtrace-1016
https://repository.apache.org/content/repositories/orgapachehtrace-1016*

This release improves the Java client with better error checking and
setting parents in a span after the span is created. Also, some issues in
the local file span receiver are fixed and work has started on a new C/C++
native client.

Please vote +1/0/-1 by Friday, May 7th, 2015.

Thanks,
Abe

1.
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%203.2.0%20AND%20project%20%3D%20HTRACE%20AND%20status%20%3D%20Resolved%20ORDER%20BY%20priority%20DESC


Release Notes - HTrace - Version 3.2.0

** Sub-task
 * [HTRACE-44] - Add htraced web UI search page skeleton
 * [HTRACE-45] - Add Span details page skeleton
 * [HTRACE-75] - UI should query htraced
 * [HTRACE-76] - Search page: Add +/- filters to search control
 * [HTRACE-77] - htraced gui: add pagination to the search page.
 * [HTRACE-78] - Search page: Enhance begin time and end time selection
widget
 * [HTRACE-79] - Search page: Make list more tabular with sorting on
front end
 * [HTRACE-108] - Search Page: Use server side pagination
 * [HTRACE-121] - Details page: Graph parents and children
 * [HTRACE-131] - Port spans.js in htrace-hbase to htraced
 * [HTRACE-134] - Port front end to marionette JS
 * [HTRACE-142] - Details page: Update graph
 * [HTRACE-146] - Search page: Remove client-side sorting and improve
pagination

** Bug
 * [HTRACE-32] - Change span timeline annotations map to be a
mapstring, string
 * [HTRACE-37] - htraced: serve index.html if the root URL was given
 * [HTRACE-47] - Add Content-Type header in REST server
 * [HTRACE-48] - use -L 1 in format.sh to make it work on macosx
 * [HTRACE-52] - FIgure out content-type handling in JSON API
 * [HTRACE-55] - Add Clean command to htrace go
 * [HTRACE-57] - Fix LocalFileSpanReceiver to avoid adding irrelevant
wrapper object
 * [HTRACE-68] - Clean up LocalFileSpanReceiver and SpanReceiverBuilder
a bit
 * [HTRACE-71] - format.sh: only format src/ directory
 * [HTRACE-90] - Remove Guava and shade other depdendencies in HTrace
subprojects
 * [HTRACE-93] - Add null check to SamplerBuilder
 * [HTRACE-97] - Support both -D and --D when specifying conf vars on
the command-line
 * [HTRACE-98] - Web Server should use PrefixHandler
 * [HTRACE-99] - log.go fails to create new log files
 * [HTRACE-100] - htraced: put query parameters in the URL, not in the
GET request body
 * [HTRACE-101] - Add better error-checking to Java HTrace client
 * [HTRACE-103] - Reset unclosed spans after TestBadClient
 * [HTRACE-107] - Support for Greater Than operator in htraced
 * [HTRACE-109] - fix TestHTracedRESTReceiver unit test failures
 * [HTRACE-110] - Fix rat check failure due to
dependency-reduced-pom.xml in htrace-hbase
 * [HTRACE-111] - HTrace Java client API fixes for 3.2
 * [HTRACE-112] - Fix LocalFileSpanReceiver to avoid BG thread and
problems around close()
 * [HTRACE-115] - The htraced datastore should use uint64 for span ids
rather than int64
 * [HTRACE-116] - htraced's data.store.clear option does not work
 * [HTRACE-117] - graph.go: sort children array to get deterministic
output
 * [HTRACE-118] - Java API: support setting the parents of a span after
the span is created
 * [HTRACE-119] - detach of NullScope singleton should not fail
 * [HTRACE-123] - fix graphviz functionality in the htrace command
 * [HTRACE-126] - BUILDING.txt should not say that godep is required,
because we auto-install godep
 * [HTRACE-127] - fix ./build/htrace dumpAll
 * [HTRACE-128] - RAT check hangs on files under
htrace-core/src/web/lib/rome-2.1.0
 * [HTRACE-132] - Trace#startSpan(String, TraceInfo) must not create
spans with spanId == 0
 * [HTRACE-133] - HTracedRESTReceiver drops spans when close() is called
 * [HTRACE-140] - Web UI of htraced causes high CPU usage if search
result is empty
 * [HTRACE-141] - Fix nits of swimlane view of htraced Web UI
 * [HTRACE-148] - Add deserialization utility method to MilliSpan for
testing outside of htrace-core
 * [HTRACE-151] - htrace-core, htrace-htraced: Use shaded import paths
 * [HTRACE-152] - Fix TestHTracedRESTReceiver
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException
 * [HTRACE-158] - htraced: add TraceEnabled, DebugEnabled, etc.
functions to log.go

Re: [VOTE] HTrace 4.0 Release Candidate 0

[Masatake Iwasaki]

> I navigate to the decompressed RC directory and try mvn clean 
install... it

> enters into a loop!
>
> ...
> [INFO] Dependency-reduced POM written at:
> 
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml

> [INFO] Dependency-reduced POM written at:
> 
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml


HTRACE-234 was filed for this. Which version of Maven are you using?

Thanks,
Masatake Iwasaki


On 9/8/15 00:47, Lewis John Mcgibbney wrote:

Hi Colin,
Nice work in getting the release candidate prepared and available for
review.
Where is the KEYS file?
I tried the one here
http://archive.apache.org/dist/incubator/htrace/KEYS
It does not contain your sig so I cannot verify the release sigs.
Moving on,

Notes Binaries Archives Standards Apache Generated Unknown 0 0 0 116 92 0 15
Unapproved licenses are as follows

bootstrap-theme.css
bootstrap-theme.min.css
bootstrap.css
bootstrap.min.css
backbone-1.1.2.js
bootstrap.js
bootstrap.min.js
d3.min.js
jquery-2.1.4.js
moment-2.10.3.js
npm.js
underscore-1.7.0.js
SpanProtos.java
dependency-reduced-pom.xml

None of the above libraries are declared within NOTICE.txt

I navigate to the decompressed RC directory and try mvn clean install... it
enters into a loop!

...
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml

I try mvn clean install -DskipTests with same result

...
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at:
/usr/local/htrace_4/htrace-4.0.0-incubating/htrace-hbase/dependency-reduced-pom.xml

Any ideas what is going on?
Thanks
Lewis

On Mon, Sep 7, 2015 at 10:15 AM, <
dev-digest-h...@htrace.incubator.apache.org> wrote:


I've posted the first release candidate here:

http://people.apache.org/~cmccabe/htrace/releases/4.0.0/rc0

The jars have been staged here:

https://repository.apache.org/content/repositories/orgapachehtrace-1017

There's a lot of great stuff in this release, including a new web UI,
many bug fixes, API improvements and enlargement of span IDs to 128
bits to avoid conflicts.

The vote will run for 5 days.

Re: [VOTE] HTrace 4.0 Release Candidate 0

[Masatake Iwasaki]

Thanks for putting this up, Colin.

- verified mds and signature
- ran "mvn install" without test failure
- ran "mvn package -Pnative" without test failure in htrace-c
- built src tarball by running "mvn clean install -DskipTests 
assembly:single -Pdist"
- launched htraced and sent test tracing spans by HTracedRESTReceiver 
and checked the spans by Web-UI


I'm +1 if the issue about NOTICE.txt Lewis pointed out is not critical 
for release.


Masatake Iwasaki

On 9/5/15 08:02, Colin P. McCabe wrote:

I've posted the first release candidate here:

http://people.apache.org/~cmccabe/htrace/releases/4.0.0/rc0

The jars have been staged here:

https://repository.apache.org/content/repositories/orgapachehtrace-1017

There's a lot of great stuff in this release, including a new web UI,
many bug fixes, API improvements and enlargement of span IDs to 128
bits to avoid conflicts.

The vote will run for 5 days.

cheers,
Colin

Release Notes - HTrace - Version 4.0

** Sub-task
 * [HTRACE-208] - Remove deprecated addKVAnnotation(byte[], byte[]) method
 * [HTRACE-209] - Make span ID 128 bit to avoid collisions
 * [HTRACE-210] - Remove TrueIfTracingSampler
 * [HTRACE-211] - Move htrace-core classes to the
org.apache.htrace.core namespace
 * [HTRACE-212] - Change version to 4.0
 * [HTRACE-214] - De-globalize Tracer.java
 * [HTRACE-215] - Simplify the Sampler type
 * [HTRACE-216] - SpanReceivers should not fill in ProcessId
 * [HTRACE-217] - Rename ProcessId to TracerId
 * [HTRACE-222] - Add SpanReceiverPool
 * [HTRACE-228] - Fix subprojects to refer to new
org.apache.htrace.core namespace
 * [HTRACE-229] - htrace-webapp needs to be updated to refer to
"tracerid" not "processid"
** Bug
 * [HTRACE-159] - libhtrace.so: use HRPC endpoint of htraced
 * [HTRACE-164] - htrace hrpc: use msgpack for serialization
 * [HTRACE-166] - Add tabbed view
 * [HTRACE-167] - Update go build instructions in BUILDING.txt
 * [HTRACE-171] - htraced godeps should use
github.com/ugorji/go/codec rather than github.com/ugorji/go
 * [HTRACE-174] - Refactor GUI
 * [HTRACE-177] - htrace-zipkin: shade all dependencies
 * [HTRACE-182] - htraced: add rpm build via -Prpm
 * [HTRACE-189] - gui: fix error handling in a few places
 * [HTRACE-190] - htraced: allow querying by process ID
 * [HTRACE-191] - gui: add "duration" to span details, filter out "selected"
 * [HTRACE-192] - gui: when expanding parents or children, sort the
spans by begin time
 * [HTRACE-193] - gui: avoid doing multiple redraws when
spanResults is updated
 * [HTRACE-196] - gui: add scrolling for spans view
 * [HTRACE-201] - htrace-web: URL-Encode query JSON
 * [HTRACE-202] - htrace-web: fix "converting circular object to
JSON" error when pressing "clear" button
 * [HTRACE-218] - Fix issues with finding json-c includes and librt
in the native library
 * [HTRACE-219] - Add -Dleveldb.prefix and -Djsonc.prefix build options
 * [HTRACE-220] - htraced: should be able to set log.path to the
empty string via "-Dlog.path=" on the command line
 * [HTRACE-223] - gobuild.sh: fix issue where maven succeeds if go
build fails
 * [HTRACE-224] - htrace C client: htrace_conf_get_u64,
htrace_conf_get_double can't handle spaces at the end of strings
 * [HTRACE-230] - Make TracerBuilder like all other Builders; an
internal rather than adjacent class
 * [HTRACE-233] - htrace-zipkin should explicitly include slf4j-api
to avoid ClassNotFoundException
 * [HTRACE-234] - Add workaround to prevent htrace-hbase from
getting in an infinite loop while creating the dependency-reduced pom
** Improvement
 * [HTRACE-29] - add javascript web UI for htraced
 * [HTRACE-160] - htraced: support continuing a query from where
the client left it off by sending a previous span
 * [HTRACE-162] - htraced hrpc: some logging improvements
 * [HTRACE-170] - Optimize use of Random in htrace-core by using
ThreadLocalRandom
 * [HTRACE-172] - Move minJdk to 1.7 (JDK 7)
 * [HTRACE-175] - Add Trace#addKVAnnotation convenience method
 * [HTRACE-176] - Expose ZipkinSpanReceiver configuration keys externally
 * [HTRACE-180] - Move the GUI to a top-level subproject
 * [HTRACE-184] - Expose PROCESS_ID_KEY configuration key
 * [HTRACE-186] - gui: support finding the parents and children of
spans, add owl
 * [HTRACE-194] - gui: support multiple selections, zooming to fit
a group of spans, deleting a group of spans
 * [HTRACE-197] - htraced build: set RUNPATH if possible
 * [HTRACE-199] - gui: Double clicking on spans should bring up span details
 * [HTRACE-203] - htrace-web: pressing enter should dismiss the
modal dialog box
 * [HTRACE-204] - htrace-web: add draggable bar which allows more
or less visual space for process name in search view

Re: Preparing for HTrace 4.0

[Masatake Iwasaki]

Sounds great to me!

Masatake

On 9/4/15 12:26, Colin P. McCabe wrote:

Hi guys,

I think it's time for a new release of HTrace.  We've resolved 58
JIRAs since our last release!  I can RM.

cheers,
Colin

Re: htrace-core compatibility policy for htrace 4.x

[Masatake Iwasaki]

I agree with the policy.


> Major releases should change the namespace of htrace-core classes so that
> both a 4.x and a 5.x jar can reside on the same CLASSPATH

Should we have a namespace convention for this?
We moved classes from org.apache.htrace to org.apache.htrace.core on 4.0
but need new word other than "core" for 5.0.
Simple way for this would be containing version number in package name
like "org.apache.htrace.5".


> Let's focus on just compatibility rules for htrace-core right now,
> since that's where our integration issues are. The other subprojects
> of htrace generally don't have the same integration issues.

I thinks it is reasonable.
Any version of receiver with the same major version keeps working
as far as htrace-core keeps compatibility.


Thanks,
Masatake Iwasaki


On 9/15/15 09:13, Colin McCabe wrote:

Hi all,

In the recent 4.0 release, we changed the htrace-core API. The API
that programs use to create traces, annotations, etc. (aka the "Java
client API") went through some changes. This was necessary to clean up
some core architectural issues (such as the use of overly short 64 bit
IDs that will collide in a real-world deployment, or the overuse of
globals.)

Since we want to make it easy for projects to integrate with HTrace, I
think we should have some compatibility rules for htrace-core for the
future.

Specifically, I think that we should include only backwards-compatible
changes to the htrace-core API in HTrace 4.x So, for example, adding a
new function is OK. Deleting an existing function or altering it in an
incompatible way is not. It is OK to add a new function to a public
abstract base class (provided you also add a default implementation in
the base), but not to add a new function to a public interface, since
that would break compilation.

We should save incompatible changes for HTrace 5.x. In general, each
"major release" such as 4.x or 5.x should contain only compatible
changes to htrace-core. There should be no guarantees between 4.x and
5.x, or between any major releases-- this is the time to address
architectural debt that can't be resolved any other way. Major
releases should change the namespace of htrace-core classes so that
both a 4.x and a 5.x jar can reside on the same CLASSPATH, similar to
how we did with 3.x and 4.x. This is important because it will require
some time for downstream projects to upgrade from 4.x to 5.x, and in
the meantime we must avoid CLASSPATH conflict issues. There is no
requirement that tracing work when the major version of the client and
the span receivers are different. However, the programs themselves
should function.

Let's focus on just compatibility rules for htrace-core right now,
since that's where our integration issues are. The other subprojects
of htrace generally don't have the same integration issues. For
example, it is easy for an admin to standardize on a single version of
htrace-hbase or htrace-htraced across the entire cluster. They simply
install the jars for the version they want. It is not easy for that
same admin to standardize htrace-core, since they might have Hadoop
pulling in 4.1 and HBase pulling in 4.0. The different subprojects are
also at different levels of maturity. For example, htrace-flume is
still very immature, whereas htrace-htraced is starting to get more
mature. So I think the subprojects should come up with their own
compatibility policies rather than trying to be one-size-fits-all.

This policy should only apply to publicly visible symbols in
htrace-core, not to private or package-private symbols.  Test
functions should also not be covered, since they don't appear in the
final htrace-core jar.

I think having a compatibility policy for htrace-core will be very
nice for the users of our core APIs. Let me know what you think.

best,
Colin

Re: Podling Report Reminder - June 2016

[Masatake Iwasaki]

I'm +1 too. Thanks, Stack.


Masatake Iwasaki


On 6/5/16 03:38, Billie Rinaldi wrote:

+1, thanks Stack.

On Sat, Jun 4, 2016 at 10:43 AM, Stack <st...@duboce.net> wrote:


Thanks for the reminder John (sorry you had to chase us)

How's this? (Its been a quiet period):



HTrace

HTrace is a tracing framework intended for use with distributed systems
written in java.

HTrace has been incubating since 2014-11-11.

Three most important issues to address in the move towards graduation:

   1. Continue to grow the HTrace community
   2. Continue to explore projects integrating HTrace
   3. Continue to develop and release stable HTrace incubating artifacts

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

   There are no issues requiring board attention at this time.

How has the community developed since the last report?

  * GSoC student, Nisala Mendis, mentored by Colin McCabe working on
making an Apache Kudu (incubating) SpanReceiver

How has the project developed since the last report?

  * 18 issues resolved since last report
  * YCSB integrated HTrace

Date of last release:

   * htrace-4.1.0-incubating on March 5th, 2016

When were the last committers or PMC members elected?

   * None this period. Last added committer: Abraham Elmahrek - 2.11.15




On Wed, Jun 1, 2016 at 4:44 PM, <johndam...@apache.org> wrote:


Dear podling,

This email was sent by an automated system on behalf of the Apache
Incubator PMC. It is an initial reminder to give you plenty of time to
prepare your quarterly board report.

The board meeting is scheduled for Wed, 15 June 2016, 10:30 am PDT.
The report for your podling will form a part of the Incubator PMC
report. The Incubator PMC requires your report to be submitted 2 weeks
before the board meeting, to allow sufficient time for review and
submission (Wed, June 1st).

Please submit your report with sufficient time to allow the Incubator
PMC, and subsequently board members to review and digest. Again, the
very latest you should submit your report is 2 weeks prior to the board
meeting.

Thanks,

The Apache Incubator PMC

Submitting your Report

--

Your report should contain the following:

*   Your project name
*   A brief description of your project, which assumes no knowledge of
 the project or necessarily of its field
*   A list of the three most important issues to address in the move
 towards graduation.
*   Any issues that the Incubator PMC or ASF Board might wish/need to be
 aware of
*   How has the community developed since the last report
*   How has the project developed since the last report.

This should be appended to the Incubator Wiki page at:

http://wiki.apache.org/incubator/June2016

Note: This is manually populated. You may need to wait a little before
this page is created from a template.

Mentors
---

Mentors should review reports for their project(s) and sign them off on
the Incubator wiki page. Signing off reports shows that you are
following the project - projects that are not signed may raise alarms
for the Incubator PMC.

Incubator PMC

Re: Time for a new release?

[Masatake Iwasaki]

I think it's time to cut the 4.1 release.


+1.

I volunteer to do release management work, if you like.

Thanks,
Masatake Iwasaki


On 1/14/16 07:52, Colin P. McCabe wrote:

Hi all,

Happy new year!

I think it's time to cut the 4.1 release.  We've fixed up a lot of
bugs, and made a lot of progress since 4.0.1.

If everyone agrees, I'll post something in the next few days.

best,
Colin

Re: git tag and branch naming

[Masatake Iwasaki]

Sorry for late reply.

I agree with the proposed naming conversion for branches and tags.
If there is no objection further, we should close HTRACE-331 and
prepare for the next release.

Thanks,
Masatake Iwasaki

On 12/15/15 04:53, Colin P. McCabe wrote:

As part of our release process, we create git tags for each release
candidate (RC)... for example, 3.1.0RC9 and 4.0.1RC1.  We also often
use release branches-- for example, the "4.0" branch.

As Sean Busbey pointed out, we should also be creating "release" tags,
so that people who want to check out the release can do so without
having to figure out which RC was anointed as the release.  I also
think we should adopt a naming convention for release branches and
tags so that people attempting to check out tags don't accidentally
check out branches, and vice versa.

The branch and tag naming is confusing right now.  For example,
someone running "git checkout 4.0" might be surprised to learn that
this checks out a branch currently containing 4.0.1, not the git tag
for the 4.0 release.

I'm thinking we should adopt the following convention:
* release tags should have "release" in the name. So the tag for
htrace 4.1 should be "4.1-release"
* RC tags continue to be "4.1-RC1" and so forth.
* release branches should have "branch" in the name. So the branch for
4.1 should be "branch-4.1".  In general, branches should not include
"RC[0-9]" or "release" in the names, to avoid confusion with the tags.

Let me know what you think.  If you guys agree, I will also create
4.0-release and 4.0.1-release tags corresponding to those releases.

best,
Colin

Re: Time for a new release?

[Masatake Iwasaki]

Hi Colin,

I can see branch-4.1 created in the repo.
Have you already started to create RC?

Regards,
Masatake Iwasaki

On 1/15/16 10:18, Masatake Iwasaki wrote:

I think it's time to cut the 4.1 release.


+1.

I volunteer to do release management work, if you like.

Thanks,
Masatake Iwasaki


On 1/14/16 07:52, Colin P. McCabe wrote:

Hi all,

Happy new year!

I think it's time to cut the 4.1 release.  We've fixed up a lot of
bugs, and made a lot of progress since 4.0.1.

If everyone agrees, I'll post something in the next few days.

best,
Colin

Re: HTrace 4.1 release candidate 2

[Masatake Iwasaki]

+1

- verified signature and checksums
- built from source and ran unit tests
- built hadoop against htrace-4.1.0-incubating and saw the traces of dfs 
put/get by htraced Web-UI.


Thanks,
Masatake Iwasaki

On 2/10/16 12:00, Colin P. McCabe wrote:

Hi all,

I've posted the second release candidate for HTrace 4.1 here:

http://people.apache.org/~cmccabe/htrace/releases/4.1.0/rc2/

The jars have been staged here:

https://repository.apache.org/content/repositories/orgapachehtrace-1022

Compared to rc1, this rc includes HTRACE-334 and HTRACE-342.

HTrace 4.1 brings a lot of robustness improvements.  There were major
improvements to htraced and the web UI, as well as new metrics added.
There were numerous build fixups, and we added Docker support, to
ensure a repeatable build.

Check it out.  The vote will run for 5 days.

cheers,
Colin


Release Notes - HTrace - Version 4.1
** Bug
 * [HTRACE-114] - Fix compilation error of htrace-hbase against hbase-1.0.0
 * [HTRACE-238] - Change maven compiler source level to 1.7 to
match targetJdk
 * [HTRACE-243] - Remove duplicate maven-assembly-plugin
configuration section in htrace-htraced/pom.xml
 * [HTRACE-245] - NOTICE.txt: change "developed by The Apache
Software...” to "developed at The Apache Software...”
 * [HTRACE-246] - HTrace WebApp not properly defined and therefore
not packaged into .war
 * [HTRACE-248] - HTraced should gracefully shutdown if stopped
 * [HTRACE-249] - Script and doc on how to publish website
 * [HTRACE-251] - Fix "mvn clean" target
 * [HTRACE-253] - Tracer loadSamplers and loadSpanReceivers logs
are too chatty
 * [HTRACE-256] - Change the artifactId for htrace-core in branch
4.0 to be htrace-core4
 * [HTRACE-257] - htrace-htraced: add web symlink rather than
generating programmatically
 * [HTRACE-262] - Temporarily suppress doclint for Java 8 to
prevent build failure
 * [HTRACE-266] - Make the CLIENT_REST_MAX_SPANS_AT_A_TIME_KEY
config key more consistent with other configs
 * [HTRACE-267] - Move owl logo licensing information from NOTICE to LICENSE
 * [HTRACE-268] - Remove Units and go-codec from LICENSE since they
are not contained in the source release
 * [HTRACE-272] - TracerPool must not load multiple inscance of
same receiver class when a simple classname is given
 * [HTRACE-279] - Fix issues where the HTracedSpanReceiver was
using the wrong JSON serialization for spans and add validation to
htraced REST ingest path
 * [HTRACE-280] - htraced: add metrics about total spans added and
dropped per address
 * [HTRACE-281] - htraced: add example/htraced-conf.xml
 * [HTRACE-282] - htraced: reap spans which are older than a
configurable interval
 * [HTRACE-283] - Heartbeater should wait for goroutine to finish on close
 * [HTRACE-284] - htrace-htraced, htrace-flume: do not treat the
shaded version of commons-logging as provided
 * [HTRACE-285] - htraced tool: fix query parsing and add query_test
 * [HTRACE-289] - Fix TraceEnabled, etc. logger methods for
conditional logging
 * [HTRACE-294] - htraced: fix some metrics issues
 * [HTRACE-297] - htraced: avoid serializing spans to json unless
TRACE logging is enabled
 * [HTRACE-300] - Reaper should be initialized before shards are activated
 * [HTRACE-301] - htraced: fix unit tests that aren't waiting for
spans to be written, use semaphore for WrittenSpans
 * [HTRACE-302] - htraced: Add admissions control to HRPC to limit
the number of incoming messages
 * [HTRACE-304] - htraced: fix bug with GREATER_THAN queries
 * [HTRACE-307] - htraced: queries sometimes return no results even
when many results exist due to confusion in iterator usage
 * [HTRACE-311] - htraced: Fix logging to stdout via -Dlog.path=
 * [HTRACE-316] - htrace-web: span.js issue: span ID string length
is 32, not 36
 * [HTRACE-317] - Fix the documentation for adding tracing to an
application to reflect HTrace 4.x API changes
 * [HTRACE-328] - htraced continues scanning in some cases even
when no more results are possible

** Improvement
 * [HTRACE-342] - centralize building instructions in BUILDING.txt
 * [HTRACE-334] - htrace-web: Make limit of search and children API
configurable
 * [HTRACE-129] - htraced: add /server/stats REST endpoint
 * [HTRACE-156] - HTrace GUI: add about view
 * [HTRACE-181] - gui: Split "about" screen
 * [HTRACE-237] - Optimize htraced span receiver
 * [HTRACE-239] - Add htrace/impl/TestZipkinSpanReceiver.java
 * [HTRACE-260] - htrace-zipkin should not set the obsolete
duration field in thrift
 * [HTRACE-271] - Add log4j.properties to all submodule tests
 * [HTRACE-276] - Shade classes into org.apache.htrace.shaded
rather than org.apache.htrace
 * [HTRACE-286] - htraced: improvements to logging, daemon startup,
and configuration
 * [HTRACE-290] - htraced: Fix per-faculty log lev

Re: Board Report

[Masatake Iwasaki]

> Is it 2015 rather than 2014? (Time is going fast though)

It is since 2014.
Time really goes by so fast:-)

On 4/7/16 04:41, Stack wrote:

On Wed, Apr 6, 2016 at 12:03 PM, Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:


Thanks for pinging, John.

I wrote a draft of board report.
Let me have feedback from HTrace PMC members.

Masatake

-
HTrace

HTrace is a tracing framework intended for use with distributed systems
written in java.

HTrace has been incubating since 2014-11-11.



Is it 2015 rather than 2014? (Time is going fast though)




Three most important issues to address in the move towards graduation:

   1. Continue to grow the HTrace community
   2. Continue to explore projects integratiing HTrace
   3. Continue to develop and release stable HTrace incubating artifacts

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

None

How has the community developed since the last report?

* YCSB community are working on integrating HTrace.
* Google Summer of Code 2016 projects for Apache HTrace are being
planned.  There are proposals such as
   * distributed tracing for s3 and other alternative Hadoop FS
   * Kudu span receiver
   * integrating HTrace into Hadoop YARN

How has the project developed since the last report?

+70 issues were fixed and released as part of 4.1.0-incubating.
* Improved htraced a lot in Web-UI, robustness, metrics and logging
* Added Docker support for development
* Added Documentation for developers

Date of last release:

* htrace-4.1.0-incubating on March 5th, 2016

When were the last committers or PMC members elected?

* None this period. Last added committer: Abraham Elmahrek - 2.11.15

Signed-off-by:

   [ ](htrace) Jake Farrell
   [ ](htrace) Todd Lipcon
   [ ](htrace) Lewis John Mcgibbney
   [ ](htrace) Andrew Purtell
   [ ](htrace) Billie Rinaldi
   [ ](htrace) Michael Stack

Shepherd/Mentor notes:

-


On 4/6/16 08:42, John D. Ament wrote:


Dear HTrace Podling,

Just a reminder about your board report, its due tomorrow.

John



Otherwise +1,

St.Ack

Re: Issues Building HTrace Master

[Masatake Iwasaki]

Thanks for reporting this, Lewis.

I can reproduce the issue by go1.4.3 linux/amd64.
The cause seems to be HTRACE-361 which replaced link flags
from "-X name value" to "-X name=value".

I think we should fix gobuild.sh to keep supporting go 1.4
at lease in htrace 4.


I filed HTRACE-371.

Masatake


On 5/6/16 02:49, Colin McCabe wrote:

Hmm.  The build works for me as well.  I also think the Go version would
be a good thing to check.  We should probably start setting a minimum Go
version to avoid build problems.

best,
Colin


On Thu, May 5, 2016, at 08:30, Stack wrote:

It works for me Lewis. godep is failing. Go version? If you run
./htrace/htraced/go/gobuild.sh directly, the issue may be clearer?
St.Ack

On Wed, May 4, 2016 at 9:00 PM, Lewis John Mcgibbney <
lewis.mcgibb...@gmail.com> wrote:


Hi Folks,
when I attempt to build master branch I get the following error

lmcgibbn@LMC-032857 /usr/local/incubator-htrace(master) $ mvn -version
Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06;
2015-04-22T04:57:37-07:00)
Maven home: /usr/local/apache-maven-3.3.3
Java version: 1.8.0_66, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"

mvn clean install -DskipTests -DcreateDependencyReducedPom=false

[INFO]

[INFO] Building htrace-htraced 4.2.0-incubating-SNAPSHOT
[INFO]

[INFO]
[INFO] --- maven-clean-plugin:2.6.1:clean (default-clean) @ htrace-htraced
---
[INFO] Deleting /usr/local/incubator-htrace/htrace-htraced/target
[INFO] Deleting /usr/local/incubator-htrace/htrace-htraced/go/build
(includes = [], excludes = [])
[INFO] Deleting /usr/local/incubator-htrace/htrace-htraced/go/pkg (includes
= [], excludes = [])
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @
htrace-htraced ---
[INFO]
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @
htrace-htraced ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/usr/local/incubator-htrace/htrace-htraced/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
htrace-htraced ---
[INFO] Compiling 8 source files to
/usr/local/incubator-htrace/htrace-htraced/target/classes
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (go_compile) @ htrace-htraced ---
[WARNING] Parameter tasks is deprecated, use target instead
[INFO] Executing tasks

main:
  [exec] Installing godep...
  [exec] godep: [WARNING]: godep should only be used inside a valid go
package directory and
  [exec] godep: [WARNING]: may not function correctly. You are probably
outside of your $GOPATH.
  [exec] godep: [WARNING]:Current Directory:
/usr/local/incubator-htrace/htrace-htraced/go
  [exec] godep: [WARNING]:$GOPATH:

/usr/local/incubator-htrace/htrace-htraced/go/build:/usr/local/incubator-htrace/htrace-htraced/go
  [exec] /usr/local/incubator-htrace/htrace-htraced/go/build/godep
restore...
  [exec] Building 4.2.0-incubating-SNAPSHOT
[4a101e0d1c435937feba9fc876882ffbf71259b1]
  [exec] github.com/ugorji/go/codec
  [exec] htrace/conf
  [exec] github.com/alecthomas/units
  [exec] github.com/gorilla/context
  [exec] github.com/jmhodges/levigo
  [exec] github.com/alecthomas/kingpin
  [exec] github.com/gorilla/mux
  [exec] htrace/common
  [exec] htrace/test
  [exec] htrace/client
  [exec] htrace/htraced
  [exec] htrace/htracedTool
  [exec] # htrace/htracedTool
  [exec] usage: 6l [options] main.6
  [exec]   -1use alternate profiling code
  [exec]   -8assume 64-bit addresses
  [exec]   -B info
  [exec] define ELF NT_GNU_BUILD_ID note
  [exec]   -Ccheck Go calls to C code
  [exec]   -D addr
  [exec] data address
  [exec]   -E sym
  [exec] entry symbol
  [exec]   -I interp
  [exec] set ELF interp
  [exec]   -L dir
  [exec] add dir to library path
  [exec]   -H head
  [exec] header type
  [exec]   -Kadd stack underflow checks
  [exec]   -Oprint pc-line tables
  [exec]   -Qdebug byte-register code gen
  [exec]   -R rnd
  [exec] address rounding
  [exec]   -Scheck type signatures
  [exec]   -T addr
  [exec] text address
  [exec]   -Vprint version and exit
  [exec]   -Wdisassemble input
  [exec]   -X name value
  [exec] define string data
  [exec]   -Zclear stack frame on entry
  [exec]   -adisassemble output
  [exec]   -cdump call graph
  [exec]   -ddisable dynamic executable
  [exec]   -extld ld
  [exec] 

Re: HTrace 3.2 - Can't build HTrace-HTraced package

[Masatake Iwasaki]

Hi Mihir,


[exec] godep: open Godeps/Godeps.json: no such file or directory
[exec] failed to set up dependencies


Did you try to build htrace-htraced on OS X?
looks like HTRACE-255: https://issues.apache.org/jira/browse/HTRACE-255

Since the fix version of HTRACE-255 is 4.0.1,
you need equivalent fix on 3.2 if that's the case.

Thanks,
Masatake Iwasaki

On 12/16/16 14:26, Mihir Monani wrote:

Hi,

I have installed all the dependency required for HTrace.

I am able to build 4.1.0 without any problems.

Here is what i did :-

*git clone https://github.com/apache/incubator-htrace.git
<https://github.com/apache/incubator-htrace.git>*
*git checkout 3.2*
*mvn clean install -DskipTests -Dmaven.javadoc.skip=true*

  I am not able to build 3.2 branch from git.

Errors which i face :-
[INFO]

[INFO]

[INFO] Building htrace-htraced 3.2.0-incubating
[INFO]


### dowanloads some dependencies ###

[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
htrace-htraced ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/home/user/git/incubator-htrace_3.2/htrace-htraced/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
htrace-htraced ---
[INFO] Compiling 1 source file to
/home/user/git/incubator-htrace_3.2/htrace-htraced/target/classes
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (go_compile) @ htrace-htraced ---
[WARNING] Parameter tasks is deprecated, use target instead
[INFO] Executing tasks

main:
  [exec] godep restore...
  [exec] godep: [WARNING]: godep should only be used inside a valid go
package directory and
  [exec] godep: [WARNING]: may not function correctly. You are probably
outside of your $GOPATH.
  [exec] godep: [WARNING]: Current Directory:
/home/user/git/incubator-htrace_3.2/htrace-htraced/src/go/build
  [exec] godep: [WARNING]: $GOPATH:
/home/user/git/incubator-htrace_3.2/htrace-htraced/src/go/build:/home/user/git/incubator-htrace_3.2/htrace-htraced/src/go
  [exec] godep: open Godeps/Godeps.json: no such file or directory
  [exec] failed to set up dependencies
[INFO]

[INFO] BUILD FAILURE
[INFO]




I am not able to think what is the problem with 3.2 ( or 3.x) version. I
also face same error with 3.1 too.

Thanks,
Mihir Monani

Re: [DISCUSS] Attic podling Apache HTrace?

[Masatake Iwasaki]

Hi Mike,

Thanks for putting this issue up.

> Want to bring up a potentially uncofortable topic for some. Is it time to
> retire/attic the project?

I would like to keep the project alive.
While we are silent for months,
many of the committers are still working on projects
using HTrace (such as Hadoop and HBase) and
we are capable to make new release if new major issues are found.

> HBase is still on version 3. So is Accumulo, I think. Hadoop is on 4.1,
> which is a good sign, but I haven't heard much from them recently.

I will look into HBASE-14451 again and try to make it move forward.
Since one of the intent of big change in HTrace-4 is
making better end-to-end tracing (e.g. from HBase to HDFS),
bumping HTrace in HBase up to 4 would reveal the next task.

Regards,
Masatake Iwasaki

On 8/17/17 02:00, Mike Drob wrote:

Hi folks,

Want to bring up a potentially uncofortable topic for some. Is it time to
retire/attic the project?

We've seen a minimal amount of activity in the past year. The last release
had two bug fixes, and had been pending for several months before somebody
reminded me to push the artifacts to subversion from the staging directory.

I'd love to see a renewed set of activity here, but I don't think there is
a ton of interest going on.

HBase is still on version 3. So is Accumulo, I think. Hadoop is on 4.1,
which is a good sign, but I haven't heard much from them recently. I
definitely do no think we are at the point where a lack of releases and
activity is a sign of super advanced maturity and stability.

Your thoughts?

Mike

Re: podling report draft

[Masatake Iwasaki]

LGTM too.
Thanks for putting this up, Colin.

Masatake Iwasaki

On 6/8/17 13:54, Stack wrote:

LGTM

On Wed, Jun 7, 2017 at 12:19 PM, Colin McCabe <cmcc...@apache.org> wrote:


Here's a draft of the podling report.  Let me know if I left out
anything...

-
HTrace is a tracing framework intended for use with distributed systems
written in java.

HTrace has been incubating since 2014-11-11.

Three most important issues to address in the move towards graduation:

   1. Continue to grow the Apache HTrace community
   2. Continue to explore projects integrating Apache HTrace
   3. Continue to develop and release stable Apache HTrace incubating
artifacts

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

   There are no issues requiring board attention at this time.

How has the community developed since the last report?

   Mailing lists have been quiet.

How has the project developed since the last report?

   We are on track to make an HTrace 4.3 release.

How would you assess the podling's maturity?
Please feel free to add your own commentary.

   [ ] Initial setup
   [ ] Working towards first release
   [x] Community building
   [ ] Nearing graduation
   [ ] Other:

Date of last release:

   2016-10-06

When were the last committers or PPMC members elected?

   2016-10-03

Re: [VOTE] HTrace-4.3.0-incubating RC3

[Masatake Iwasaki]

+1

* verified signature and mds.
* built and installed from source tarball by maven-3.0.4.
* built hadoop trunk against htrace-4.3.0-incubating.
* verified that spans in HDFS works by HTracedSpanReceiver and htraced.

Thanks,
Masatake Iwasaki

On 5/17/17 05:40, Mike Drob wrote:

Hi Devs,

Please consider the following release candidate for Apache HTrace
4.3.0-incubating!

http://people.apache.org/~mdrob/htrace-4.3.0-incubating-rc3/

The jars have been staged here:

https://repository.apache.org/content/repositories/orgapachehtrace-1029

There's a few minor fixes in this release:

** Bug
 * [HTRACE-387] - htrace-c should have a test dependency on htrace
-htraced
 * [HTRACE-393] - Binary annotations are being shown as base64 in the
zipkin frontend
 * [HTRACE-398] - Enforce requirement to build with Maven 3.0.4 from
HTRACE-236

** Improvement
 * [HTRACE-386] - htrace-c: add functions for retrieving the span ID to
the public API

RC History:
RC0 - Discarded due to user error.
RC1 - Missing some version string updates, vote failed.
RC2 - Discarded due to missing HTRACE-398
RC3 - Current

This vote will run for 5 days ( Sun May 21 21:30:00 UTC 2017 )

Thanks,
Mike

Re: Podling Report Reminder - September 2017

[Masatake Iwasaki]

+1

Thanks,
Masatake Iwasaki

On 9/9/17 01:46, Colin McCabe wrote:

Thanks, Stack!  You're the best.

+1.



On Thu, Sep 7, 2017, at 15:10, Stack wrote:

Ok. I pushed it (and signed off on the report too... You want to Billie?)

Thanks,
S

On Thu, Sep 7, 2017 at 3:04 PM, Billie Rinaldi <bil...@apache.org> wrote:


+1. Thanks, Stack :)

On Thu, Sep 7, 2017 at 2:44 PM, Stack <st...@duboce.net> wrote:


Report is due. How is this? I'd like to put it up today since we are late
already.
Thanks,
St.Ack





HTrace

HTrace is a tracing framework intended for use with distributed systems
written in java.

HTrace has been incubating since 2014-11-11.

Three most important issues to address in the move towards graduation:

  1. Continue to grow the Apache HTrace community
  2. Continue to explore projects integrating Apache HTrace
  3. Continue to develop and release stable Apache HTrace incubating
artifacts

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be

aware

of?

  In a DISCUSSION thread on our dev list [1], one of us suggested
  moving the podling htrace to the attic because of inactivity.

  A healthy back and forth suggested that we are not
  dead yet, that we cut back our scope radically, and
  that we give it another try before calling it quits.

1. https://s.apache.org/9ybr

How has the community developed since the last report?

  Mailing lists continue quiet (bar the above noted thread).

How has the project developed since the last report?

  No development.

How would you assess the podling's maturity? Please feel free to add your
own
commentary.

  [ ] Initial setup
  [ ] Working towards first release
  [x] Community building
  [ ] Nearing graduation
  [ ] Other:

Date of last release:

  2016-10-06

When were the last committers or PPMC members elected?

  2016-10-03

On Tue, Sep 5, 2017 at 6:40 PM, <johndam...@apache.org> wrote:


Dear podling,

This email was sent by an automated system on behalf of the Apache
Incubator PMC. It is an initial reminder to give you plenty of time to
prepare your quarterly board report.

The board meeting is scheduled for Wed, 20 September 2017, 10:30 am

PDT.

The report for your podling will form a part of the Incubator PMC
report. The Incubator PMC requires your report to be submitted 2 weeks
before the board meeting, to allow sufficient time for review and
submission (Wed, September 06).

Please submit your report with sufficient time to allow the Incubator
PMC, and subsequently board members to review and digest. Again, the
very latest you should submit your report is 2 weeks prior to the board
meeting.

Thanks,

The Apache Incubator PMC

Submitting your Report

--

Your report should contain the following:

*   Your project name
*   A brief description of your project, which assumes no knowledge of
 the project or necessarily of its field
*   A list of the three most important issues to address in the move
 towards graduation.
*   Any issues that the Incubator PMC or ASF Board might wish/need to

be

 aware of
*   How has the community developed since the last report
*   How has the project developed since the last report.
*   How does the podling rate their own maturity.

This should be appended to the Incubator Wiki page at:

https://wiki.apache.org/incubator/September2017

Note: This is manually populated. You may need to wait a little before
this page is created from a template.

Mentors
---

Mentors should review reports for their project(s) and sign them off on
the Incubator wiki page. Signing off reports shows that you are
following the project - projects that are not signed may raise alarms
for the Incubator PMC.

Incubator PMC

Re: [VOTE] Retire HTrace from Incubation

[Masatake Iwasaki]

>   - HTrace PPMC to engage with the VOTE if possible
>   - Sean Busbey/Sheng Wu/Christopher (ctubbsii)/Billie Rinaldi step 
up and

>   propose a path for HTrace to graduate as a subproject of project X.

+1 to retire.

As Billie pointed out, htrace does not need active and growing community
since it is a library providing minimal functionality.
If we need to revive the development for users, I'm happy to work on it.

If HTrace become subproject of any other project, I'm happy to be on board.

Masatake


On 3/13/18 13:27, lewis john mcgibbney wrote:

Hi Folks,
The decision to initiate retirement is not usually an enjoyable one. It
would appear that those whose of us who have VOTE'd have made it quite
clear that retirement is the right thing to do.
According to whimsy however [0], we do not currently have enough PPMC
VOTE's to move forward with closing this VOTE with any RESULT. Actually,
not one member of the PPMC has VOTE'd yet which is pretty discouraging. At
this stage I think that we should most likely consult the general@incubator
list to seek consensus on how to proceed.
It should be noted that it is really the Mentors who have contributed to
this VOTE thread with VOTE's. I think that this speaks volumes for HTrace's
ability to sustain itself as a viable podling.

The VOTE has been open for at least 72 hours with the following outcome.

[4] +1 retire the HTrace podling
Sean Busbey
Jake Farrell
Mike Drob
Michael Stack

[0] -1 NOT NOT retire the HTrace podling (please provide justification)

I have not yet VOTE'd. I am completely aware of the fact however that this
is crunch time for HTrace. I think we either need to graduate as a
subproject OR draft a retirement resolution and propose that HTrace be
sucked into the Attic. I personally would like to see one/more of the
following taking place

- HTrace PPMC to engage with the VOTE if possible
- Sean Busbey/Sheng Wu/Christopher (ctubbsii)/Billie Rinaldi step up and
propose a path for HTrace to graduate as a subproject of project X.

I'll hang on with the VOTE for another ~72 hours and if nothing come of the
above then I will most likely VOTE to retire HTrace.

Lewis

[0] https://whimsy.apache.org/roster/ppmc/htrace

On Thu, Mar 8, 2018 at 6:49 AM, lewis john mcgibbney 
wrote:


Hi Folks,
Below I've populated our report for the month of March 2018.
https://wiki.apache.org/incubator/March2018 (also below)
I've basically initiated the Retirement sentiment which has been prevalent
within the HTrace community for some time. I would therefore like to
formally open a VOTE to retire HTrace from the Incubator.
The VOTE will be open for at least 72 hours.
[ ] +1 retire the HTrace podling
[ ] -1 NOT NOT retire the HTrace podling (please provide justification)
Lewis

===

HTrace

HTrace is a tracing framework intended for use with distributed systems
written in java.

HTrace has been incubating since 2014-11-11.

Three most important issues to address in the move towards graduation:

   1. -
   2. -
   3. -

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

The HTrace podling has seen no code activity in months. Discussion of
sending it to the Attic have been circulating for some time as well.
After literally no activity again over the last few months, it looks like
it is time to progress with sending HTrace to the Attic.

How has the community developed since the last report?

It has not.

How has the project developed since the last report?

It has not.

How would you assess the podling's maturity?
Please feel free to add your own commentary.

   [X] Initial setup
   [X] Working towards first release
   [X] Community building
   [ ] Nearing graduation
   [X] Other: No actiity.

Date of last release:

   2016-03-04

When were the last committers or PPMC members elected?

   2016-10-03

Signed-off-by:

   [ ](htrace) Jake Farrell
  Comments:
   [ ](htrace) Todd Lipcon
  Comments:
   [X](htrace) Lewis John Mcgibbney
  Comments: In our January 2018 report we mentioned that
  "HTrace is very quiet. There is opportunity to continue
  work if we engage with projects already using HTrace. If that does not
  happen, we should retire the podling." We should therefore retire the
podling.
   [ ](htrace) Andrew Purtell
  Comments:
   [ ](htrace) Billie Rinaldi
  Comments:
   [ ](htrace) Michael Stack

--
http://home.apache.org/~lewismc/
http://people.apache.org/keys/committer/lewismc