[accumulo] branch rel/1.9.3 created (now 74afbab)
This is an automated email from the ASF dual-hosted git repository. billie pushed a change to branch rel/1.9.3 in repository https://gitbox.apache.org/repos/asf/accumulo.git. at 74afbab Fix warning and simplify ScannerBaseTest No new revisions were added by this update.
[accumulo-website] branch asf-site updated: Jekyll build from master:818ebed
This is an automated email from the ASF dual-hosted git repository. billie pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/accumulo-website.git The following commit(s) were added to refs/heads/asf-site by this push: new e410cb1 Jekyll build from master:818ebed e410cb1 is described below commit e410cb1ea940ef2ad32cdcc11f930a959aa7dc08 Author: Billie Rinaldi AuthorDate: Thu Nov 14 10:30:57 2019 -0800 Jekyll build from master:818ebed Update blog to include openssl path to JAVA_OPTS in accumulo-env.sh (#208) --- blog/2019/10/15/accumulo-adlsgen2-notes.html | 15 +-- feed.xml | 39 ++-- redirects.json | 2 +- search_data.json | 2 +- tour/getting-started/index.html | 2 +- 5 files changed, 41 insertions(+), 19 deletions(-) diff --git a/blog/2019/10/15/accumulo-adlsgen2-notes.html b/blog/2019/10/15/accumulo-adlsgen2-notes.html index 2a9deb1..1c2cdd4 100644 --- a/blog/2019/10/15/accumulo-adlsgen2-notes.html +++ b/blog/2019/10/15/accumulo-adlsgen2-notes.html @@ -216,8 +216,19 @@ following versions were included with Hadoop 3.2.0. export CLASSPATH -Tried adding -Dorg.wildfly.openssl.path to JAVA_OPTS in accumulo-env.sh, but it -did not appear to work, this needs further investigation. +Include -Dorg.wildfly.openssl.path to JAVA_OPTS in accumulo-env.sh as shown below. This +java property is an optional performance enhancement for TLS. + +JAVA_OPTS=("${ACCUMULO_JAVA_OPTS[@]}" + '-XX:+UseConcMarkSweepGC' + '-XX:CMSInitiatingOccupancyFraction=75' + '-XX:+CMSClassUnloadingEnabled' + '-XX:OnOutOfMemoryError=kill -9 %p' + '-XX:-OmitStackTraceInFastThrow' + '-Djava.net.preferIPv4Stack=true' + '-Dorg.wildfly.openssl.path=/usr/lib64' + "-Daccumulo.native.lib.path=${lib}/native") + Set the following in accumulo.properties and then run accumulo init, but don’t start Accumulo. diff --git a/feed.xml b/feed.xml index 2ce1521..7cc4f1b 100644 --- a/feed.xml +++ b/feed.xml @@ -6,8 +6,8 @@ https://accumulo.apache.org/ https://accumulo.apache.org/feed.xml; rel="self" type="application/rss+xml"/> -Thu, 07 Nov 2019 13:26:50 -0500 -Thu, 07 Nov 2019 13:26:50 -0500 +Thu, 14 Nov 2019 10:30:38 -0800 +Thu, 14 Nov 2019 10:30:38 -0800 Jekyll v4.0.0 @@ -97,7 +97,7 @@ they can be removed. This strategy prevents new usages of non-public APIs while to the maven-checkstyle-plugin configuration./p -Mon, 04 Nov 2019 00:00:00 -0500 +Mon, 04 Nov 2019 00:00:00 -0800 https://accumulo.apache.org/blog/2019/11/04/checkstyle-import-control.html https://accumulo.apache.org/blog/2019/11/04/checkstyle-import-control.html @@ -178,8 +178,19 @@ following versions were included with Hadoop 3.2.0./p span class=nbexport /spanCLASSPATH /code/pre/div/div -pTried adding code class=highlighter-rouge-Dorg.wildfly.openssl.path/code to code class=highlighter-rougeJAVA_OPTS/code in code class=highlighter-rougeaccumulo-env.sh/code, but it -did not appear to work, this needs further investigation./p +pInclude code class=highlighter-rouge-Dorg.wildfly.openssl.path/code to code class=highlighter-rougeJAVA_OPTS/code in code class=highlighter-rougeaccumulo-env.sh/code as shown below. This +java property is an optional performance enhancement for TLS./p + +div class=language-bash highlighter-rougediv class=highlightpre class=highlightcodespan class=nvJAVA_OPTS/spanspan class=o=(/spanspan class=s2/spanspan class=k${/spanspan class=nvACCUMULO_JAVA_OPTS/spanspan class=p[@]/spanspan class=k [...] + span class=s1'-XX:+UseConcMarkSweepGC'/span + span class=s1'-XX:CMSInitiatingOccupancyFraction=75'/span + span class=s1'-XX:+CMSClassUnloadingEnabled'/span + span class=s1'-XX:OnOutOfMemoryError=kill -9 %p'/span + span class=s1'-XX:-OmitStackTraceInFastThrow'/span + span class=s1'-Djava.net.preferIPv4Stack=true'/span + span class=s1'-Dorg.wildfly.openssl.path=/usr/lib64'/span + span class=s2-Daccumulo.native.lib.path=/spanspan class=k${/spanspan class=nvlib/spanspan class=k}/spanspan class=s2/native/spanspan class=o)/span +/code/pre/div/div pSet the following in code class=highlighter-rougeaccumulo.properties/code and then run code class=highlighter-rougeaccumulo init/code, but don’t start Accumulo./p @@ -229,7 +240,7 @@ we are able to successfully run the continuos ingest test. Going forward, we’l with ADLS Gen2 and add/update blog as we come along./p -Tue, 15 Oct 2019 00:00:00 -0400 +Tue, 15 Oct 2019 00:00:00 -0700 https://accumulo.apache.org/blog/2019/10/15/accumulo-adlsgen2-notes.html https://accumulo.apache.org/blog/2019/10/15/accumulo-adlsgen2-notes.html @@ -553,7 +564,7 @@ to Accumulo to make working with EC a bit easier. Ou
[accumulo-testing] branch master updated: Fix #37 Restart GC during agitation (#117)
This is an automated email from the ASF dual-hosted git repository. billie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git The following commit(s) were added to refs/heads/master by this push: new 98f1d34 Fix #37 Restart GC during agitation (#117) 98f1d34 is described below commit 98f1d34315b1549004620f6e0a882089b8bc6bb2 Author: billierinaldi AuthorDate: Tue Nov 12 13:52:39 2019 -0800 Fix #37 Restart GC during agitation (#117) --- libexec/master-agitator.pl | 21 + 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/libexec/master-agitator.pl b/libexec/master-agitator.pl index 949cfbb..e90c70d 100755 --- a/libexec/master-agitator.pl +++ b/libexec/master-agitator.pl @@ -51,6 +51,14 @@ for $master (@mastersRaw){ while(1){ sleep($sleep1 * 60); $t = strftime "%Y%m%d %H:%M:%S", localtime; + + $gcfile = ''; + if (-e "$accumuloConfDir/gc") { + $gcfile = 'gc'; + } else { + $gcfile = 'masters'; + } + if(rand(1) < .5){ $masterNodeToWack = $masters[int(rand(scalar(@masters)))]; print STDERR "$t Killing master on $masterNodeToWack\n"; @@ -63,14 +71,7 @@ while(1){ print "$t $cmd\n"; system($cmd); - $file = ''; - if (-e "$accumuloConfDir/gc") { - $file = 'gc'; - } else { - $file = 'masters'; - } - - $cmd = "pssh -h $accumuloConfDir/$file \"pkill -f '[ ]org.apache.accumulo.start.*gc'\" < /dev/null"; + $cmd = "pssh -h $accumuloConfDir/$gcfile \"pkill -f '[ ]org.apache.accumulo.start.*gc'\" < /dev/null"; print "$t $cmd\n"; system($cmd); } @@ -82,6 +83,10 @@ while(1){ $cmd = "pssh -h $accumuloConfDir/masters \"$accumuloHome/bin/accumulo-service master start\" < /dev/null"; print "$t $cmd\n"; system($cmd); + + $cmd = "pssh -h $accumuloConfDir/$gcfile \"$accumuloHome/bin/accumulo-service gc start\" < /dev/null"; + print "$t $cmd\n"; + system($cmd); }
[accumulo-website] branch asf-site updated: Jekyll build from master:6499f63
This is an automated email from the ASF dual-hosted git repository. billie pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/accumulo-website.git The following commit(s) were added to refs/heads/asf-site by this push: new ab15ce3 Jekyll build from master:6499f63 ab15ce3 is described below commit ab15ce376cb3f7ae60dc168acf3c7173986377a7 Author: Billie Rinaldi AuthorDate: Tue Oct 22 14:33:08 2019 -0700 Jekyll build from master:6499f63 Update billie's org --- docs/2.x/development/iterators.html | 18 +++--- feed.xml| 24 people/index.html | 2 +- redirects.json | 2 +- search_data.json| 2 +- 5 files changed, 18 insertions(+), 30 deletions(-) diff --git a/docs/2.x/development/iterators.html b/docs/2.x/development/iterators.html index 496c0e1..7a23ec6 100644 --- a/docs/2.x/development/iterators.html +++ b/docs/2.x/development/iterators.html @@ -431,24 +431,13 @@ merge multiple Iterators into a single view. In this sense, a collection of Iter a tree-structure than a list, but there is always a sense of a flow of Key-Value pairs through some Iterators. Iterators are not designed to act as triggers nor are they designed to operate outside of the purview of a single table. -Understanding how TabletServers invoke the methods on a https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0.0/org/apache/accumulo/core/iterators/SortedKeyValueIterator.html;>SortedKeyValueIterator can be obtuse as the actual code is -buried within the implementation of the TabletServer; however, it is generally unnecessary to have a strong -understanding of this as the interface provides clear definitions about what each method should take. This -chapter aims to provide a more detailed description of how Iterators are invoked, some best practices and some common +This guide aims to provide a more detailed description of how Iterators are invoked, some best practices and some common pitfalls. Instantiation To invoke an Accumulo Iterator inside of the TabletServer, the Iterator class must be on the classpath of every -TabletServer. For production environments, it is common to place a JAR file which contains the Iterator in -lib/. In development environments, it is convenient to instead place the JAR file in lib/ext/ as JAR files -in this directory are dynamically reloaded by the TabletServers alleviating the need to restart Accumulo while -testing an Iterator. Advanced classloader features which enable other types of filesystems and per-table classpath -configurations (as opposed to process-wide classpaths). These features are not covered here, but elsewhere in the user -manual. - -Accumulo references the Iterator class by name and uses Java reflection to instantiate the Iterator. This means that -Iterators must have a public no-args constructor. +TabletServer. It is common to place a JAR file which contains the Iterator in lib/. Accumulo references the Iterator class by name and uses Java reflection to instantiate the Iterator. This means that Iterators must have a public no-args constructor. Interface @@ -490,8 +479,7 @@ optimize itself. seek -The seek method is likely the most confusing method on the Iterator interface. The purpose of this method is to -advance the stream of Key-Value pairs to a certain point in the iteration (the Accumulo table). It is common that before +The purpose of the seek method is to advance the stream of Key-Value pairs to a certain point in the iteration (the Accumulo table). It is common that before the implementation of this method returns some additional processing is performed which may further advance the current position past the startKey of the https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0.0/org/apache/accumulo/core/data/Range.html;>Range. This, however, is dependent on the functionality the iterator provides. For example, a filtering iterator would consume a number Key-Value pairs which do not meets its criteria before seek diff --git a/feed.xml b/feed.xml index 2353fcb..1460354 100644 --- a/feed.xml +++ b/feed.xml @@ -6,8 +6,8 @@ https://accumulo.apache.org/ https://accumulo.apache.org/feed.xml; rel="self" type="application/rss+xml"/> -Thu, 17 Oct 2019 11:08:16 -0400 -Thu, 17 Oct 2019 11:08:16 -0400 +Tue, 22 Oct 2019 14:32:52 -0700 +Tue, 22 Oct 2019 14:32:52 -0700 Jekyll v4.0.0 @@ -134,7 +134,7 @@ we are able to successfully run the continuos ingest test. Going forward, we’l with ADLS Gen2 and add/update blog as we come along./p -Tue, 15 Oct 2019 00:00:00 -0400 +Tue, 15 Oct 2019 00:00:00 -0700 https://accumulo.apache.org/blog/2019/10/15/accumulo-adlsgen2-notes.html https://accumulo.apache.org/blog/2019/10/15/accumulo-adlsgen2-n
[accumulo-website] branch master updated: Update billie's org
This is an automated email from the ASF dual-hosted git repository. billie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo-website.git The following commit(s) were added to refs/heads/master by this push: new 6499f63 Update billie's org 6499f63 is described below commit 6499f63aeb3b6d5851d6fc3fba09e3f4c7fb7077 Author: Billie Rinaldi AuthorDate: Tue Oct 22 13:48:21 2019 -0700 Update billie's org --- pages/people.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/people.md b/pages/people.md index 8d6d1cd..570bc3f 100644 --- a/pages/people.md +++ b/pages/people.md @@ -14,7 +14,7 @@ permalink: /people/ | afuchs| Adam Fuchs| [sqrrl][SQRRL] | [ET][ET] | | alerman | Adam Lerman | [Applied Technology Group][ATG]| [ET][ET] | | bhavanki | Bill Havanki | [Cloudera][CLOUDERA] | [ET][ET] | -| billie| Billie Rinaldi| [Hortonworks][HORTONWORKS] | [ET][ET] | +| billie| Billie Rinaldi| [Microsoft][MICROSOFT] | [ET][ET] | | bimargulies | Benson Margulies | [Basis Technology Corp.][BASISTECH]| [ET][ET] | | brianloss | Brian Loss| [Praxis Engineering][PRAXIS] | [ET][ET] | | busbey| Sean Busbey | [Cloudera][CLOUDERA] | [CT][CT] |
[2/2] accumulo-website git commit: Jekyll build from master:7ccd338
Jekyll build from master:7ccd338 add apachecon link to main page Project: http://git-wip-us.apache.org/repos/asf/accumulo-website/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo-website/commit/3fa0dba4 Tree: http://git-wip-us.apache.org/repos/asf/accumulo-website/tree/3fa0dba4 Diff: http://git-wip-us.apache.org/repos/asf/accumulo-website/diff/3fa0dba4 Branch: refs/heads/asf-site Commit: 3fa0dba4d9fb3977a5df5dffd7a8513e7981bb6f Parents: b6cdc11 Author: Billie Rinaldi <bil...@apache.org> Authored: Thu Jan 26 09:10:39 2017 -0800 Committer: Billie Rinaldi <bil...@apache.org> Committed: Thu Jan 26 09:10:39 2017 -0800 -- css/accumulo.css | 2 ++ feed.xml | 4 ++-- index.html | 1 + 3 files changed, 5 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/3fa0dba4/css/accumulo.css -- diff --git a/css/accumulo.css b/css/accumulo.css index 7784523..15f993b 100644 --- a/css/accumulo.css +++ b/css/accumulo.css @@ -119,6 +119,8 @@ footer > p { width: 60%; margin-left: auto; margin-right: auto; } #accumulo-summit-logo img { height: 40px; } +#apachecon img { padding-bottom: 10px; } + .header-link { padding-left: 5px; opacity: 0; \-webkit-transition: opacity 0.2s ease-in-out 0.1s; \-moz-transition: opacity 0.2s ease-in-out 0.1s; \-ms-transition: opacity 0.2s ease-in-out 0.1s; } h2:hover .header-link, h3:hover .header-link, h4:hover .header-link, h5:hover .header-link, h6:hover .header-link { opacity: 1; } http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/3fa0dba4/feed.xml -- diff --git a/feed.xml b/feed.xml index e87b5d5..8a0fb37 100644 --- a/feed.xml +++ b/feed.xml @@ -6,8 +6,8 @@ https://accumulo.apache.org/ https://accumulo.apache.org/feed.xml; rel="self" type="application/rss+xml"/> -Mon, 09 Jan 2017 13:46:46 -0800 -Mon, 09 Jan 2017 13:46:46 -0800 +Thu, 26 Jan 2017 09:10:31 -0800 +Thu, 26 Jan 2017 09:10:31 -0800 Jekyll v3.3.1 http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/3fa0dba4/index.html -- diff --git a/index.html b/index.html index 880b425..a3ef129 100644 --- a/index.html +++ b/index.html @@ -222,6 +222,7 @@ +http://www.apache.org/events/current-event.html;>http://www.apache.org/events/current-event-234x60.png; /> http://accumulosummit.com/;>
[1/2] accumulo-website git commit: add apachecon link to main page
Repository: accumulo-website Updated Branches: refs/heads/asf-site b6cdc1109 -> 3fa0dba4d refs/heads/master 76f2d8c7e -> 7ccd338ab add apachecon link to main page Project: http://git-wip-us.apache.org/repos/asf/accumulo-website/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo-website/commit/7ccd338a Tree: http://git-wip-us.apache.org/repos/asf/accumulo-website/tree/7ccd338a Diff: http://git-wip-us.apache.org/repos/asf/accumulo-website/diff/7ccd338a Branch: refs/heads/master Commit: 7ccd338ab94099f3e195b5c9b2a50b80e688e314 Parents: 76f2d8c Author: Billie Rinaldi <bil...@apache.org> Authored: Thu Jan 26 08:46:53 2017 -0800 Committer: Billie Rinaldi <bil...@apache.org> Committed: Thu Jan 26 09:08:04 2017 -0800 -- css/accumulo.scss | 4 index.md | 1 + 2 files changed, 5 insertions(+) -- http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/7ccd338a/css/accumulo.scss -- diff --git a/css/accumulo.scss b/css/accumulo.scss index 3537ea4..a9126d9 100644 --- a/css/accumulo.scss +++ b/css/accumulo.scss @@ -135,6 +135,10 @@ footer > p { height: 40px } +#apachecon img { + padding-bottom: 10px; +} + .header-link { padding-left: 5px; opacity: 0; http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/7ccd338a/index.md -- diff --git a/index.md b/index.md index bd045e8..2f8cb99 100644 --- a/index.md +++ b/index.md @@ -35,6 +35,7 @@ legal_notice: Apache Accumulo, Accumulo, Apache Hadoop, Apache Thrift, Apache, t {% capture social-include %}{% include social.md %}{% endcapture %}{{ social-include | markdownify }} +http://www.apache.org/events/current-event.html;>http://www.apache.org/events/current-event-234x60.png"/> http://accumulosummit.com/;>
[2/2] accumulo-website git commit: Jekyll build from master:76f2d8c
Jekyll build from master:76f2d8c ACCUMULO-4564 Replace with on the website Project: http://git-wip-us.apache.org/repos/asf/accumulo-website/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo-website/commit/b6cdc110 Tree: http://git-wip-us.apache.org/repos/asf/accumulo-website/tree/b6cdc110 Diff: http://git-wip-us.apache.org/repos/asf/accumulo-website/diff/b6cdc110 Branch: refs/heads/asf-site Commit: b6cdc11091e03f3476c99ac259f1cd792527016f Parents: a102c91 Author: Billie Rinaldi <bil...@apache.org> Authored: Mon Jan 9 13:46:54 2017 -0800 Committer: Billie Rinaldi <bil...@apache.org> Committed: Mon Jan 9 13:46:54 2017 -0800 -- contributor/source.html | 2 +- downloads/index.html| 6 +++--- features/index.html | 2 +- feed.xml| 24 index.html | 2 +- 5 files changed, 18 insertions(+), 18 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/b6cdc110/contributor/source.html -- diff --git a/contributor/source.html b/contributor/source.html index 5531562..7fd38d5 100644 --- a/contributor/source.html +++ b/contributor/source.html @@ -153,7 +153,7 @@ Apache Accumulo -Apache Accumulo⢠source code is maintained using https://git-scm.com;>Git version control +Apache Accumulo® source code is maintained using https://git-scm.com;>Git version control (https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=summary;>browse|checkout). It builds with https://maven.apache.org;>Apache Maven. Instructions for configuring git are https://git-wip-us.apache.org;>here. http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/b6cdc110/downloads/index.html -- diff --git a/downloads/index.html b/downloads/index.html index 12c7c38..0d65648 100644 --- a/downloads/index.html +++ b/downloads/index.html @@ -224,7 +224,7 @@ $(function() { $.getJSON(mirrorURL + "?as_json", mirrorsCallback); }); 1.8.0 latest -The most recent Apache Accumulo⢠release is version 1.8.0. See the release notes and https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12329879; title="1.8.0 CHANGES">CHANGES. +The most recent Apache Accumulo® release is version 1.8.0. See the release notes and https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12329879; title="1.8.0 CHANGES">CHANGES. For convenience, https://www.apache.org/dist/accumulo/1.8.0/MD5SUM; title="1.8.0 MD5 file hashes">MD5 and https://www.apache.org/dist/accumulo/1.8.0/SHA1SUM; title="1.8.0 SHA1 file hashes">SHA1 hashes are also available. @@ -253,7 +253,7 @@ $(function() { $.getJSON(mirrorURL + "?as_json", mirrorsCallback); }); 1.7.2 -The most recent 1.7.x release of Apache Accumulo⢠is version 1.7.2. See the release notes and https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12333776; title="1.7.2 CHANGES">CHANGES. +The most recent 1.7.x release of Apache Accumulo® is version 1.7.2. See the release notes and https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12333776; title="1.7.2 CHANGES">CHANGES. For convenience, https://www.apache.org/dist/accumulo/1.7.2/MD5SUM; title="1.7.2 MD5 file hashes">MD5 and https://www.apache.org/dist/accumulo/1.7.2/SHA1SUM; title="1.7.2 SHA1 file hashes">SHA1 hashes are also available. @@ -282,7 +282,7 @@ $(function() { $.getJSON(mirrorURL + "?as_json", mirrorsCallback); }); 1.6.6 -The most recent 1.6.x release of Apache Accumulo⢠is version 1.6.6. See the release notes and https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12334846; title="1.6.6 CHANGES">CHANGES. +The most recent 1.6.x release of Apache Accumulo® is version 1.6.6. See the release notes and https://issues.apache.org/jira/browse/ACCUMULO/fixforversion/12334846; title="1.6.6 CHANGES">CHANGES. For convenience, https://www.apache.org/dist/accumulo/1.6.6/MD5SUM; title="1.6.6 MD5 file hashes">MD5 and https://www.apache.org/dist/accumulo/1.6.6/SHA1SUM; title="1.6.6 SHA1 file hashes">SHA1 hashes are also available. http://git-wip-us.apache.org/repos/asf/accumulo-website/blob/b6cdc110/features/index.html -- diff --git a/features/index.html b/features/index.html index 155e364..c41cf56 100644 --- a/features/index.html +++ b/features/index.html @@ -584,7 +584,7 @@ beginning and end of the range are split, compacted, and then merged. -The following is a screenshot of the Apache Accumulo monitor overview web page. Accumul
svn commit: r11817 - /release/accumulo/1.5.4/
Author: billie Date: Mon Jan 4 20:33:15 2016 New Revision: 11817 Log: ACCUMULO-4096 archive 1.5.4 release Removed: release/accumulo/1.5.4/
svn commit: r977044 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/ 1.6/apidocs/ 1.7/apidocs/
Author: billie Date: Mon Jan 4 20:27:26 2016 New Revision: 977044 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r977043, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r977043, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r977043, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r977043, websites/production/accumulo/content/1.6/apidocs/ websites/production/accumulo/content/1.7/apidocs/ - copied from r977043, websites/production/accumulo/content/1.7/apidocs/
svn commit: r1722957 - in /accumulo/site/trunk: content/contrib.mdtext content/downloads/index.mdtext content/old_documentation.mdtext templates/nav.html templates/skeleton.html
Author: billie Date: Mon Jan 4 20:08:59 2016 New Revision: 1722957 URL: http://svn.apache.org/viewvc?rev=1722957=rev Log: ACCUMULO-4096 archive references to 1.5 on website Modified: accumulo/site/trunk/content/contrib.mdtext accumulo/site/trunk/content/downloads/index.mdtext accumulo/site/trunk/content/old_documentation.mdtext accumulo/site/trunk/templates/nav.html accumulo/site/trunk/templates/skeleton.html Modified: accumulo/site/trunk/content/contrib.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/contrib.mdtext?rev=1722957=1722956=1722957=diff == --- accumulo/site/trunk/content/contrib.mdtext (original) +++ accumulo/site/trunk/content/contrib.mdtext Mon Jan 4 20:08:59 2016 @@ -45,5 +45,5 @@ All contributions to the various Apache ## Adding a new Contrib Project Proposals for new contrib projects should be sent to the [Accumulo mailing list](mailing_list.html) for [developers](mailto:d...@accumulo.apache.org?subject=[Accumulo Contrib Proposal]). -If an example application only makes use of a single Accumulo feature, it is probably better off as an Accumulo version-specific example. You can see several of these demonstrative applications in the [simple example codebase](https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=tree;f=examples/simple;) and the related published documentation for versions [1.5](1.5/examples/) and [1.6](1.6/examples/). +If an example application only makes use of a single Accumulo feature, it is probably better off as an Accumulo version-specific example. You can see several of these demonstrative applications in the [simple example codebase](https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=tree;f=examples/simple;) and the related published documentation for versions [1.7](1.7/examples/) and [1.6](1.6/examples/). Modified: accumulo/site/trunk/content/downloads/index.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/downloads/index.mdtext?rev=1722957=1722956=1722957=diff == --- accumulo/site/trunk/content/downloads/index.mdtext (original) +++ accumulo/site/trunk/content/downloads/index.mdtext Mon Jan 4 20:08:59 2016 @@ -154,57 +154,6 @@ For convenience, [MD5][MD5SUM_16] and [S * [examples][EXAMPLES_16] * Javadoc -### 1.5.4 - -The most recent Apache Accumulo 1.5 release is version 1.5.4. See the [release notes][REL_NOTES_15] and [CHANGES][CHANGES_15]. - -For convenience, [MD5][MD5SUM_15] and [SHA1][SHA1SUM_15] hashes are also available. - - - -Generic Binaries -https://www.apache.org/dyn/closer.lua/accumulo/1.5.4/accumulo-1.5.4-bin.tar.gz; link-suffix="/accumulo/1.5.4/accumulo-1.5.4-bin.tar.gz" class="download_external" id="/downloads/accumulo-1.5.4-bin.tar.gz">accumulo-1.5.4-bin.tar.gz -https://www.apache.org/dist/accumulo/1.5.4/accumulo-1.5.4-bin.tar.gz.asc;>ASC - - -Platform Binaries -https://www.apache.org/dyn/closer.lua/accumulo/1.5.4/accumulo-1.5.4-bin.rpm; link-suffix="/accumulo/1.5.4/accumulo-1.5.4-bin.rpm" class="download_external" id="/downloads/accumulo-1.5.4-bin.rpm">accumulo-1.5.4-bin.rpm -https://www.apache.org/dist/accumulo/1.5.4/accumulo-1.5.4-bin.rpm.asc;>ASC - - -https://www.apache.org/dyn/closer.lua/accumulo/1.5.4/accumulo-1.5.4-native.rpm; link-suffix="/accumulo/1.5.4/accumulo-1.5.4-native.rpm" class="download_external" id="/downloads/accumulo-1.5.4-native.rpm">accumulo-1.5.4-native.rpm -https://www.apache.org/dist/accumulo/1.5.4/accumulo-1.5.4-native.rpm.asc;>ASC - - -https://www.apache.org/dyn/closer.lua/accumulo/1.5.4/accumulo-1.5.4-test.rpm; link-suffix="/accumulo/1.5.4/accumulo-1.5.4-test.rpm" class="download_external" id="/downloads/accumulo-1.5.4-test.rpm">accumulo-1.5.4-test.rpm -https://www.apache.org/dist/accumulo/1.5.4/accumulo-1.5.4-test.rpm.asc;>ASC - - -https://www.apache.org/dyn/closer.lua/accumulo/1.5.4/accumulo-1.5.4-bin.deb; link-suffix="/accumulo/1.5.4/accumulo-1.5.4-bin.deb" class="download_external" id="/downloads/accumulo-1.5.4-bin.deb">accumulo-1.5.4-bin.deb -https://www.apache.org/dist/accumulo/1.5.4/accumulo-1.5.4-bin.deb.asc;>ASC - - -https://www.apache.org/dyn/closer.lua/accumulo/1.5.4/accumulo-1.5.4-native.deb; link-suffix="/accumulo/1.5.4/accumulo-1.5.4-native.deb" class="download_external" id="/downloads/accumulo-1.5.4-native.deb">accumulo-1.5.4-native.deb -https://www.apache.org/dist/accumulo/1.5.4/accumulo-1.5.4-native.deb.asc;>ASC - - -https://www.apache.org/dyn/closer.lua/accumulo/1.5.4/accumulo-1.5.4-test.deb; link-suffix="/accumulo/1.5.4/accumulo-1.5.4-test.deb" class="download_external" id="/downloads/accumul
svn commit: r1706848 - /accumulo/site/trunk/content/releasing.mdtext
Author: billie Date: Mon Oct 5 15:03:15 2015 New Revision: 1706848 URL: http://svn.apache.org/viewvc?rev=1706848=rev Log: add doap to release procedures Modified: accumulo/site/trunk/content/releasing.mdtext Modified: accumulo/site/trunk/content/releasing.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/releasing.mdtext?rev=1706848=1706847=1706848=diff == --- accumulo/site/trunk/content/releasing.mdtext (original) +++ accumulo/site/trunk/content/releasing.mdtext Mon Oct 5 15:03:15 2015 @@ -110,6 +110,7 @@ After a successful vote, this website ne * Create release notes (ensure notes contain link to JIRA changes for that version) * Remove previous bug-fix release (if applicable) * Update examples README files + * Update doap_Accumulo.rdf ### Javadocs
svn commit: r957385 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/ 1.6/apidocs/ 1.7/apidocs/
Author: billie Date: Tue Jul 7 17:20:40 2015 New Revision: 957385 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r957384, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r957384, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r957384, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r957384, websites/production/accumulo/content/1.6/apidocs/ websites/production/accumulo/content/1.7/apidocs/ - copied from r957384, websites/production/accumulo/content/1.7/apidocs/
svn commit: r1689714 - /accumulo/site/trunk/content/doap_Accumulo.rdf
Author: billie Date: Tue Jul 7 17:20:03 2015 New Revision: 1689714 URL: http://svn.apache.org/r1689714 Log: add 1.7.0 to doap Modified: accumulo/site/trunk/content/doap_Accumulo.rdf Modified: accumulo/site/trunk/content/doap_Accumulo.rdf URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/doap_Accumulo.rdf?rev=1689714r1=1689713r2=1689714view=diff == --- accumulo/site/trunk/content/doap_Accumulo.rdf (original) +++ accumulo/site/trunk/content/doap_Accumulo.rdf Tue Jul 7 17:20:03 2015 @@ -36,6 +36,13 @@ release Version nameApache Accumulo/name +created2015-05-19/created +revision1.7.0/revision + /Version +/release +release + Version +nameApache Accumulo/name created2015-07-04/created revision1.6.3/revision /Version
accumulo git commit: ACCUMULO-3890 switch cached providers map to a ConcurrentHashMap
Repository: accumulo Updated Branches: refs/heads/1.6 a2e131bda - df67ad236 ACCUMULO-3890 switch cached providers map to a ConcurrentHashMap Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/df67ad23 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/df67ad23 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/df67ad23 Branch: refs/heads/1.6 Commit: df67ad236450526086d31f47bdece65ae2c2f023 Parents: a2e131b Author: Billie Rinaldi bil...@apache.org Authored: Thu Jun 11 10:33:14 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Jun 11 11:54:44 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +--- 1 file changed, 11 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/df67ad23/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java index 9af3e00..388f3ca 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java @@ -21,9 +21,8 @@ import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import java.util.ArrayList; import java.util.Collections; -import java.util.HashMap; import java.util.List; -import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; import org.apache.accumulo.core.util.CachedConfiguration; import org.apache.hadoop.conf.Configuration; @@ -65,7 +64,7 @@ public class CredentialProviderFactoryShim { private static Boolean hadoopClassesAvailable = null; // access to cachedProviders should be synchronized when necessary (for example see getCredentialProviders) - private static final MapString,ListObject cachedProviders = new HashMapString,ListObject(); + private static final ConcurrentHashMapString,ListObject cachedProviders = new ConcurrentHashMapString,ListObject(); /** * Determine if we can load the necessary CredentialProvider classes. Only loaded the first time, so subsequent invocations of this method should return fast. @@ -210,8 +209,9 @@ public class CredentialProviderFactoryShim { return null; } -if (cachedProviders.containsKey(path)) { - return cachedProviders.get(path); +ListObject providersList = cachedProviders.get(path); +if (providersList != null) { + return providersList; } // Call CredentialProviderFactory.getProviders(Configuration) @@ -231,15 +231,13 @@ public class CredentialProviderFactoryShim { // Cast the Object to ListObject (actually ListCredentialProvider) try { - ListObject providersList = (ListObject) providersObj; - synchronized (cachedProviders) { -if (cachedProviders.containsKey(path)) { - return cachedProviders.get(path); -} else { - cachedProviders.put(path, providersList); -} + providersList = (ListObject) providersObj; + ListObject previousValue = cachedProviders.putIfAbsent(path, providersList); + if (previousValue != null) { +return previousValue; + } else { +return providersList; } - return providersList; } catch (ClassCastException e) { log.error(Expected a List from {} method, HADOOP_CRED_PROVIDER_FACTORY_GET_PROVIDERS_METHOD_NAME, e); return null;
[1/2] accumulo git commit: ACCUMULO-3890 switch cached providers map to a ConcurrentHashMap
Repository: accumulo Updated Branches: refs/heads/1.7 724bfb312 - 5312fac1c ACCUMULO-3890 switch cached providers map to a ConcurrentHashMap Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/df67ad23 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/df67ad23 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/df67ad23 Branch: refs/heads/1.7 Commit: df67ad236450526086d31f47bdece65ae2c2f023 Parents: a2e131b Author: Billie Rinaldi bil...@apache.org Authored: Thu Jun 11 10:33:14 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Jun 11 11:54:44 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +--- 1 file changed, 11 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/df67ad23/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java index 9af3e00..388f3ca 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java @@ -21,9 +21,8 @@ import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import java.util.ArrayList; import java.util.Collections; -import java.util.HashMap; import java.util.List; -import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; import org.apache.accumulo.core.util.CachedConfiguration; import org.apache.hadoop.conf.Configuration; @@ -65,7 +64,7 @@ public class CredentialProviderFactoryShim { private static Boolean hadoopClassesAvailable = null; // access to cachedProviders should be synchronized when necessary (for example see getCredentialProviders) - private static final MapString,ListObject cachedProviders = new HashMapString,ListObject(); + private static final ConcurrentHashMapString,ListObject cachedProviders = new ConcurrentHashMapString,ListObject(); /** * Determine if we can load the necessary CredentialProvider classes. Only loaded the first time, so subsequent invocations of this method should return fast. @@ -210,8 +209,9 @@ public class CredentialProviderFactoryShim { return null; } -if (cachedProviders.containsKey(path)) { - return cachedProviders.get(path); +ListObject providersList = cachedProviders.get(path); +if (providersList != null) { + return providersList; } // Call CredentialProviderFactory.getProviders(Configuration) @@ -231,15 +231,13 @@ public class CredentialProviderFactoryShim { // Cast the Object to ListObject (actually ListCredentialProvider) try { - ListObject providersList = (ListObject) providersObj; - synchronized (cachedProviders) { -if (cachedProviders.containsKey(path)) { - return cachedProviders.get(path); -} else { - cachedProviders.put(path, providersList); -} + providersList = (ListObject) providersObj; + ListObject previousValue = cachedProviders.putIfAbsent(path, providersList); + if (previousValue != null) { +return previousValue; + } else { +return providersList; } - return providersList; } catch (ClassCastException e) { log.error(Expected a List from {} method, HADOOP_CRED_PROVIDER_FACTORY_GET_PROVIDERS_METHOD_NAME, e); return null;
[2/2] accumulo git commit: Merge branch '1.6' into 1.7
Merge branch '1.6' into 1.7 Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/5312fac1 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/5312fac1 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/5312fac1 Branch: refs/heads/1.7 Commit: 5312fac1ce3cede1f6e4464f8b0ebc04cb2f355d Parents: 724bfb3 df67ad2 Author: Billie Rinaldi bil...@apache.org Authored: Thu Jun 11 13:29:00 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Jun 11 13:29:00 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +--- 1 file changed, 11 insertions(+), 13 deletions(-) --
[1/3] accumulo git commit: ACCUMULO-3890 switch cached providers map to a ConcurrentHashMap
Repository: accumulo Updated Branches: refs/heads/master 3fe4f690b - 455b99aa0 ACCUMULO-3890 switch cached providers map to a ConcurrentHashMap Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/df67ad23 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/df67ad23 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/df67ad23 Branch: refs/heads/master Commit: df67ad236450526086d31f47bdece65ae2c2f023 Parents: a2e131b Author: Billie Rinaldi bil...@apache.org Authored: Thu Jun 11 10:33:14 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Jun 11 11:54:44 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +--- 1 file changed, 11 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/df67ad23/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java index 9af3e00..388f3ca 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java @@ -21,9 +21,8 @@ import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import java.util.ArrayList; import java.util.Collections; -import java.util.HashMap; import java.util.List; -import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; import org.apache.accumulo.core.util.CachedConfiguration; import org.apache.hadoop.conf.Configuration; @@ -65,7 +64,7 @@ public class CredentialProviderFactoryShim { private static Boolean hadoopClassesAvailable = null; // access to cachedProviders should be synchronized when necessary (for example see getCredentialProviders) - private static final MapString,ListObject cachedProviders = new HashMapString,ListObject(); + private static final ConcurrentHashMapString,ListObject cachedProviders = new ConcurrentHashMapString,ListObject(); /** * Determine if we can load the necessary CredentialProvider classes. Only loaded the first time, so subsequent invocations of this method should return fast. @@ -210,8 +209,9 @@ public class CredentialProviderFactoryShim { return null; } -if (cachedProviders.containsKey(path)) { - return cachedProviders.get(path); +ListObject providersList = cachedProviders.get(path); +if (providersList != null) { + return providersList; } // Call CredentialProviderFactory.getProviders(Configuration) @@ -231,15 +231,13 @@ public class CredentialProviderFactoryShim { // Cast the Object to ListObject (actually ListCredentialProvider) try { - ListObject providersList = (ListObject) providersObj; - synchronized (cachedProviders) { -if (cachedProviders.containsKey(path)) { - return cachedProviders.get(path); -} else { - cachedProviders.put(path, providersList); -} + providersList = (ListObject) providersObj; + ListObject previousValue = cachedProviders.putIfAbsent(path, providersList); + if (previousValue != null) { +return previousValue; + } else { +return providersList; } - return providersList; } catch (ClassCastException e) { log.error(Expected a List from {} method, HADOOP_CRED_PROVIDER_FACTORY_GET_PROVIDERS_METHOD_NAME, e); return null;
[3/3] accumulo git commit: Merge branch '1.7'
Merge branch '1.7' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/455b99aa Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/455b99aa Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/455b99aa Branch: refs/heads/master Commit: 455b99aa050c00e28db2856fea42aef6888168ee Parents: 3fe4f69 5312fac Author: Billie Rinaldi bil...@apache.org Authored: Thu Jun 11 13:30:00 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Jun 11 13:30:00 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +--- 1 file changed, 11 insertions(+), 13 deletions(-) --
[2/3] accumulo git commit: Merge branch '1.6' into 1.7
Merge branch '1.6' into 1.7 Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/5312fac1 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/5312fac1 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/5312fac1 Branch: refs/heads/master Commit: 5312fac1ce3cede1f6e4464f8b0ebc04cb2f355d Parents: 724bfb3 df67ad2 Author: Billie Rinaldi bil...@apache.org Authored: Thu Jun 11 13:29:00 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Jun 11 13:29:00 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +--- 1 file changed, 11 insertions(+), 13 deletions(-) --
accumulo git commit: ACCUMULO-3890 cache CredentialProvider objects to prevent unneeded NN ops
Repository: accumulo Updated Branches: refs/heads/1.6 c6d1ce598 - a2e131bda ACCUMULO-3890 cache CredentialProvider objects to prevent unneeded NN ops Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/a2e131bd Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/a2e131bd Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/a2e131bd Branch: refs/heads/1.6 Commit: a2e131bda969061b93e8639e11c3445d450e5bc3 Parents: c6d1ce5 Author: Billie Rinaldi bil...@apache.org Authored: Wed Jun 10 14:25:13 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Wed Jun 10 18:04:44 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +++- 1 file changed, 23 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/a2e131bd/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java index 3c3c051..9af3e00 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java @@ -21,7 +21,9 @@ import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import java.util.ArrayList; import java.util.Collections; +import java.util.HashMap; import java.util.List; +import java.util.Map; import org.apache.accumulo.core.util.CachedConfiguration; import org.apache.hadoop.conf.Configuration; @@ -62,6 +64,9 @@ public class CredentialProviderFactoryShim { private static Method flushMethod = null; private static Boolean hadoopClassesAvailable = null; + // access to cachedProviders should be synchronized when necessary (for example see getCredentialProviders) + private static final MapString,ListObject cachedProviders = new HashMapString,ListObject(); + /** * Determine if we can load the necessary CredentialProvider classes. Only loaded the first time, so subsequent invocations of this method should return fast. * @@ -200,6 +205,15 @@ public class CredentialProviderFactoryShim { */ @SuppressWarnings(unchecked) protected static ListObject getCredentialProviders(Configuration conf) { +String path = conf.get(CREDENTIAL_PROVIDER_PATH); +if (path == null || path.isEmpty()) { + return null; +} + +if (cachedProviders.containsKey(path)) { + return cachedProviders.get(path); +} + // Call CredentialProviderFactory.getProviders(Configuration) Object providersObj = null; try { @@ -217,7 +231,15 @@ public class CredentialProviderFactoryShim { // Cast the Object to ListObject (actually ListCredentialProvider) try { - return (ListObject) providersObj; + ListObject providersList = (ListObject) providersObj; + synchronized (cachedProviders) { +if (cachedProviders.containsKey(path)) { + return cachedProviders.get(path); +} else { + cachedProviders.put(path, providersList); +} + } + return providersList; } catch (ClassCastException e) { log.error(Expected a List from {} method, HADOOP_CRED_PROVIDER_FACTORY_GET_PROVIDERS_METHOD_NAME, e); return null;
[2/3] accumulo git commit: Merge branch '1.6' into 1.7
Merge branch '1.6' into 1.7 Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/724bfb31 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/724bfb31 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/724bfb31 Branch: refs/heads/master Commit: 724bfb3128cc87f5259d43bf41e7559c8ad82619 Parents: 2481323 a2e131b Author: Billie Rinaldi bil...@apache.org Authored: Wed Jun 10 18:07:04 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Wed Jun 10 18:07:04 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +++- 1 file changed, 23 insertions(+), 1 deletion(-) --
[1/3] accumulo git commit: ACCUMULO-3890 cache CredentialProvider objects to prevent unneeded NN ops
Repository: accumulo Updated Branches: refs/heads/master a5e4ae849 - 3fe4f690b ACCUMULO-3890 cache CredentialProvider objects to prevent unneeded NN ops Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/a2e131bd Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/a2e131bd Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/a2e131bd Branch: refs/heads/master Commit: a2e131bda969061b93e8639e11c3445d450e5bc3 Parents: c6d1ce5 Author: Billie Rinaldi bil...@apache.org Authored: Wed Jun 10 14:25:13 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Wed Jun 10 18:04:44 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +++- 1 file changed, 23 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/a2e131bd/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java index 3c3c051..9af3e00 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java @@ -21,7 +21,9 @@ import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import java.util.ArrayList; import java.util.Collections; +import java.util.HashMap; import java.util.List; +import java.util.Map; import org.apache.accumulo.core.util.CachedConfiguration; import org.apache.hadoop.conf.Configuration; @@ -62,6 +64,9 @@ public class CredentialProviderFactoryShim { private static Method flushMethod = null; private static Boolean hadoopClassesAvailable = null; + // access to cachedProviders should be synchronized when necessary (for example see getCredentialProviders) + private static final MapString,ListObject cachedProviders = new HashMapString,ListObject(); + /** * Determine if we can load the necessary CredentialProvider classes. Only loaded the first time, so subsequent invocations of this method should return fast. * @@ -200,6 +205,15 @@ public class CredentialProviderFactoryShim { */ @SuppressWarnings(unchecked) protected static ListObject getCredentialProviders(Configuration conf) { +String path = conf.get(CREDENTIAL_PROVIDER_PATH); +if (path == null || path.isEmpty()) { + return null; +} + +if (cachedProviders.containsKey(path)) { + return cachedProviders.get(path); +} + // Call CredentialProviderFactory.getProviders(Configuration) Object providersObj = null; try { @@ -217,7 +231,15 @@ public class CredentialProviderFactoryShim { // Cast the Object to ListObject (actually ListCredentialProvider) try { - return (ListObject) providersObj; + ListObject providersList = (ListObject) providersObj; + synchronized (cachedProviders) { +if (cachedProviders.containsKey(path)) { + return cachedProviders.get(path); +} else { + cachedProviders.put(path, providersList); +} + } + return providersList; } catch (ClassCastException e) { log.error(Expected a List from {} method, HADOOP_CRED_PROVIDER_FACTORY_GET_PROVIDERS_METHOD_NAME, e); return null;
[3/3] accumulo git commit: Merge branch '1.7'
Merge branch '1.7' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/3fe4f690 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/3fe4f690 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/3fe4f690 Branch: refs/heads/master Commit: 3fe4f690b59cd9cf33172c5638986c90cb634b9d Parents: a5e4ae8 724bfb3 Author: Billie Rinaldi bil...@apache.org Authored: Wed Jun 10 18:10:21 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Wed Jun 10 18:10:21 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +++- 1 file changed, 23 insertions(+), 1 deletion(-) --
[2/2] accumulo git commit: Merge branch '1.6' into 1.7
Merge branch '1.6' into 1.7 Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/724bfb31 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/724bfb31 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/724bfb31 Branch: refs/heads/1.7 Commit: 724bfb3128cc87f5259d43bf41e7559c8ad82619 Parents: 2481323 a2e131b Author: Billie Rinaldi bil...@apache.org Authored: Wed Jun 10 18:07:04 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Wed Jun 10 18:07:04 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +++- 1 file changed, 23 insertions(+), 1 deletion(-) --
[1/2] accumulo git commit: ACCUMULO-3890 cache CredentialProvider objects to prevent unneeded NN ops
Repository: accumulo Updated Branches: refs/heads/1.7 24813234f - 724bfb312 ACCUMULO-3890 cache CredentialProvider objects to prevent unneeded NN ops Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/a2e131bd Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/a2e131bd Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/a2e131bd Branch: refs/heads/1.7 Commit: a2e131bda969061b93e8639e11c3445d450e5bc3 Parents: c6d1ce5 Author: Billie Rinaldi bil...@apache.org Authored: Wed Jun 10 14:25:13 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Wed Jun 10 18:04:44 2015 -0700 -- .../conf/CredentialProviderFactoryShim.java | 24 +++- 1 file changed, 23 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/a2e131bd/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java index 3c3c051..9af3e00 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/CredentialProviderFactoryShim.java @@ -21,7 +21,9 @@ import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import java.util.ArrayList; import java.util.Collections; +import java.util.HashMap; import java.util.List; +import java.util.Map; import org.apache.accumulo.core.util.CachedConfiguration; import org.apache.hadoop.conf.Configuration; @@ -62,6 +64,9 @@ public class CredentialProviderFactoryShim { private static Method flushMethod = null; private static Boolean hadoopClassesAvailable = null; + // access to cachedProviders should be synchronized when necessary (for example see getCredentialProviders) + private static final MapString,ListObject cachedProviders = new HashMapString,ListObject(); + /** * Determine if we can load the necessary CredentialProvider classes. Only loaded the first time, so subsequent invocations of this method should return fast. * @@ -200,6 +205,15 @@ public class CredentialProviderFactoryShim { */ @SuppressWarnings(unchecked) protected static ListObject getCredentialProviders(Configuration conf) { +String path = conf.get(CREDENTIAL_PROVIDER_PATH); +if (path == null || path.isEmpty()) { + return null; +} + +if (cachedProviders.containsKey(path)) { + return cachedProviders.get(path); +} + // Call CredentialProviderFactory.getProviders(Configuration) Object providersObj = null; try { @@ -217,7 +231,15 @@ public class CredentialProviderFactoryShim { // Cast the Object to ListObject (actually ListCredentialProvider) try { - return (ListObject) providersObj; + ListObject providersList = (ListObject) providersObj; + synchronized (cachedProviders) { +if (cachedProviders.containsKey(path)) { + return cachedProviders.get(path); +} else { + cachedProviders.put(path, providersList); +} + } + return providersList; } catch (ClassCastException e) { log.error(Expected a List from {} method, HADOOP_CRED_PROVIDER_FACTORY_GET_PROVIDERS_METHOD_NAME, e); return null;
[1/2] accumulo git commit: ACCUMULO-3883 fix how StandaloneAccumuloClusterConfiguration loads ClientConfiguration
Repository: accumulo Updated Branches: refs/heads/master 0b7d8eeae - 3d9336b83 ACCUMULO-3883 fix how StandaloneAccumuloClusterConfiguration loads ClientConfiguration Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/92d19eeb Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/92d19eeb Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/92d19eeb Branch: refs/heads/master Commit: 92d19eeb7df0a902c594eec5081173d56f06488e Parents: 5bd993b Author: Billie Rinaldi bil...@apache.org Authored: Tue Jun 2 19:10:24 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Jun 2 19:10:24 2015 -0700 -- .../harness/conf/StandaloneAccumuloClusterConfiguration.java | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/92d19eeb/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java -- diff --git a/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java b/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java index e04c10c..4cf145b 100644 --- a/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java +++ b/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java @@ -86,9 +86,8 @@ public class StandaloneAccumuloClusterConfiguration extends AccumuloClusterPrope this.conf = getConfiguration(type); this.clientConfFile = clientConfFile; -this.clientConf = ClientConfiguration.loadDefault(); try { - clientConf.addConfiguration(new ClientConfiguration(clientConfFile)); + this.clientConf = new ClientConfiguration(clientConfFile); } catch (ConfigurationException e) { throw new RuntimeException(Failed to load client configuration from + clientConfFile); }
[2/2] accumulo git commit: Merge branch '1.7'
Merge branch '1.7' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/3d9336b8 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/3d9336b8 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/3d9336b8 Branch: refs/heads/master Commit: 3d9336b83fc5a90774968b5700d60e761905fb47 Parents: 0b7d8ee 92d19ee Author: Billie Rinaldi bil...@apache.org Authored: Tue Jun 2 19:11:49 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Jun 2 19:11:49 2015 -0700 -- .../harness/conf/StandaloneAccumuloClusterConfiguration.java | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --
accumulo git commit: ACCUMULO-3883 remove ClientConfiguration.loadDefault from TransportCachingIT
Repository: accumulo Updated Branches: refs/heads/1.7 92d19eeb7 - c475af0f9 ACCUMULO-3883 remove ClientConfiguration.loadDefault from TransportCachingIT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/c475af0f Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/c475af0f Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/c475af0f Branch: refs/heads/1.7 Commit: c475af0f94bf3a9a7871159c448cf30bd6f52bb9 Parents: 92d19ee Author: Billie Rinaldi bil...@apache.org Authored: Tue Jun 2 19:44:09 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Jun 2 19:44:09 2015 -0700 -- .../test/java/org/apache/accumulo/test/TransportCachingIT.java| 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/c475af0f/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java b/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java index 5f9d601..5bfb207 100644 --- a/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java +++ b/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java @@ -55,8 +55,7 @@ public class TransportCachingIT extends AccumuloClusterIT { public void testCachedTransport() { Connector conn = getConnector(); Instance instance = conn.getInstance(); -ClientConfiguration clientConf = ClientConfiguration.loadDefault(); - clientConf.withInstance(instance.getInstanceName()).withZkHosts(instance.getZooKeepers()); +ClientConfiguration clientConf = cluster.getClientConfig(); ClientContext context = new ClientContext(instance, new Credentials(getAdminPrincipal(), getAdminToken()), clientConf); long rpcTimeout = DefaultConfiguration.getTimeInMillis(Property.GENERAL_RPC_TIMEOUT.getDefaultValue());
[2/2] accumulo git commit: Merge branch '1.7'
Merge branch '1.7' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/e6e85cfa Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/e6e85cfa Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/e6e85cfa Branch: refs/heads/master Commit: e6e85cfa1e50b094d4111f4a0a66de68f112c0fe Parents: 3d9336b c475af0 Author: Billie Rinaldi bil...@apache.org Authored: Tue Jun 2 20:17:11 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Jun 2 20:17:11 2015 -0700 -- .../test/java/org/apache/accumulo/test/TransportCachingIT.java| 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --
[1/2] accumulo git commit: ACCUMULO-3883 remove ClientConfiguration.loadDefault from TransportCachingIT
Repository: accumulo Updated Branches: refs/heads/master 3d9336b83 - e6e85cfa1 ACCUMULO-3883 remove ClientConfiguration.loadDefault from TransportCachingIT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/c475af0f Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/c475af0f Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/c475af0f Branch: refs/heads/master Commit: c475af0f94bf3a9a7871159c448cf30bd6f52bb9 Parents: 92d19ee Author: Billie Rinaldi bil...@apache.org Authored: Tue Jun 2 19:44:09 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Jun 2 19:44:09 2015 -0700 -- .../test/java/org/apache/accumulo/test/TransportCachingIT.java| 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/c475af0f/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java b/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java index 5f9d601..5bfb207 100644 --- a/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java +++ b/test/src/test/java/org/apache/accumulo/test/TransportCachingIT.java @@ -55,8 +55,7 @@ public class TransportCachingIT extends AccumuloClusterIT { public void testCachedTransport() { Connector conn = getConnector(); Instance instance = conn.getInstance(); -ClientConfiguration clientConf = ClientConfiguration.loadDefault(); - clientConf.withInstance(instance.getInstanceName()).withZkHosts(instance.getZooKeepers()); +ClientConfiguration clientConf = cluster.getClientConfig(); ClientContext context = new ClientContext(instance, new Credentials(getAdminPrincipal(), getAdminToken()), clientConf); long rpcTimeout = DefaultConfiguration.getTimeInMillis(Property.GENERAL_RPC_TIMEOUT.getDefaultValue());
accumulo git commit: ACCUMULO-3883 fix how StandaloneAccumuloClusterConfiguration loads ClientConfiguration
Repository: accumulo Updated Branches: refs/heads/1.7 5bd993bae - 92d19eeb7 ACCUMULO-3883 fix how StandaloneAccumuloClusterConfiguration loads ClientConfiguration Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/92d19eeb Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/92d19eeb Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/92d19eeb Branch: refs/heads/1.7 Commit: 92d19eeb7df0a902c594eec5081173d56f06488e Parents: 5bd993b Author: Billie Rinaldi bil...@apache.org Authored: Tue Jun 2 19:10:24 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Jun 2 19:10:24 2015 -0700 -- .../harness/conf/StandaloneAccumuloClusterConfiguration.java | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/92d19eeb/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java -- diff --git a/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java b/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java index e04c10c..4cf145b 100644 --- a/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java +++ b/test/src/test/java/org/apache/accumulo/harness/conf/StandaloneAccumuloClusterConfiguration.java @@ -86,9 +86,8 @@ public class StandaloneAccumuloClusterConfiguration extends AccumuloClusterPrope this.conf = getConfiguration(type); this.clientConfFile = clientConfFile; -this.clientConf = ClientConfiguration.loadDefault(); try { - clientConf.addConfiguration(new ClientConfiguration(clientConfFile)); + this.clientConf = new ClientConfiguration(clientConfFile); } catch (ConfigurationException e) { throw new RuntimeException(Failed to load client configuration from + clientConfFile); }
[1/2] accumulo git commit: ACCUMULO-3862 improved how AsyncSpanReceiver drops short spans, added test for min span length
Repository: accumulo Updated Branches: refs/heads/master fe6451091 - 6021fdb2d ACCUMULO-3862 improved how AsyncSpanReceiver drops short spans, added test for min span length Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/0eef354c Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/0eef354c Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/0eef354c Branch: refs/heads/master Commit: 0eef354c56e9ef2788c72f4dbaac596653e25bbe Parents: 61bfbb2 Author: Billie Rinaldi bil...@apache.org Authored: Fri May 29 13:29:32 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Fri May 29 13:29:40 2015 -0700 -- .../accumulo/tracer/AsyncSpanReceiver.java | 14 +- .../accumulo/tracer/AsyncSpanReceiverTest.java | 129 +++ 2 files changed, 134 insertions(+), 9 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/0eef354c/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java -- diff --git a/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java b/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java index 28a9088..a35734d 100644 --- a/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java +++ b/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java @@ -55,7 +55,7 @@ public abstract class AsyncSpanReceiverSpanKey,Destination implements SpanRece public static final String SEND_TIMER_MILLIS = tracer.send.timer.millis; public static final String QUEUE_SIZE = tracer.queue.size; - private static final String SPAN_MIN_MS = tracer.span.min.ms; + public static final String SPAN_MIN_MS = tracer.span.min.ms; private final MapSpanKey,Destination clients = new HashMapSpanKey,Destination(); @@ -109,14 +109,6 @@ public abstract class AsyncSpanReceiverSpanKey,Destination implements SpanRece while (!sendQueue.isEmpty()) { boolean sent = false; RemoteSpan s = sendQueue.peek(); - if (s.stop - s.start minSpanSize) { -synchronized (sendQueue) { - sendQueue.remove(); - sendQueue.notifyAll(); - sendQueueSize.decrementAndGet(); -} -continue; - } SpanKey dest = getSpanKey(s.data); Destination client = clients.get(dest); if (client == null) { @@ -167,6 +159,10 @@ public abstract class AsyncSpanReceiverSpanKey,Destination implements SpanRece @Override public void receiveSpan(Span s) { +if (s.getStopTimeMillis() - s.getStartTimeMillis() minSpanSize) { + return; +} + MapString,String data = convertToStrings(s.getKVAnnotations()); SpanKey dest = getSpanKey(data); http://git-wip-us.apache.org/repos/asf/accumulo/blob/0eef354c/server/tracer/src/test/java/org/apache/accumulo/tracer/AsyncSpanReceiverTest.java -- diff --git a/server/tracer/src/test/java/org/apache/accumulo/tracer/AsyncSpanReceiverTest.java b/server/tracer/src/test/java/org/apache/accumulo/tracer/AsyncSpanReceiverTest.java new file mode 100644 index 000..6744efc --- /dev/null +++ b/server/tracer/src/test/java/org/apache/accumulo/tracer/AsyncSpanReceiverTest.java @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the License); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.tracer; + +import org.apache.accumulo.tracer.thrift.RemoteSpan; +import org.apache.htrace.HTraceConfiguration; +import org.apache.htrace.Span; +import org.apache.htrace.impl.MilliSpan; +import org.junit.Test; + +import java.util.Collections; +import java.util.Map; +import java.util.concurrent.atomic.AtomicInteger; + +import static org.junit.Assert.assertEquals; + +public class AsyncSpanReceiverTest { + static class TestReceiver extends AsyncSpanReceiverString,String { +AtomicInteger spansSent = new AtomicInteger(0); + +TestReceiver() { + super(HTraceConfiguration.EMPTY
[2/2] accumulo git commit: Merge branch '1.7'
Merge branch '1.7' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/6021fdb2 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/6021fdb2 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/6021fdb2 Branch: refs/heads/master Commit: 6021fdb2d64cf75d12e94e06cf89b36eede68c7f Parents: fe64510 0eef354 Author: Billie Rinaldi bil...@apache.org Authored: Fri May 29 14:06:17 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Fri May 29 14:06:17 2015 -0700 -- .../accumulo/tracer/AsyncSpanReceiver.java | 14 +- .../accumulo/tracer/AsyncSpanReceiverTest.java | 129 +++ 2 files changed, 134 insertions(+), 9 deletions(-) --
accumulo git commit: ACCUMULO-3862 improved how AsyncSpanReceiver drops short spans, added test for min span length
Repository: accumulo Updated Branches: refs/heads/1.7 61bfbb22e - 0eef354c5 ACCUMULO-3862 improved how AsyncSpanReceiver drops short spans, added test for min span length Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/0eef354c Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/0eef354c Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/0eef354c Branch: refs/heads/1.7 Commit: 0eef354c56e9ef2788c72f4dbaac596653e25bbe Parents: 61bfbb2 Author: Billie Rinaldi bil...@apache.org Authored: Fri May 29 13:29:32 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Fri May 29 13:29:40 2015 -0700 -- .../accumulo/tracer/AsyncSpanReceiver.java | 14 +- .../accumulo/tracer/AsyncSpanReceiverTest.java | 129 +++ 2 files changed, 134 insertions(+), 9 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/0eef354c/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java -- diff --git a/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java b/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java index 28a9088..a35734d 100644 --- a/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java +++ b/server/tracer/src/main/java/org/apache/accumulo/tracer/AsyncSpanReceiver.java @@ -55,7 +55,7 @@ public abstract class AsyncSpanReceiverSpanKey,Destination implements SpanRece public static final String SEND_TIMER_MILLIS = tracer.send.timer.millis; public static final String QUEUE_SIZE = tracer.queue.size; - private static final String SPAN_MIN_MS = tracer.span.min.ms; + public static final String SPAN_MIN_MS = tracer.span.min.ms; private final MapSpanKey,Destination clients = new HashMapSpanKey,Destination(); @@ -109,14 +109,6 @@ public abstract class AsyncSpanReceiverSpanKey,Destination implements SpanRece while (!sendQueue.isEmpty()) { boolean sent = false; RemoteSpan s = sendQueue.peek(); - if (s.stop - s.start minSpanSize) { -synchronized (sendQueue) { - sendQueue.remove(); - sendQueue.notifyAll(); - sendQueueSize.decrementAndGet(); -} -continue; - } SpanKey dest = getSpanKey(s.data); Destination client = clients.get(dest); if (client == null) { @@ -167,6 +159,10 @@ public abstract class AsyncSpanReceiverSpanKey,Destination implements SpanRece @Override public void receiveSpan(Span s) { +if (s.getStopTimeMillis() - s.getStartTimeMillis() minSpanSize) { + return; +} + MapString,String data = convertToStrings(s.getKVAnnotations()); SpanKey dest = getSpanKey(data); http://git-wip-us.apache.org/repos/asf/accumulo/blob/0eef354c/server/tracer/src/test/java/org/apache/accumulo/tracer/AsyncSpanReceiverTest.java -- diff --git a/server/tracer/src/test/java/org/apache/accumulo/tracer/AsyncSpanReceiverTest.java b/server/tracer/src/test/java/org/apache/accumulo/tracer/AsyncSpanReceiverTest.java new file mode 100644 index 000..6744efc --- /dev/null +++ b/server/tracer/src/test/java/org/apache/accumulo/tracer/AsyncSpanReceiverTest.java @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the License); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.tracer; + +import org.apache.accumulo.tracer.thrift.RemoteSpan; +import org.apache.htrace.HTraceConfiguration; +import org.apache.htrace.Span; +import org.apache.htrace.impl.MilliSpan; +import org.junit.Test; + +import java.util.Collections; +import java.util.Map; +import java.util.concurrent.atomic.AtomicInteger; + +import static org.junit.Assert.assertEquals; + +public class AsyncSpanReceiverTest { + static class TestReceiver extends AsyncSpanReceiverString,String { +AtomicInteger spansSent = new AtomicInteger(0); + +TestReceiver() { + super(HTraceConfiguration.EMPTY
svn commit: r950609 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/ 1.6/apidocs/
Author: billie Date: Fri May 8 17:30:40 2015 New Revision: 950609 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r950608, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r950608, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r950608, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r950608, websites/production/accumulo/content/1.6/apidocs/
svn commit: r1678398 - /accumulo/site/trunk/content/people.mdtext
Author: billie Date: Fri May 8 17:23:54 2015 New Revision: 1678398 URL: http://svn.apache.org/r1678398 Log: updated contributors list based on git log Modified: accumulo/site/trunk/content/people.mdtext Modified: accumulo/site/trunk/content/people.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/people.mdtext?rev=1678398r1=1678397r2=1678398view=diff == --- accumulo/site/trunk/content/people.mdtext (original) +++ accumulo/site/trunk/content/people.mdtext Fri May 8 17:23:54 2015 @@ -56,36 +56,45 @@ Contributors trtd/tdtdAlex Moundalexis/tdtda href=http://www.cloudera.com;Cloudera/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtd/tdtdAndrew George Wells/tdtda href=http://clearedgeit.com/;ClearEdgeIT/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtd/tdtdArshak Navruzyan/tdtda href=http://www.argyledata.com/;Argyle Data/a/tdtd/td/tr + trtd/tdtdBob Thorman/tdtda href=http://www.att.com/;ATT/a/tdtd/td/tr trtd/tdtdChris Bennight/tdtd/tdtd/td/tr trtd/tdtdChris McCubbin/tdtda href=http://sqrrl.com/sqrrl/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtd/tdtdChristian Rohling/tdtda href=http://endgame.com/Endgame/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtd/tdtdDamon Brown/tdtda href=http://www.tetraconcepts.com;Tetra Concepts LLC/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtd/tdtdDave Wang/tdtda href=http://www.cloudera.com/;Cloudera/a/tdtdPT (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/pst.html-8/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/pdt.html-7/a)/td/tr trtd/tdtdDavid M. Lyle/tdtd/tdtd/td/tr + trtd/tdtdDavid Protzman/tdtd/tdtd/td/tr trtd/tdtdDennis Patrone/tdtda href=http://www.jhuapl.edu/The Johns Hopkins UniversitybrApplied Physics Laboratory/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr + trtd/tdtdDylan Hutchison/tdtd/tdtd/td/tr trtdetcoleman/tdtdEd Coleman/tdtd/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtd/tdtdEd Kohlwey/tdtda href=http://www.boozallen.com/Booz Allen Hamilton/a/tdtd/td/tr trtdedwardyoon/tdtdEdward Yoon/tdtd/tdtd/td/tr + trtd/tdtdEugene Cheipesh/tdtd/tdtd/td/tr trtd/tdtdGary Singh/tdtda href=http://www.sabre-eng.com;Sabre Engineering/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr + trtd/tdtdHayden Marchant/tdtd/tdtd/td/tr trtd/tdtdHung Pham/tdtda href=http://www.cloudera.com/;Cloudera/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtd/tdtdJacob Meisler/tdtda href=http://www.boozallen.com//;Booz Allen Hamilton/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtd/tdtdJames Fiori/tdtda href=http://flywheeldata.com/;Flywheel Data/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtdjaredwinick/tdtdJared Winick/tdtda href=http://www.koverse.com;Koverse/a/tdtdMT (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/mst.html-7/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/mdt.html-6/a)/td/tr trtd/tdtdJason Then/tdtd/tdtd/td/tr trtd/tdtdJay Shipper/tdtd/tdtd/td/tr - trtd/tdtdJesse Yates/tdtd/tdtd/td/tr - trtd/tdtdJessica Seastrom/tdtda href=http://www.cloudera.com;Cloudera/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtdjfield/tdtdJeff Field/tdtda
svn commit: r1678399 - /accumulo/site/trunk/content/people.mdtext
Author: billie Date: Fri May 8 17:28:15 2015 New Revision: 1678399 URL: http://svn.apache.org/r1678399 Log: fix a couple of mistakes in last commit Modified: accumulo/site/trunk/content/people.mdtext Modified: accumulo/site/trunk/content/people.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/people.mdtext?rev=1678399r1=1678398r2=1678399view=diff == --- accumulo/site/trunk/content/people.mdtext (original) +++ accumulo/site/trunk/content/people.mdtext Fri May 8 17:28:15 2015 @@ -123,8 +123,6 @@ Contributors trtd/tdtdTim Reardon/tdtd/tdtd/td/tr trtd/tdtdTravis Pinney/tdtd/tdtd/td/tr trtd/tdtdTristen Georgiou/tdtda href=http://phemi.com;PHEMI/a/tdtdPT (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/pst.html-8/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/pdt.html-7/a)/td/tr - trtdpctony/tdtdTony Stevenson/tdtd/tdtd/td/tr - trtd/tdtdTristen Georgiou/tdtd/tdtd/td/tr trtd/tdtdVicky Kak/tdtd/tdtd/td/tr trtd/tdtdVikram Srivastava/tdtda href=http://www.cloudera.com;Cloudera/a/tdtdPT (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/pst.html-8/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/pdt.html-7/a)/td/tr trtd/tdtdVincent Russell/tdtd/tdtd/td/tr
accumulo git commit: ACCUMULO-3760 make kv data in RemoteSpan strings instead of binary and remove unused methods of internal Span wrapper
Repository: accumulo Updated Branches: refs/heads/1.7 42588869a - f99638795 ACCUMULO-3760 make kv data in RemoteSpan strings instead of binary and remove unused methods of internal Span wrapper Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f9963879 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f9963879 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f9963879 Branch: refs/heads/1.7 Commit: f9963879503b5a9b4db9e29aad5bb004c4bb08d6 Parents: 4258886 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 28 17:24:30 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 28 17:24:30 2015 -0700 -- .../org/apache/accumulo/core/trace/Span.java| 87 +--- .../gc/GarbageCollectWriteAheadLogs.java| 2 +- .../monitor/servlets/trace/ShowTrace.java | 10 +-- .../accumulo/monitor/ShowTraceLinkTypeTest.java | 3 +- .../accumulo/tracer/AsyncSpanReceiver.java | 13 +-- .../accumulo/tracer/SendSpansViaThrift.java | 9 +- .../apache/accumulo/tracer/TraceFormatter.java | 9 +- .../apache/accumulo/tracer/ZooTraceClient.java | 3 +- .../accumulo/tracer/thrift/RemoteSpan.java | 68 +++ server/tracer/src/main/thrift/tracer.thrift | 4 +- .../apache/accumulo/trace/instrument/Span.java | 20 +++-- 11 files changed, 70 insertions(+), 158 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/f9963879/core/src/main/java/org/apache/accumulo/core/trace/Span.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/trace/Span.java b/core/src/main/java/org/apache/accumulo/core/trace/Span.java index 702c9c1..412feec 100644 --- a/core/src/main/java/org/apache/accumulo/core/trace/Span.java +++ b/core/src/main/java/org/apache/accumulo/core/trace/Span.java @@ -17,18 +17,14 @@ package org.apache.accumulo.core.trace; import org.apache.htrace.NullScope; -import org.apache.htrace.TimelineAnnotation; import org.apache.htrace.TraceScope; import static java.nio.charset.StandardCharsets.UTF_8; -import java.util.List; -import java.util.Map; - /** * This is a wrapper for a TraceScope object, which is a wrapper for a Span and its parent. Not recommended for client use. */ -public class Span implements org.apache.htrace.Span { +public class Span { public static final Span NULL_SPAN = new Span(NullScope.INSTANCE); private TraceScope scope = null; protected org.apache.htrace.Span span = null; @@ -59,7 +55,6 @@ public class Span implements org.apache.htrace.Span { span.addKVAnnotation(k.getBytes(UTF_8), v.getBytes(UTF_8)); } - @Override public void stop() { if (scope == null) { if (span != null) { @@ -69,84 +64,4 @@ public class Span implements org.apache.htrace.Span { scope.close(); } } - - @Override - public long getStartTimeMillis() { -return span.getStartTimeMillis(); - } - - @Override - public long getStopTimeMillis() { -return span.getStopTimeMillis(); - } - - @Override - public long getAccumulatedMillis() { -return span.getAccumulatedMillis(); - } - - @Override - public boolean isRunning() { -return span.isRunning(); - } - - @Override - public String getDescription() { -return span.getDescription(); - } - - @Override - public long getSpanId() { -return span.getSpanId(); - } - - @Override - public long getTraceId() { -return span.getTraceId(); - } - - @Override - public Span child(String s) { -return new Span(span.child(s)); - } - - @Override - public long getParentId() { -return span.getParentId(); - } - - @Override - public void addKVAnnotation(byte[] k, byte[] v) { -span.addKVAnnotation(k, v); - } - - @Override - public void addTimelineAnnotation(String s) { -span.addTimelineAnnotation(s); - } - - @Override - public Mapbyte[],byte[] getKVAnnotations() { -return span.getKVAnnotations(); - } - - @Override - public ListTimelineAnnotation getTimelineAnnotations() { -return span.getTimelineAnnotations(); - } - - @Override - public String getProcessId() { -return span.getProcessId(); - } - - @Override - public String toJson() { -return span.toJson(); - } - - @Override - public String toString() { -return span.toString(); - } } http://git-wip-us.apache.org/repos/asf/accumulo/blob/f9963879/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java -- diff --git a/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java b/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java index 9f537af..a9a3f65 100644
[1/2] accumulo git commit: ACCUMULO-3760 make kv data in RemoteSpan strings instead of binary and remove unused methods of internal Span wrapper
Repository: accumulo Updated Branches: refs/heads/master 2e336b832 - 26d66914a ACCUMULO-3760 make kv data in RemoteSpan strings instead of binary and remove unused methods of internal Span wrapper Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f9963879 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f9963879 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f9963879 Branch: refs/heads/master Commit: f9963879503b5a9b4db9e29aad5bb004c4bb08d6 Parents: 4258886 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 28 17:24:30 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 28 17:24:30 2015 -0700 -- .../org/apache/accumulo/core/trace/Span.java| 87 +--- .../gc/GarbageCollectWriteAheadLogs.java| 2 +- .../monitor/servlets/trace/ShowTrace.java | 10 +-- .../accumulo/monitor/ShowTraceLinkTypeTest.java | 3 +- .../accumulo/tracer/AsyncSpanReceiver.java | 13 +-- .../accumulo/tracer/SendSpansViaThrift.java | 9 +- .../apache/accumulo/tracer/TraceFormatter.java | 9 +- .../apache/accumulo/tracer/ZooTraceClient.java | 3 +- .../accumulo/tracer/thrift/RemoteSpan.java | 68 +++ server/tracer/src/main/thrift/tracer.thrift | 4 +- .../apache/accumulo/trace/instrument/Span.java | 20 +++-- 11 files changed, 70 insertions(+), 158 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/f9963879/core/src/main/java/org/apache/accumulo/core/trace/Span.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/trace/Span.java b/core/src/main/java/org/apache/accumulo/core/trace/Span.java index 702c9c1..412feec 100644 --- a/core/src/main/java/org/apache/accumulo/core/trace/Span.java +++ b/core/src/main/java/org/apache/accumulo/core/trace/Span.java @@ -17,18 +17,14 @@ package org.apache.accumulo.core.trace; import org.apache.htrace.NullScope; -import org.apache.htrace.TimelineAnnotation; import org.apache.htrace.TraceScope; import static java.nio.charset.StandardCharsets.UTF_8; -import java.util.List; -import java.util.Map; - /** * This is a wrapper for a TraceScope object, which is a wrapper for a Span and its parent. Not recommended for client use. */ -public class Span implements org.apache.htrace.Span { +public class Span { public static final Span NULL_SPAN = new Span(NullScope.INSTANCE); private TraceScope scope = null; protected org.apache.htrace.Span span = null; @@ -59,7 +55,6 @@ public class Span implements org.apache.htrace.Span { span.addKVAnnotation(k.getBytes(UTF_8), v.getBytes(UTF_8)); } - @Override public void stop() { if (scope == null) { if (span != null) { @@ -69,84 +64,4 @@ public class Span implements org.apache.htrace.Span { scope.close(); } } - - @Override - public long getStartTimeMillis() { -return span.getStartTimeMillis(); - } - - @Override - public long getStopTimeMillis() { -return span.getStopTimeMillis(); - } - - @Override - public long getAccumulatedMillis() { -return span.getAccumulatedMillis(); - } - - @Override - public boolean isRunning() { -return span.isRunning(); - } - - @Override - public String getDescription() { -return span.getDescription(); - } - - @Override - public long getSpanId() { -return span.getSpanId(); - } - - @Override - public long getTraceId() { -return span.getTraceId(); - } - - @Override - public Span child(String s) { -return new Span(span.child(s)); - } - - @Override - public long getParentId() { -return span.getParentId(); - } - - @Override - public void addKVAnnotation(byte[] k, byte[] v) { -span.addKVAnnotation(k, v); - } - - @Override - public void addTimelineAnnotation(String s) { -span.addTimelineAnnotation(s); - } - - @Override - public Mapbyte[],byte[] getKVAnnotations() { -return span.getKVAnnotations(); - } - - @Override - public ListTimelineAnnotation getTimelineAnnotations() { -return span.getTimelineAnnotations(); - } - - @Override - public String getProcessId() { -return span.getProcessId(); - } - - @Override - public String toJson() { -return span.toJson(); - } - - @Override - public String toString() { -return span.toString(); - } } http://git-wip-us.apache.org/repos/asf/accumulo/blob/f9963879/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java -- diff --git a/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java b/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java index 9f537af..a9a3f65 100644
[2/2] accumulo git commit: Merge branch '1.7'
Merge branch '1.7' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/26d66914 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/26d66914 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/26d66914 Branch: refs/heads/master Commit: 26d66914a9b2b0ff4150a7144ab9fa4e5ce7bc03 Parents: 2e336b8 f996387 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 28 17:25:08 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 28 17:25:08 2015 -0700 -- .../org/apache/accumulo/core/trace/Span.java| 87 +--- .../gc/GarbageCollectWriteAheadLogs.java| 2 +- .../monitor/servlets/trace/ShowTrace.java | 10 +-- .../accumulo/monitor/ShowTraceLinkTypeTest.java | 3 +- .../accumulo/tracer/AsyncSpanReceiver.java | 13 +-- .../accumulo/tracer/SendSpansViaThrift.java | 9 +- .../apache/accumulo/tracer/TraceFormatter.java | 9 +- .../apache/accumulo/tracer/ZooTraceClient.java | 3 +- .../accumulo/tracer/thrift/RemoteSpan.java | 68 +++ server/tracer/src/main/thrift/tracer.thrift | 4 +- .../apache/accumulo/trace/instrument/Span.java | 20 +++-- 11 files changed, 70 insertions(+), 158 deletions(-) --
[1/2] accumulo git commit: Revert ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly
Repository: accumulo Updated Branches: refs/heads/1.7 d3f74f7be - 0fb7f3a32 Revert ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly This reverts commit f5c7f05af7bdb95a2485bc0566bd2c4968f1427f. Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/dd9cc1cf Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/dd9cc1cf Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/dd9cc1cf Branch: refs/heads/1.7 Commit: dd9cc1cfb29928db8d2fc162e8cb04d9f7c2aba3 Parents: 5a42b1f Author: Billie Rinaldi bil...@apache.org Authored: Thu Apr 23 06:44:40 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Apr 23 06:44:40 2015 -0700 -- .../apache/accumulo/core/cli/ClientOpts.java| 3 +- .../core/client/ClientConfiguration.java| 32 +--- .../core/util/shell/ShellOptionsJC.java | 4 ++- .../core/conf/ClientConfigurationTest.java | 21 - .../src/test/resources/multi-valued.client.conf | 16 -- .../minicluster/MiniAccumuloInstance.java | 5 +-- 6 files changed, 7 insertions(+), 74 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/dd9cc1cf/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java index 4d00d97..8bb8b3f 100644 --- a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java +++ b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java @@ -46,6 +46,7 @@ import org.apache.accumulo.core.security.ColumnVisibility; import org.apache.accumulo.core.volume.VolumeConfiguration; import org.apache.accumulo.core.zookeeper.ZooUtil; import org.apache.accumulo.trace.instrument.Trace; +import org.apache.commons.configuration.PropertiesConfiguration; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.mapreduce.Job; @@ -229,7 +230,7 @@ public class ClientOpts extends Help { if (clientConfigFile == null) clientConfig = ClientConfiguration.loadDefault(); else -clientConfig = new ClientConfiguration(clientConfigFile); +clientConfig = new ClientConfiguration(new PropertiesConfiguration(clientConfigFile)); } catch (Exception e) { throw new IllegalArgumentException(e); } http://git-wip-us.apache.org/repos/asf/accumulo/blob/dd9cc1cf/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java index 91cda6a..17ad10b 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java +++ b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java @@ -27,13 +27,10 @@ import java.util.UUID; import org.apache.accumulo.core.conf.Property; import org.apache.accumulo.core.conf.PropertyType; import org.apache.accumulo.core.util.ArgumentChecker; -import org.apache.commons.configuration.AbstractConfiguration; import org.apache.commons.configuration.CompositeConfiguration; import org.apache.commons.configuration.Configuration; import org.apache.commons.configuration.ConfigurationException; import org.apache.commons.configuration.PropertiesConfiguration; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; /** * Contains a list of property keys recognized by the Accumulo client and convenience methods for setting them. @@ -41,8 +38,6 @@ import org.slf4j.LoggerFactory; * @since 1.6.0 */ public class ClientConfiguration extends CompositeConfiguration { - private static final Logger log = LoggerFactory.getLogger(ClientConfiguration.class); - public static final String USER_ACCUMULO_DIR_NAME = .accumulo; public static final String USER_CONF_FILENAME = config; public static final String GLOBAL_CONF_FILENAME = client.conf; @@ -110,31 +105,10 @@ public class ClientConfiguration extends CompositeConfiguration { } }; - public ClientConfiguration(String configFile) throws ConfigurationException { -this(new PropertiesConfiguration(), configFile); - } - - private ClientConfiguration(PropertiesConfiguration propertiesConfiguration, String configFile) throws ConfigurationException { -super(propertiesConfiguration); -// Don't do list interpolation -propertiesConfiguration.setListDelimiter('\0'); -propertiesConfiguration.load(configFile); - } - public
[2/2] accumulo git commit: Merge branch '1.6' into 1.7
Merge branch '1.6' into 1.7 Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/0fb7f3a3 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/0fb7f3a3 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/0fb7f3a3 Branch: refs/heads/1.7 Commit: 0fb7f3a32d1b643cdc978acade5919d31697bfa3 Parents: d3f74f7 dd9cc1c Author: Billie Rinaldi bil...@apache.org Authored: Thu Apr 23 08:37:29 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Apr 23 08:37:29 2015 -0700 -- --
accumulo git commit: ACCUMULO-3742 always setDelimiterParsingDisabled when ClientConfiguration is constructed to avoid unneeded warnings
Repository: accumulo Updated Branches: refs/heads/1.7 0fb7f3a32 - f220709c1 ACCUMULO-3742 always setDelimiterParsingDisabled when ClientConfiguration is constructed to avoid unneeded warnings Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f220709c Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f220709c Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f220709c Branch: refs/heads/1.7 Commit: f220709c1b87e5e697f5a76740974800a3c60c99 Parents: 0fb7f3a Author: Billie Rinaldi bil...@apache.org Authored: Thu Apr 23 08:51:10 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Apr 23 08:51:10 2015 -0700 -- .../java/org/apache/accumulo/core/client/ClientConfiguration.java | 2 ++ 1 file changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/f220709c/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java index 7aab80c..f404bd8 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java +++ b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java @@ -154,6 +154,7 @@ public class ClientConfiguration extends CompositeConfiguration { private ClientConfiguration(PropertiesConfiguration propertiesConfiguration, String configFile) throws ConfigurationException { super(propertiesConfiguration); // Don't do list interpolation +this.setDelimiterParsingDisabled(true); propertiesConfiguration.setDelimiterParsingDisabled(true); propertiesConfiguration.load(configFile); } @@ -165,6 +166,7 @@ public class ClientConfiguration extends CompositeConfiguration { private ClientConfiguration(PropertiesConfiguration propertiesConfiguration, File configFile) throws ConfigurationException { super(propertiesConfiguration); // Don't do list interpolation +this.setDelimiterParsingDisabled(true); propertiesConfiguration.setDelimiterParsingDisabled(true); propertiesConfiguration.load(configFile); }
accumulo git commit: ACCUMULO-3742 return to setListDelimiter for configurations and test escape characters in values
Repository: accumulo Updated Branches: refs/heads/1.7 765099f26 - d04487be9 ACCUMULO-3742 return to setListDelimiter for configurations and test escape characters in values Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/d04487be Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/d04487be Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/d04487be Branch: refs/heads/1.7 Commit: d04487be92c1dbcfcfd1abb0670396e7d6a46595 Parents: 765099f Author: Billie Rinaldi bil...@apache.org Authored: Thu Apr 23 13:57:33 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Apr 23 13:57:33 2015 -0700 -- .../accumulo/core/client/ClientConfiguration.java | 14 +++--- .../accumulo/core/client/ClientConfigurationTest.java | 8 core/src/test/resources/multi-valued.client.conf | 2 +- .../accumulo/minicluster/MiniAccumuloInstance.java| 2 +- 4 files changed, 13 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/d04487be/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java index f404bd8..b531f26 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java +++ b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java @@ -154,8 +154,8 @@ public class ClientConfiguration extends CompositeConfiguration { private ClientConfiguration(PropertiesConfiguration propertiesConfiguration, String configFile) throws ConfigurationException { super(propertiesConfiguration); // Don't do list interpolation -this.setDelimiterParsingDisabled(true); -propertiesConfiguration.setDelimiterParsingDisabled(true); +this.setListDelimiter('\0'); +propertiesConfiguration.setListDelimiter('\0'); propertiesConfiguration.load(configFile); } @@ -166,22 +166,22 @@ public class ClientConfiguration extends CompositeConfiguration { private ClientConfiguration(PropertiesConfiguration propertiesConfiguration, File configFile) throws ConfigurationException { super(propertiesConfiguration); // Don't do list interpolation -this.setDelimiterParsingDisabled(true); -propertiesConfiguration.setDelimiterParsingDisabled(true); +this.setListDelimiter('\0'); +propertiesConfiguration.setListDelimiter('\0'); propertiesConfiguration.load(configFile); } public ClientConfiguration(List? extends Configuration configs) { super(configs); // Don't do list interpolation -this.setDelimiterParsingDisabled(true); +this.setListDelimiter('\0'); for (Configuration c : configs) { if (c instanceof AbstractConfiguration) { AbstractConfiguration abstractConfiguration = (AbstractConfiguration) c; if (!abstractConfiguration.isDelimiterParsingDisabled() abstractConfiguration.getListDelimiter() != '\0') { log.warn(Client configuration constructed with a Configuration that did not have list delimiter disabled or overridden, multi-valued config + properties may be unavailable); - abstractConfiguration.setDelimiterParsingDisabled(true); + abstractConfiguration.setListDelimiter('\0'); } } } @@ -230,7 +230,7 @@ public class ClientConfiguration extends CompositeConfiguration { public static ClientConfiguration deserialize(String serializedConfig) { PropertiesConfiguration propConfig = new PropertiesConfiguration(); -propConfig.setDelimiterParsingDisabled(true); +propConfig.setListDelimiter('\0'); try { propConfig.load(new StringReader(serializedConfig)); } catch (ConfigurationException e) { http://git-wip-us.apache.org/repos/asf/accumulo/blob/d04487be/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java -- diff --git a/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java b/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java index 3e8e0f3..6f91b0b 100644 --- a/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java +++ b/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java @@ -90,7 +90,7 @@ public class ClientConfigurationTest { // not the recommended way to construct a client configuration, but it works PropertiesConfiguration propertiesConfiguration = new PropertiesConfiguration
[2/2] accumulo git commit: Merge branch '1.7'
Merge branch '1.7' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/de715e70 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/de715e70 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/de715e70 Branch: refs/heads/master Commit: de715e70e2103e4a1ba6cb28efef8c447826dcd4 Parents: 5605dc7 d04487b Author: Billie Rinaldi bil...@apache.org Authored: Thu Apr 23 13:58:03 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Apr 23 13:58:03 2015 -0700 -- .../accumulo/core/client/ClientConfiguration.java | 14 +++--- .../accumulo/core/client/ClientConfigurationTest.java | 8 core/src/test/resources/multi-valued.client.conf | 2 +- .../accumulo/minicluster/MiniAccumuloInstance.java| 2 +- 4 files changed, 13 insertions(+), 13 deletions(-) --
accumulo git commit: ACCUMULO-3746 fix ClientConfiguration.getAllPropertiesWithPrefix and add test
Repository: accumulo Updated Branches: refs/heads/1.7 312b8762d - 47c64d9a0 ACCUMULO-3746 fix ClientConfiguration.getAllPropertiesWithPrefix and add test Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/47c64d9a Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/47c64d9a Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/47c64d9a Branch: refs/heads/1.7 Commit: 47c64d9a0e161f4083749e05dcbd015ab9e5d2a4 Parents: 312b876 Author: Billie Rinaldi bil...@apache.org Authored: Thu Apr 23 06:01:12 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Apr 23 06:01:12 2015 -0700 -- .../accumulo/core/client/ClientConfiguration.java | 8 ++-- .../accumulo/core/client/ClientConfigurationTest.java | 13 + 2 files changed, 19 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/47c64d9a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java index a926d35..7aab80c 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java +++ b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java @@ -302,8 +302,12 @@ public class ClientConfiguration extends CompositeConfiguration { public MapString,String getAllPropertiesWithPrefix(ClientProperty property) { checkType(property, PropertyType.PREFIX); -MapString,String propMap = new HashMapString,String(); -Iterator? iter = this.getKeys(property.getKey()); +MapString,String propMap = new HashMap(); +String prefix = property.getKey(); +if (prefix.endsWith(.)) { + prefix = prefix.substring(0, prefix.length() - 1); +} +Iterator? iter = this.getKeys(prefix); while (iter.hasNext()) { String p = (String) iter.next(); propMap.put(p, getString(p)); http://git-wip-us.apache.org/repos/asf/accumulo/blob/47c64d9a/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java -- diff --git a/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java b/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java index 225298b..3e8e0f3 100644 --- a/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java +++ b/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java @@ -123,4 +123,17 @@ public class ClientConfigurationTest { assertEquals(val, conf.get(ClientProperty.TRACE_SPAN_RECEIVERS)); assertEquals(1, conf.getList(ClientProperty.TRACE_SPAN_RECEIVERS.getKey()).size()); } + + @Test + public void testGetAllPropertiesWithPrefix() { +ClientConfiguration conf = new ClientConfiguration(); +conf.addProperty(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX.getKey() + first, 1st); +conf.addProperty(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX.getKey() + second, 2nd); +conf.addProperty(other, value); + +MapString, String props = conf.getAllPropertiesWithPrefix(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX); +assertEquals(2, props.size()); +assertEquals(1st, props.get(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX.getKey() + first)); +assertEquals(2nd, props.get(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX.getKey() + second)); + } }
[1/2] accumulo git commit: ACCUMULO-3746 fix ClientConfiguration.getAllPropertiesWithPrefix and add test
Repository: accumulo Updated Branches: refs/heads/master 58cfee84d - dd01cbfae ACCUMULO-3746 fix ClientConfiguration.getAllPropertiesWithPrefix and add test Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/47c64d9a Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/47c64d9a Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/47c64d9a Branch: refs/heads/master Commit: 47c64d9a0e161f4083749e05dcbd015ab9e5d2a4 Parents: 312b876 Author: Billie Rinaldi bil...@apache.org Authored: Thu Apr 23 06:01:12 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Apr 23 06:01:12 2015 -0700 -- .../accumulo/core/client/ClientConfiguration.java | 8 ++-- .../accumulo/core/client/ClientConfigurationTest.java | 13 + 2 files changed, 19 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/47c64d9a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java index a926d35..7aab80c 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java +++ b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java @@ -302,8 +302,12 @@ public class ClientConfiguration extends CompositeConfiguration { public MapString,String getAllPropertiesWithPrefix(ClientProperty property) { checkType(property, PropertyType.PREFIX); -MapString,String propMap = new HashMapString,String(); -Iterator? iter = this.getKeys(property.getKey()); +MapString,String propMap = new HashMap(); +String prefix = property.getKey(); +if (prefix.endsWith(.)) { + prefix = prefix.substring(0, prefix.length() - 1); +} +Iterator? iter = this.getKeys(prefix); while (iter.hasNext()) { String p = (String) iter.next(); propMap.put(p, getString(p)); http://git-wip-us.apache.org/repos/asf/accumulo/blob/47c64d9a/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java -- diff --git a/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java b/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java index 225298b..3e8e0f3 100644 --- a/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java +++ b/core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java @@ -123,4 +123,17 @@ public class ClientConfigurationTest { assertEquals(val, conf.get(ClientProperty.TRACE_SPAN_RECEIVERS)); assertEquals(1, conf.getList(ClientProperty.TRACE_SPAN_RECEIVERS.getKey()).size()); } + + @Test + public void testGetAllPropertiesWithPrefix() { +ClientConfiguration conf = new ClientConfiguration(); +conf.addProperty(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX.getKey() + first, 1st); +conf.addProperty(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX.getKey() + second, 2nd); +conf.addProperty(other, value); + +MapString, String props = conf.getAllPropertiesWithPrefix(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX); +assertEquals(2, props.size()); +assertEquals(1st, props.get(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX.getKey() + first)); +assertEquals(2nd, props.get(ClientProperty.TRACE_SPAN_RECEIVER_PREFIX.getKey() + second)); + } }
accumulo git commit: Revert ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly
Repository: accumulo Updated Branches: refs/heads/1.6 5a42b1f2b - dd9cc1cfb Revert ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly This reverts commit f5c7f05af7bdb95a2485bc0566bd2c4968f1427f. Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/dd9cc1cf Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/dd9cc1cf Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/dd9cc1cf Branch: refs/heads/1.6 Commit: dd9cc1cfb29928db8d2fc162e8cb04d9f7c2aba3 Parents: 5a42b1f Author: Billie Rinaldi bil...@apache.org Authored: Thu Apr 23 06:44:40 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Apr 23 06:44:40 2015 -0700 -- .../apache/accumulo/core/cli/ClientOpts.java| 3 +- .../core/client/ClientConfiguration.java| 32 +--- .../core/util/shell/ShellOptionsJC.java | 4 ++- .../core/conf/ClientConfigurationTest.java | 21 - .../src/test/resources/multi-valued.client.conf | 16 -- .../minicluster/MiniAccumuloInstance.java | 5 +-- 6 files changed, 7 insertions(+), 74 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/dd9cc1cf/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java index 4d00d97..8bb8b3f 100644 --- a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java +++ b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java @@ -46,6 +46,7 @@ import org.apache.accumulo.core.security.ColumnVisibility; import org.apache.accumulo.core.volume.VolumeConfiguration; import org.apache.accumulo.core.zookeeper.ZooUtil; import org.apache.accumulo.trace.instrument.Trace; +import org.apache.commons.configuration.PropertiesConfiguration; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.mapreduce.Job; @@ -229,7 +230,7 @@ public class ClientOpts extends Help { if (clientConfigFile == null) clientConfig = ClientConfiguration.loadDefault(); else -clientConfig = new ClientConfiguration(clientConfigFile); +clientConfig = new ClientConfiguration(new PropertiesConfiguration(clientConfigFile)); } catch (Exception e) { throw new IllegalArgumentException(e); } http://git-wip-us.apache.org/repos/asf/accumulo/blob/dd9cc1cf/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java index 91cda6a..17ad10b 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java +++ b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java @@ -27,13 +27,10 @@ import java.util.UUID; import org.apache.accumulo.core.conf.Property; import org.apache.accumulo.core.conf.PropertyType; import org.apache.accumulo.core.util.ArgumentChecker; -import org.apache.commons.configuration.AbstractConfiguration; import org.apache.commons.configuration.CompositeConfiguration; import org.apache.commons.configuration.Configuration; import org.apache.commons.configuration.ConfigurationException; import org.apache.commons.configuration.PropertiesConfiguration; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; /** * Contains a list of property keys recognized by the Accumulo client and convenience methods for setting them. @@ -41,8 +38,6 @@ import org.slf4j.LoggerFactory; * @since 1.6.0 */ public class ClientConfiguration extends CompositeConfiguration { - private static final Logger log = LoggerFactory.getLogger(ClientConfiguration.class); - public static final String USER_ACCUMULO_DIR_NAME = .accumulo; public static final String USER_CONF_FILENAME = config; public static final String GLOBAL_CONF_FILENAME = client.conf; @@ -110,31 +105,10 @@ public class ClientConfiguration extends CompositeConfiguration { } }; - public ClientConfiguration(String configFile) throws ConfigurationException { -this(new PropertiesConfiguration(), configFile); - } - - private ClientConfiguration(PropertiesConfiguration propertiesConfiguration, String configFile) throws ConfigurationException { -super(propertiesConfiguration); -// Don't do list interpolation -propertiesConfiguration.setListDelimiter('\0'); -propertiesConfiguration.load(configFile); - } - public
[2/2] accumulo git commit: Merge branch '1.7'
Merge branch '1.7' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/dd01cbfa Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/dd01cbfa Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/dd01cbfa Branch: refs/heads/master Commit: dd01cbfaea002cf3c736304fd045948add067aeb Parents: 58cfee8 47c64d9 Author: Billie Rinaldi bil...@apache.org Authored: Thu Apr 23 06:42:46 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Apr 23 06:42:46 2015 -0700 -- .../accumulo/core/client/ClientConfiguration.java | 8 ++-- .../accumulo/core/client/ClientConfigurationTest.java | 13 + 2 files changed, 19 insertions(+), 2 deletions(-) --
[2/3] accumulo git commit: Merge branch '1.6' into 1.7
Merge branch '1.6' into 1.7 Conflicts: core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java shell/src/main/java/org/apache/accumulo/shell/ShellOptionsJC.java Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/61b76b50 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/61b76b50 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/61b76b50 Branch: refs/heads/master Commit: 61b76b507356ec2d5650b011ec22bd550ccbbd56 Parents: b29bd92 f5c7f05 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 21 12:36:59 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 21 12:36:59 2015 -0700 -- .../apache/accumulo/core/cli/ClientOpts.java| 5 +- .../core/client/ClientConfiguration.java| 42 +++- .../core/client/ClientConfigurationTest.java| 51 ++-- .../src/test/resources/multi-valued.client.conf | 17 +++ .../minicluster/MiniAccumuloInstance.java | 5 +- .../org/apache/accumulo/proxy/ProxyServer.java | 3 +- .../apache/accumulo/shell/ShellOptionsJC.java | 4 +- .../StandaloneAccumuloClusterConfiguration.java | 3 +- .../org/apache/accumulo/test/ShellServerIT.java | 3 +- .../test/functional/ConfigurableMacIT.java | 3 +- 10 files changed, 116 insertions(+), 20 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/61b76b50/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java -- diff --cc core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java index 0975ec8,4d00d97..54e8b53 --- a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java +++ b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java @@@ -44,13 -43,12 +44,12 @@@ import org.apache.accumulo.core.conf.De import org.apache.accumulo.core.conf.Property; import org.apache.accumulo.core.security.Authorizations; import org.apache.accumulo.core.security.ColumnVisibility; +import org.apache.accumulo.core.trace.Trace; import org.apache.accumulo.core.volume.VolumeConfiguration; import org.apache.accumulo.core.zookeeper.ZooUtil; - import org.apache.commons.configuration.PropertiesConfiguration; -import org.apache.accumulo.trace.instrument.Trace; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; -import org.apache.hadoop.mapreduce.Job; +import org.apache.hadoop.security.UserGroupInformation; import org.apache.log4j.Level; import org.apache.log4j.Logger; @@@ -202,50 -191,6 +201,50 @@@ public class ClientOpts extends Help Trace.off(); } + /** + * Automatically update the options to use a KerberosToken when SASL is enabled for RPCs. Don't overwrite the options if the user has provided something + * specifically. + */ + public void updateKerberosCredentials() { +ClientConfiguration clientConfig; +try { + if (clientConfigFile == null) +clientConfig = ClientConfiguration.loadDefault(); + else - clientConfig = new ClientConfiguration(new PropertiesConfiguration(clientConfigFile)); ++clientConfig = new ClientConfiguration(clientConfigFile); +} catch (Exception e) { + throw new IllegalArgumentException(e); +} +updateKerberosCredentials(clientConfig); + } + + /** + * Automatically update the options to use a KerberosToken when SASL is enabled for RPCs. Don't overwrite the options if the user has provided something + * specifically. + */ + public void updateKerberosCredentials(ClientConfiguration clientConfig) { +final boolean clientConfSaslEnabled = Boolean.parseBoolean(clientConfig.get(ClientProperty.INSTANCE_RPC_SASL_ENABLED)); +if ((saslEnabled || clientConfSaslEnabled) null == tokenClassName) { + tokenClassName = KerberosToken.CLASS_NAME; + // ACCUMULO-3701 We need to ensure we're logged in before parseArgs returns as the MapReduce Job is going to make a copy of the current user (UGI) + // when it is instantiated. + if (null != keytabPath) { +File keytab = new File(keytabPath); +if (!keytab.exists() || !keytab.isFile()) { + throw new IllegalArgumentException(Keytab isn't a normal file: + keytabPath); +} +if (null == principal) { + throw new IllegalArgumentException(Principal must be provided if logging in via Keytab); +} +try { + UserGroupInformation.loginUserFromKeytab(principal, keytab.getAbsolutePath()); +} catch (IOException e) { + throw
[1/3] accumulo git commit: ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly
Repository: accumulo Updated Branches: refs/heads/master 321f57331 - 7d2eb6711 ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f5c7f05a Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f5c7f05a Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f5c7f05a Branch: refs/heads/master Commit: f5c7f05af7bdb95a2485bc0566bd2c4968f1427f Parents: f0b3487 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 21 10:12:23 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 21 10:12:23 2015 -0700 -- .../apache/accumulo/core/cli/ClientOpts.java| 3 +- .../core/client/ClientConfiguration.java| 32 +++- .../core/util/shell/ShellOptionsJC.java | 4 +-- .../core/conf/ClientConfigurationTest.java | 21 + .../src/test/resources/multi-valued.client.conf | 16 ++ .../minicluster/MiniAccumuloInstance.java | 5 ++- 6 files changed, 74 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/f5c7f05a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java index 8bb8b3f..4d00d97 100644 --- a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java +++ b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java @@ -46,7 +46,6 @@ import org.apache.accumulo.core.security.ColumnVisibility; import org.apache.accumulo.core.volume.VolumeConfiguration; import org.apache.accumulo.core.zookeeper.ZooUtil; import org.apache.accumulo.trace.instrument.Trace; -import org.apache.commons.configuration.PropertiesConfiguration; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.mapreduce.Job; @@ -230,7 +229,7 @@ public class ClientOpts extends Help { if (clientConfigFile == null) clientConfig = ClientConfiguration.loadDefault(); else -clientConfig = new ClientConfiguration(new PropertiesConfiguration(clientConfigFile)); +clientConfig = new ClientConfiguration(clientConfigFile); } catch (Exception e) { throw new IllegalArgumentException(e); } http://git-wip-us.apache.org/repos/asf/accumulo/blob/f5c7f05a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java index 17ad10b..91cda6a 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java +++ b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java @@ -27,10 +27,13 @@ import java.util.UUID; import org.apache.accumulo.core.conf.Property; import org.apache.accumulo.core.conf.PropertyType; import org.apache.accumulo.core.util.ArgumentChecker; +import org.apache.commons.configuration.AbstractConfiguration; import org.apache.commons.configuration.CompositeConfiguration; import org.apache.commons.configuration.Configuration; import org.apache.commons.configuration.ConfigurationException; import org.apache.commons.configuration.PropertiesConfiguration; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * Contains a list of property keys recognized by the Accumulo client and convenience methods for setting them. @@ -38,6 +41,8 @@ import org.apache.commons.configuration.PropertiesConfiguration; * @since 1.6.0 */ public class ClientConfiguration extends CompositeConfiguration { + private static final Logger log = LoggerFactory.getLogger(ClientConfiguration.class); + public static final String USER_ACCUMULO_DIR_NAME = .accumulo; public static final String USER_CONF_FILENAME = config; public static final String GLOBAL_CONF_FILENAME = client.conf; @@ -105,10 +110,31 @@ public class ClientConfiguration extends CompositeConfiguration { } }; + public ClientConfiguration(String configFile) throws ConfigurationException { +this(new PropertiesConfiguration(), configFile); + } + + private ClientConfiguration(PropertiesConfiguration propertiesConfiguration, String configFile) throws ConfigurationException { +super(propertiesConfiguration); +// Don't do list interpolation +propertiesConfiguration.setListDelimiter('\0'); +propertiesConfiguration.load(configFile); + } + public ClientConfiguration(List? extends
[3/3] accumulo git commit: Merge branch '1.7'
Merge branch '1.7' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/7d2eb671 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/7d2eb671 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/7d2eb671 Branch: refs/heads/master Commit: 7d2eb6711632c3766fe42d64c160b8687e08ca45 Parents: 321f573 61b76b5 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 21 12:42:06 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 21 12:42:06 2015 -0700 -- .../apache/accumulo/core/cli/ClientOpts.java| 5 +- .../core/client/ClientConfiguration.java| 42 +++- .../core/client/ClientConfigurationTest.java| 51 ++-- .../src/test/resources/multi-valued.client.conf | 17 +++ .../minicluster/MiniAccumuloInstance.java | 5 +- .../org/apache/accumulo/proxy/ProxyServer.java | 3 +- .../apache/accumulo/shell/ShellOptionsJC.java | 4 +- .../StandaloneAccumuloClusterConfiguration.java | 3 +- .../org/apache/accumulo/test/ShellServerIT.java | 3 +- .../test/functional/ConfigurableMacIT.java | 3 +- 10 files changed, 116 insertions(+), 20 deletions(-) --
[2/2] accumulo git commit: Merge branch '1.6' into 1.7
Merge branch '1.6' into 1.7 Conflicts: core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java shell/src/main/java/org/apache/accumulo/shell/ShellOptionsJC.java Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/61b76b50 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/61b76b50 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/61b76b50 Branch: refs/heads/1.7 Commit: 61b76b507356ec2d5650b011ec22bd550ccbbd56 Parents: b29bd92 f5c7f05 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 21 12:36:59 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 21 12:36:59 2015 -0700 -- .../apache/accumulo/core/cli/ClientOpts.java| 5 +- .../core/client/ClientConfiguration.java| 42 +++- .../core/client/ClientConfigurationTest.java| 51 ++-- .../src/test/resources/multi-valued.client.conf | 17 +++ .../minicluster/MiniAccumuloInstance.java | 5 +- .../org/apache/accumulo/proxy/ProxyServer.java | 3 +- .../apache/accumulo/shell/ShellOptionsJC.java | 4 +- .../StandaloneAccumuloClusterConfiguration.java | 3 +- .../org/apache/accumulo/test/ShellServerIT.java | 3 +- .../test/functional/ConfigurableMacIT.java | 3 +- 10 files changed, 116 insertions(+), 20 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/61b76b50/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java -- diff --cc core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java index 0975ec8,4d00d97..54e8b53 --- a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java +++ b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java @@@ -44,13 -43,12 +44,12 @@@ import org.apache.accumulo.core.conf.De import org.apache.accumulo.core.conf.Property; import org.apache.accumulo.core.security.Authorizations; import org.apache.accumulo.core.security.ColumnVisibility; +import org.apache.accumulo.core.trace.Trace; import org.apache.accumulo.core.volume.VolumeConfiguration; import org.apache.accumulo.core.zookeeper.ZooUtil; - import org.apache.commons.configuration.PropertiesConfiguration; -import org.apache.accumulo.trace.instrument.Trace; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; -import org.apache.hadoop.mapreduce.Job; +import org.apache.hadoop.security.UserGroupInformation; import org.apache.log4j.Level; import org.apache.log4j.Logger; @@@ -202,50 -191,6 +201,50 @@@ public class ClientOpts extends Help Trace.off(); } + /** + * Automatically update the options to use a KerberosToken when SASL is enabled for RPCs. Don't overwrite the options if the user has provided something + * specifically. + */ + public void updateKerberosCredentials() { +ClientConfiguration clientConfig; +try { + if (clientConfigFile == null) +clientConfig = ClientConfiguration.loadDefault(); + else - clientConfig = new ClientConfiguration(new PropertiesConfiguration(clientConfigFile)); ++clientConfig = new ClientConfiguration(clientConfigFile); +} catch (Exception e) { + throw new IllegalArgumentException(e); +} +updateKerberosCredentials(clientConfig); + } + + /** + * Automatically update the options to use a KerberosToken when SASL is enabled for RPCs. Don't overwrite the options if the user has provided something + * specifically. + */ + public void updateKerberosCredentials(ClientConfiguration clientConfig) { +final boolean clientConfSaslEnabled = Boolean.parseBoolean(clientConfig.get(ClientProperty.INSTANCE_RPC_SASL_ENABLED)); +if ((saslEnabled || clientConfSaslEnabled) null == tokenClassName) { + tokenClassName = KerberosToken.CLASS_NAME; + // ACCUMULO-3701 We need to ensure we're logged in before parseArgs returns as the MapReduce Job is going to make a copy of the current user (UGI) + // when it is instantiated. + if (null != keytabPath) { +File keytab = new File(keytabPath); +if (!keytab.exists() || !keytab.isFile()) { + throw new IllegalArgumentException(Keytab isn't a normal file: + keytabPath); +} +if (null == principal) { + throw new IllegalArgumentException(Principal must be provided if logging in via Keytab); +} +try { + UserGroupInformation.loginUserFromKeytab(principal, keytab.getAbsolutePath()); +} catch (IOException e) { + throw new
[1/2] accumulo git commit: ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly
Repository: accumulo Updated Branches: refs/heads/1.7 b29bd921b - 61b76b507 ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f5c7f05a Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f5c7f05a Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f5c7f05a Branch: refs/heads/1.7 Commit: f5c7f05af7bdb95a2485bc0566bd2c4968f1427f Parents: f0b3487 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 21 10:12:23 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 21 10:12:23 2015 -0700 -- .../apache/accumulo/core/cli/ClientOpts.java| 3 +- .../core/client/ClientConfiguration.java| 32 +++- .../core/util/shell/ShellOptionsJC.java | 4 +-- .../core/conf/ClientConfigurationTest.java | 21 + .../src/test/resources/multi-valued.client.conf | 16 ++ .../minicluster/MiniAccumuloInstance.java | 5 ++- 6 files changed, 74 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/f5c7f05a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java index 8bb8b3f..4d00d97 100644 --- a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java +++ b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java @@ -46,7 +46,6 @@ import org.apache.accumulo.core.security.ColumnVisibility; import org.apache.accumulo.core.volume.VolumeConfiguration; import org.apache.accumulo.core.zookeeper.ZooUtil; import org.apache.accumulo.trace.instrument.Trace; -import org.apache.commons.configuration.PropertiesConfiguration; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.mapreduce.Job; @@ -230,7 +229,7 @@ public class ClientOpts extends Help { if (clientConfigFile == null) clientConfig = ClientConfiguration.loadDefault(); else -clientConfig = new ClientConfiguration(new PropertiesConfiguration(clientConfigFile)); +clientConfig = new ClientConfiguration(clientConfigFile); } catch (Exception e) { throw new IllegalArgumentException(e); } http://git-wip-us.apache.org/repos/asf/accumulo/blob/f5c7f05a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java index 17ad10b..91cda6a 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java +++ b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java @@ -27,10 +27,13 @@ import java.util.UUID; import org.apache.accumulo.core.conf.Property; import org.apache.accumulo.core.conf.PropertyType; import org.apache.accumulo.core.util.ArgumentChecker; +import org.apache.commons.configuration.AbstractConfiguration; import org.apache.commons.configuration.CompositeConfiguration; import org.apache.commons.configuration.Configuration; import org.apache.commons.configuration.ConfigurationException; import org.apache.commons.configuration.PropertiesConfiguration; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * Contains a list of property keys recognized by the Accumulo client and convenience methods for setting them. @@ -38,6 +41,8 @@ import org.apache.commons.configuration.PropertiesConfiguration; * @since 1.6.0 */ public class ClientConfiguration extends CompositeConfiguration { + private static final Logger log = LoggerFactory.getLogger(ClientConfiguration.class); + public static final String USER_ACCUMULO_DIR_NAME = .accumulo; public static final String USER_CONF_FILENAME = config; public static final String GLOBAL_CONF_FILENAME = client.conf; @@ -105,10 +110,31 @@ public class ClientConfiguration extends CompositeConfiguration { } }; + public ClientConfiguration(String configFile) throws ConfigurationException { +this(new PropertiesConfiguration(), configFile); + } + + private ClientConfiguration(PropertiesConfiguration propertiesConfiguration, String configFile) throws ConfigurationException { +super(propertiesConfiguration); +// Don't do list interpolation +propertiesConfiguration.setListDelimiter('\0'); +propertiesConfiguration.load(configFile); + } + public ClientConfiguration(List? extends
accumulo git commit: ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly
Repository: accumulo Updated Branches: refs/heads/1.6 f0b3487bb - f5c7f05af ACCUMULO-3742 add ClientConfiguration constructor that reads from a file and handles multi-valued properties correctly Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f5c7f05a Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f5c7f05a Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f5c7f05a Branch: refs/heads/1.6 Commit: f5c7f05af7bdb95a2485bc0566bd2c4968f1427f Parents: f0b3487 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 21 10:12:23 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 21 10:12:23 2015 -0700 -- .../apache/accumulo/core/cli/ClientOpts.java| 3 +- .../core/client/ClientConfiguration.java| 32 +++- .../core/util/shell/ShellOptionsJC.java | 4 +-- .../core/conf/ClientConfigurationTest.java | 21 + .../src/test/resources/multi-valued.client.conf | 16 ++ .../minicluster/MiniAccumuloInstance.java | 5 ++- 6 files changed, 74 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/f5c7f05a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java index 8bb8b3f..4d00d97 100644 --- a/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java +++ b/core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java @@ -46,7 +46,6 @@ import org.apache.accumulo.core.security.ColumnVisibility; import org.apache.accumulo.core.volume.VolumeConfiguration; import org.apache.accumulo.core.zookeeper.ZooUtil; import org.apache.accumulo.trace.instrument.Trace; -import org.apache.commons.configuration.PropertiesConfiguration; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.mapreduce.Job; @@ -230,7 +229,7 @@ public class ClientOpts extends Help { if (clientConfigFile == null) clientConfig = ClientConfiguration.loadDefault(); else -clientConfig = new ClientConfiguration(new PropertiesConfiguration(clientConfigFile)); +clientConfig = new ClientConfiguration(clientConfigFile); } catch (Exception e) { throw new IllegalArgumentException(e); } http://git-wip-us.apache.org/repos/asf/accumulo/blob/f5c7f05a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java index 17ad10b..91cda6a 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java +++ b/core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java @@ -27,10 +27,13 @@ import java.util.UUID; import org.apache.accumulo.core.conf.Property; import org.apache.accumulo.core.conf.PropertyType; import org.apache.accumulo.core.util.ArgumentChecker; +import org.apache.commons.configuration.AbstractConfiguration; import org.apache.commons.configuration.CompositeConfiguration; import org.apache.commons.configuration.Configuration; import org.apache.commons.configuration.ConfigurationException; import org.apache.commons.configuration.PropertiesConfiguration; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * Contains a list of property keys recognized by the Accumulo client and convenience methods for setting them. @@ -38,6 +41,8 @@ import org.apache.commons.configuration.PropertiesConfiguration; * @since 1.6.0 */ public class ClientConfiguration extends CompositeConfiguration { + private static final Logger log = LoggerFactory.getLogger(ClientConfiguration.class); + public static final String USER_ACCUMULO_DIR_NAME = .accumulo; public static final String USER_CONF_FILENAME = config; public static final String GLOBAL_CONF_FILENAME = client.conf; @@ -105,10 +110,31 @@ public class ClientConfiguration extends CompositeConfiguration { } }; + public ClientConfiguration(String configFile) throws ConfigurationException { +this(new PropertiesConfiguration(), configFile); + } + + private ClientConfiguration(PropertiesConfiguration propertiesConfiguration, String configFile) throws ConfigurationException { +super(propertiesConfiguration); +// Don't do list interpolation +propertiesConfiguration.setListDelimiter('\0'); +propertiesConfiguration.load(configFile); + } + public ClientConfiguration(List? extends
accumulo git commit: ACCUMULO-3725 stop minc tracing before follow-on majc is initiated
Repository: accumulo Updated Branches: refs/heads/master fad97f7a6 - aba5278bd ACCUMULO-3725 stop minc tracing before follow-on majc is initiated Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/aba5278b Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/aba5278b Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/aba5278b Branch: refs/heads/master Commit: aba5278bd7c4b2f1db2c41783dc3dfae59cbd0ec Parents: fad97f7 Author: Billie Rinaldi billie.rina...@gmail.com Authored: Tue Apr 14 11:16:21 2015 -0700 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Tue Apr 14 17:52:42 2015 -0700 -- .../apache/accumulo/tserver/tablet/MinorCompactionTask.java | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/aba5278b/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/MinorCompactionTask.java -- diff --git a/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/MinorCompactionTask.java b/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/MinorCompactionTask.java index 53bf5c2..1b8347e 100644 --- a/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/MinorCompactionTask.java +++ b/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/MinorCompactionTask.java @@ -85,6 +85,11 @@ class MinorCompactionTask implements Runnable { mergeFile, true, queued, commitSession, flushId, mincReason); span.stop(); + minorCompaction.data(extent, tablet.getExtent().toString()); + minorCompaction.data(numEntries, Long.toString(this.stats.getNumEntries())); + minorCompaction.data(size, Long.toString(this.stats.getSize())); + minorCompaction.stop(); + if (tablet.needsSplit()) { tablet.getTabletServer().executeSplit(tablet); } else { @@ -95,9 +100,6 @@ class MinorCompactionTask implements Runnable { throw new RuntimeException(t); } finally { tablet.minorCompactionComplete(); - minorCompaction.data(extent, tablet.getExtent().toString()); - minorCompaction.data(numEntries, Long.toString(this.stats.getNumEntries())); - minorCompaction.data(size, Long.toString(this.stats.getSize())); minorCompaction.stop(); } }
[2/3] accumulo git commit: ACCUMULO-3715 adjust, document and standardize span receiver configuration properties
ACCUMULO-3715 adjust, document and standardize span receiver configuration properties Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/7e72d563 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/7e72d563 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/7e72d563 Branch: refs/heads/master Commit: 7e72d56331d84500d3887f8f304d1a0668f3aa89 Parents: 281c2f3 Author: Billie Rinaldi billie.rina...@gmail.com Authored: Wed Apr 8 11:54:22 2015 -0700 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Fri Apr 10 17:01:00 2015 -0700 -- .../org/apache/accumulo/core/conf/Property.java | 2 +- .../accumulo/core/trace/CountSampler.java | 3 + .../accumulo/core/trace/DistributedTrace.java | 1 - .../accumulo/core/trace/ProbabilitySampler.java | 3 + .../main/asciidoc/chapters/administration.txt | 58 --- .../accumulo/tracer/AsyncSpanReceiver.java | 15 +- .../apache/accumulo/tracer/TraceTableStats.java | 157 +++ 7 files changed, 217 insertions(+), 22 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/7e72d563/core/src/main/java/org/apache/accumulo/core/conf/Property.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/Property.java b/core/src/main/java/org/apache/accumulo/core/conf/Property.java index f9d8662..08ba3a2 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/Property.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/Property.java @@ -382,7 +382,7 @@ public enum Property { TRACE_SPAN_RECEIVERS(trace.span.receivers, org.apache.accumulo.tracer.ZooTraceClient, PropertyType.CLASSNAMELIST, A list of span receiver classes to send trace spans), TRACE_SPAN_RECEIVER_PREFIX(trace.span.receiver., null, PropertyType.PREFIX, Prefix for span receiver configuration properties), - TRACE_ZK_PATH(trace.span.receiver.zookeeper.path, Constants.ZTRACERS, PropertyType.STRING, The zookeeper node where tracers are registered), + TRACE_ZK_PATH(trace.zookeeper.path, Constants.ZTRACERS, PropertyType.STRING, The zookeeper node where tracers are registered), TRACE_PORT(trace.port.client, 12234, PropertyType.PORT, The listening port for the trace server), TRACE_TABLE(trace.table, trace, PropertyType.STRING, The name of the table to store distributed traces), TRACE_USER(trace.user, root, PropertyType.STRING, The name of the user to store distributed traces), http://git-wip-us.apache.org/repos/asf/accumulo/blob/7e72d563/core/src/main/java/org/apache/accumulo/core/trace/CountSampler.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/trace/CountSampler.java b/core/src/main/java/org/apache/accumulo/core/trace/CountSampler.java index 852fa9d..a4f206a 100644 --- a/core/src/main/java/org/apache/accumulo/core/trace/CountSampler.java +++ b/core/src/main/java/org/apache/accumulo/core/trace/CountSampler.java @@ -20,6 +20,9 @@ import org.apache.htrace.HTraceConfiguration; import java.util.Collections; +/** + * This wrapper intended for internal Accumulo tracing makes creating a CountSampler easier. + */ public class CountSampler extends org.apache.htrace.impl.CountSampler { public CountSampler(long frequency) { super(HTraceConfiguration.fromMap(Collections.singletonMap(CountSampler.SAMPLER_FREQUENCY_CONF_KEY, Long.toString(frequency; http://git-wip-us.apache.org/repos/asf/accumulo/blob/7e72d563/core/src/main/java/org/apache/accumulo/core/trace/DistributedTrace.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/trace/DistributedTrace.java b/core/src/main/java/org/apache/accumulo/core/trace/DistributedTrace.java index c5b6eac..14886f1 100644 --- a/core/src/main/java/org/apache/accumulo/core/trace/DistributedTrace.java +++ b/core/src/main/java/org/apache/accumulo/core/trace/DistributedTrace.java @@ -47,7 +47,6 @@ public class DistributedTrace { public static final String TRACE_HOST_PROPERTY = trace.host; public static final String TRACE_SERVICE_PROPERTY = trace.service; - public static final String TRACE_QUEUE_SIZE_PROPERTY = trace.queue.size; public static final String TRACER_ZK_HOST = tracer.zookeeper.host; public static final String TRACER_ZK_TIMEOUT = tracer.zookeeper.timeout; http://git-wip-us.apache.org/repos/asf/accumulo/blob/7e72d563/core/src/main/java/org/apache/accumulo/core/trace/ProbabilitySampler.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/trace/ProbabilitySampler.java b/core/src/main/java/org/apache/accumulo
[3/3] accumulo git commit: ACCUMULO-3715 handle unrooted spans better and fix TraceDump options
ACCUMULO-3715 handle unrooted spans better and fix TraceDump options Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/3a99300a Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/3a99300a Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/3a99300a Branch: refs/heads/master Commit: 3a99300a2b897f9f8740a3efa0271c43536ef0cc Parents: 7e72d56 Author: Billie Rinaldi billie.rina...@gmail.com Authored: Thu Apr 9 08:30:06 2015 -0700 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Fri Apr 10 17:01:00 2015 -0700 -- .../monitor/servlets/trace/ShowTrace.java | 90 ++-- .../org/apache/accumulo/tracer/TraceDump.java | 19 ++--- 2 files changed, 54 insertions(+), 55 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/3a99300a/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java -- diff --git a/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java b/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java index 4e05b89..a4a67d7 100644 --- a/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java +++ b/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java @@ -130,61 +130,63 @@ public class ShowTrace extends Basic { SetLong visited = tree.visit(new SpanTreeVisitor() { @Override public void visit(int level, RemoteSpan parent, RemoteSpan node, CollectionRemoteSpan children) { -sb.append(tr\n); -sb.append(String.format(td class='right'%d+/tdtd class='left'%d/td%n, node.stop - node.start, node.start - finalStart)); -sb.append(String.format(td style='text-indent: %dpx'%s@%s/td%n, level * 10, node.svc, node.sender)); -sb.append(td + node.description + /td); -boolean hasData = node.data != null !node.data.isEmpty(); -boolean hasAnnotations = node.annotations != null !node.annotations.isEmpty(); -if (hasData || hasAnnotations) { - String hexSpanId = Long.toHexString(node.spanId); - sb.append(tdinput type='checkbox' id=\); - sb.append(hexSpanId); - sb.append(checkboxIdSuffix); - sb.append(\ onclick='toggle(\ + Long.toHexString(node.spanId) + \)'/td\n); -} else { - sb.append(td/td\n); -} -sb.append(/tr\n); -sb.append(tr id=' + Long.toHexString(node.spanId) + ' style='display:none'); -sb.append(td colspan='5'\n); -if (hasData || hasAnnotations) { - sb.append( table class='indent,noborder'\n); - if (hasData) { -sb.append( trthKey/ththValue/th/tr\n); -for (EntryByteBuffer,ByteBuffer entry : node.data.entrySet()) { - String key = new String(entry.getKey().array(), entry.getKey().arrayOffset(), entry.getKey().limit(), UTF_8); - String value = new String(entry.getValue().array(), entry.getValue().arrayOffset(), entry.getValue().limit(), UTF_8); - sb.append( trtd + BasicServlet.sanitize(key) + /td); - sb.append(td + BasicServlet.sanitize(value) + /td/tr\n); -} - } - if (hasAnnotations) { -sb.append( trthAnnotation/ththTime Offset/th/tr\n); -for (Annotation entry : node.annotations) { - sb.append( trtd + BasicServlet.sanitize(entry.getMsg()) + /td); - sb.append(String.format(td%d/td/tr\n, entry.getTime() - finalStart)); -} - } - sb.append( /table); -} -sb.append(/td\n); -sb.append(/tr\n); +appendRow(sb, level, node, finalStart); } }); tree.nodes.keySet().removeAll(visited); if (!tree.nodes.isEmpty()) { - sb.append(span type='warn'Warning: the following spans are not rooted!/span\n); - sb.append(ul\n); + sb.append(trtd colspan=10The following spans are not rooted (probably due to a parent span of length 0ms):td/tr\n); for (RemoteSpan span : TraceDump.sortByStart(tree.nodes.values())) { -sb.append(String.format(li%s %s %s/li\n, Long.toHexString(span.spanId), Long.toHexString(span.parentId), span.description)); +appendRow(sb, 0, span, finalStart); } - sb.append(/ul\n); } sb.append(/table\n); sb.append(/div\n); } + private static void appendRow(StringBuilder sb, int level, RemoteSpan node, long finalStart) { +sb.append(tr\n); +sb.append(String.format(td class='right'%d+/tdtd class='left'%d/td%n, node.stop - node.start, node.start - finalStart)); +sb.append(String.format(td style='text-indent: %dpx'%s
[1/3] accumulo git commit: ACCUMULO-3715 add configurable sampling to internal tracing
Repository: accumulo Updated Branches: refs/heads/master 9c8dcaf0a - 3a99300a2 ACCUMULO-3715 add configurable sampling to internal tracing Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/281c2f32 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/281c2f32 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/281c2f32 Branch: refs/heads/master Commit: 281c2f3243bf3f90043ca8f17f7bed6a01b60297 Parents: 9c8dcaf Author: Billie Rinaldi billie.rina...@gmail.com Authored: Wed Apr 8 08:35:32 2015 -0700 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Fri Apr 10 10:09:28 2015 -0700 -- .../org/apache/accumulo/core/conf/Property.java | 7 + .../accumulo/core/trace/ProbabilitySampler.java | 27 .../accumulo/gc/SimpleGarbageCollector.java | 4 +-- .../master/replication/ReplicationDriver.java | 4 +-- .../replication/AccumuloReplicaSystem.java | 5 +++- .../tserver/tablet/MinorCompactionTask.java | 8 -- .../apache/accumulo/tserver/tablet/Tablet.java | 10 +--- 7 files changed, 55 insertions(+), 10 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/281c2f32/core/src/main/java/org/apache/accumulo/core/conf/Property.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/Property.java b/core/src/main/java/org/apache/accumulo/core/conf/Property.java index 9dceb1e..f9d8662 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/Property.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/Property.java @@ -275,6 +275,10 @@ public enum Property { The maximum number of concurrent major compactions for a tablet server), TSERV_MINC_MAXCONCURRENT(tserver.compaction.minor.concurrent.max, 4, PropertyType.COUNT, The maximum number of concurrent minor compactions for a tablet server), + TSERV_MAJC_TRACE_PERCENT(tserver.compaction.major.trace.percent, 0.1, PropertyType.FRACTION, + The percent of major compactions to trace), + TSERV_MINC_TRACE_PERCENT(tserver.compaction.minor.trace.percent, 0.1, PropertyType.FRACTION, + The percent of minor compactions to trace), TSERV_COMPACTION_WARN_TIME(tserver.compaction.warn.time, 10m, PropertyType.TIMEDURATION, When a compaction has not made progress for this time period, a warning will be logged), TSERV_BLOOM_LOAD_MAXCONCURRENT(tserver.bloom.load.concurrent.max, 4, PropertyType.COUNT, @@ -344,6 +348,7 @@ public enum Property { GC_DELETE_THREADS(gc.threads.delete, 16, PropertyType.COUNT, The number of threads used to delete files), GC_TRASH_IGNORE(gc.trash.ignore, false, PropertyType.BOOLEAN, Do not use the Trash, even if it is configured), GC_FILE_ARCHIVE(gc.file.archive, false, PropertyType.BOOLEAN, Archive any files/directories instead of moving to the HDFS trash or deleting), + GC_TRACE_PERCENT(gc.trace.percent, 0.01, PropertyType.FRACTION, Percent of gc cycles to trace), // properties that are specific to the monitor server behavior MONITOR_PREFIX(monitor., null, PropertyType.PREFIX, Properties in this category affect the behavior of the monitor web server.), @@ -556,6 +561,8 @@ public enum Property { Amount of time to wait before first checking for replication work, not useful outside of tests), REPLICATION_WORK_PROCESSOR_PERIOD(replication.work.processor.period, 0s, PropertyType.TIMEDURATION, Amount of time to wait before re-checking for replication work, not useful outside of tests), + REPLICATION_TRACE_PERCENT(replication.trace.percent, 0.1, PropertyType.FRACTION, + The sampling percentage to use for replication traces), ; http://git-wip-us.apache.org/repos/asf/accumulo/blob/281c2f32/core/src/main/java/org/apache/accumulo/core/trace/ProbabilitySampler.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/trace/ProbabilitySampler.java b/core/src/main/java/org/apache/accumulo/core/trace/ProbabilitySampler.java new file mode 100644 index 000..ff02fd7 --- /dev/null +++ b/core/src/main/java/org/apache/accumulo/core/trace/ProbabilitySampler.java @@ -0,0 +1,27 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the License); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed
accumulo git commit: ACCUMULO-2629 increase indentation in hierarchical trace view
Repository: accumulo Updated Branches: refs/heads/master 0a5a5682d - 58aad7d5c ACCUMULO-2629 increase indentation in hierarchical trace view Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/58aad7d5 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/58aad7d5 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/58aad7d5 Branch: refs/heads/master Commit: 58aad7d5c489f5a1b85d819a8c7b2807fb4b063a Parents: 0a5a568 Author: Billie Rinaldi bil...@apache.org Authored: Tue Apr 7 10:10:12 2015 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Tue Apr 7 10:10:12 2015 -0700 -- .../java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/58aad7d5/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java -- diff --git a/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java b/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java index 4bf64fe..4e05b89 100644 --- a/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java +++ b/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java @@ -132,7 +132,7 @@ public class ShowTrace extends Basic { public void visit(int level, RemoteSpan parent, RemoteSpan node, CollectionRemoteSpan children) { sb.append(tr\n); sb.append(String.format(td class='right'%d+/tdtd class='left'%d/td%n, node.stop - node.start, node.start - finalStart)); -sb.append(String.format(td style='text-indent: %dpx'%s@%s/td%n, level * 5, node.svc, node.sender)); +sb.append(String.format(td style='text-indent: %dpx'%s@%s/td%n, level * 10, node.svc, node.sender)); sb.append(td + node.description + /td); boolean hasData = node.data != null !node.data.isEmpty(); boolean hasAnnotations = node.annotations != null !node.annotations.isEmpty();
svn commit: r946500 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/ 1.6/apidocs/
Author: billie Date: Mon Apr 6 15:17:28 2015 New Revision: 946500 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r946499, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r946499, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r946499, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r946499, websites/production/accumulo/content/1.6/apidocs/
svn commit: r1671548 - /accumulo/site/trunk/content/doap_Accumulo.rdf
Author: billie Date: Mon Apr 6 15:16:56 2015 New Revision: 1671548 URL: http://svn.apache.org/r1671548 Log: add 1.6.2 to doap Modified: accumulo/site/trunk/content/doap_Accumulo.rdf Modified: accumulo/site/trunk/content/doap_Accumulo.rdf URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/doap_Accumulo.rdf?rev=1671548r1=1671547r2=1671548view=diff == --- accumulo/site/trunk/content/doap_Accumulo.rdf (original) +++ accumulo/site/trunk/content/doap_Accumulo.rdf Mon Apr 6 15:16:56 2015 @@ -36,6 +36,11 @@ release Version nameApache Accumulo/name +created2015-02-16/created +revision1.6.2/revision + /Version + Version +nameApache Accumulo/name created2014-09-24/created revision1.6.1/revision /Version
accumulo git commit: ACCUMULO-3606 make keystore and truststore types configurable for monitor SSL
Repository: accumulo Updated Branches: refs/heads/master 1c5bef324 - 76ce7d926 ACCUMULO-3606 make keystore and truststore types configurable for monitor SSL Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/76ce7d92 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/76ce7d92 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/76ce7d92 Branch: refs/heads/master Commit: 76ce7d9264addffcb940bbda000bf03efc95197f Parents: 1c5bef3 Author: Billie Rinaldi billie.rina...@gmail.com Authored: Mon Mar 2 16:34:17 2015 -0800 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Tue Mar 3 15:13:40 2015 -0800 -- .../org/apache/accumulo/core/conf/Property.java | 2 + .../accumulo/monitor/EmbeddedWebServer.java | 2 + .../accumulo/test/functional/MonitorSslIT.java | 128 +++ 3 files changed, 132 insertions(+) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/76ce7d92/core/src/main/java/org/apache/accumulo/core/conf/Property.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/Property.java b/core/src/main/java/org/apache/accumulo/core/conf/Property.java index 01f03cf..9ceebed 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/Property.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/Property.java @@ -355,9 +355,11 @@ public enum Property { MONITOR_SSL_KEYSTORE(monitor.ssl.keyStore, , PropertyType.PATH, The keystore for enabling monitor SSL.), @Sensitive MONITOR_SSL_KEYSTOREPASS(monitor.ssl.keyStorePassword, , PropertyType.STRING, The keystore password for enabling monitor SSL.), + MONITOR_SSL_KEYSTORETYPE(monitor.ssl.keyStoreType, , PropertyType.STRING, Type of SSL keystore), MONITOR_SSL_TRUSTSTORE(monitor.ssl.trustStore, , PropertyType.PATH, The truststore for enabling monitor SSL.), @Sensitive MONITOR_SSL_TRUSTSTOREPASS(monitor.ssl.trustStorePassword, , PropertyType.STRING, The truststore password for enabling monitor SSL.), + MONITOR_SSL_TRUSTSTORETYPE(monitor.ssl.trustStoreType, , PropertyType.STRING, Type of SSL truststore), MONITOR_SSL_INCLUDE_CIPHERS(monitor.ssl.include.ciphers, , PropertyType.STRING, A comma-separated list of allows SSL Ciphers, see monitor.ssl.exclude.ciphers to disallow ciphers), MONITOR_SSL_EXCLUDE_CIPHERS(monitor.ssl.exclude.ciphers, , PropertyType.STRING, http://git-wip-us.apache.org/repos/asf/accumulo/blob/76ce7d92/server/monitor/src/main/java/org/apache/accumulo/monitor/EmbeddedWebServer.java -- diff --git a/server/monitor/src/main/java/org/apache/accumulo/monitor/EmbeddedWebServer.java b/server/monitor/src/main/java/org/apache/accumulo/monitor/EmbeddedWebServer.java index 7142ef1..8c23a16 100644 --- a/server/monitor/src/main/java/org/apache/accumulo/monitor/EmbeddedWebServer.java +++ b/server/monitor/src/main/java/org/apache/accumulo/monitor/EmbeddedWebServer.java @@ -51,8 +51,10 @@ public class EmbeddedWebServer { SslContextFactory sslContextFactory = new SslContextFactory(); sslContextFactory.setKeyStorePath(conf.get(Property.MONITOR_SSL_KEYSTORE)); sslContextFactory.setKeyStorePassword(conf.get(Property.MONITOR_SSL_KEYSTOREPASS)); + sslContextFactory.setKeyStoreType(conf.get(Property.MONITOR_SSL_KEYSTORETYPE)); sslContextFactory.setTrustStorePath(conf.get(Property.MONITOR_SSL_TRUSTSTORE)); sslContextFactory.setTrustStorePassword(conf.get(Property.MONITOR_SSL_TRUSTSTOREPASS)); + sslContextFactory.setTrustStoreType(conf.get(Property.MONITOR_SSL_TRUSTSTORETYPE)); final String includedCiphers = conf.get(Property.MONITOR_SSL_INCLUDE_CIPHERS); if (!Property.MONITOR_SSL_INCLUDE_CIPHERS.getDefaultValue().equals(includedCiphers)) { http://git-wip-us.apache.org/repos/asf/accumulo/blob/76ce7d92/test/src/test/java/org/apache/accumulo/test/functional/MonitorSslIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/functional/MonitorSslIT.java b/test/src/test/java/org/apache/accumulo/test/functional/MonitorSslIT.java new file mode 100644 index 000..c112139 --- /dev/null +++ b/test/src/test/java/org/apache/accumulo/test/functional/MonitorSslIT.java @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the License); you may not use this file except in compliance with + * the License. You may
accumulo git commit: ACCUMULO-3528 upgrade htrace dependency to o.a.htrace 3.1.0-incubating
Repository: accumulo Updated Branches: refs/heads/master 50d351201 - 7e659af1a ACCUMULO-3528 upgrade htrace dependency to o.a.htrace 3.1.0-incubating Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/7e659af1 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/7e659af1 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/7e659af1 Branch: refs/heads/master Commit: 7e659af1ac439031a435c26a288efdda3950ea3e Parents: 50d3512 Author: Billie Rinaldi billie.rina...@gmail.com Authored: Tue Feb 17 11:51:24 2015 -0800 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Fri Feb 20 10:53:37 2015 -0800 -- assemble/pom.xml| 8 +- assemble/src/main/assemblies/component.xml | 2 +- core/pom.xml| 8 +- .../impl/TabletServerBatchReaderIterator.java | 2 +- .../accumulo/core/trace/CountSampler.java | 8 +- .../accumulo/core/trace/DistributedTrace.java | 40 ++- .../org/apache/accumulo/core/trace/Span.java| 21 ++-- .../org/apache/accumulo/core/trace/Trace.java | 28 ++--- .../org/apache/accumulo/core/trace/Tracer.java | 4 +- .../core/trace/wrappers/TraceCallable.java | 69 .../trace/wrappers/TraceExecutorService.java| 110 --- .../core/trace/wrappers/TraceRunnable.java | 79 - examples/simple/pom.xml | 4 + .../examples/simple/client/TracingExample.java | 6 +- pom.xml | 12 +- server/base/pom.xml | 4 + .../accumulo/server/client/BulkImporter.java| 2 +- server/master/pom.xml | 4 + .../accumulo/master/tableOps/BulkImport.java| 2 +- server/tracer/pom.xml | 8 +- .../accumulo/tracer/AsyncSpanReceiver.java | 41 --- .../accumulo/tracer/SendSpansViaThrift.java | 10 +- .../org/apache/accumulo/tracer/SpanTree.java| 2 +- .../org/apache/accumulo/tracer/TraceDump.java | 2 +- .../org/apache/accumulo/tracer/TraceServer.java | 2 +- .../apache/accumulo/tracer/ZooTraceClient.java | 33 +++--- .../org/apache/accumulo/tracer/TracerTest.java | 21 ++-- server/tserver/pom.xml | 4 + .../accumulo/tserver/CompactionQueue.java | 30 +++-- .../tserver/TabletServerResourceManager.java| 2 +- trace/pom.xml | 2 +- .../accumulo/trace/instrument/CountSampler.java | 8 +- .../accumulo/trace/instrument/Sampler.java | 2 +- .../apache/accumulo/trace/instrument/Span.java | 6 +- .../apache/accumulo/trace/instrument/Trace.java | 6 +- 35 files changed, 175 insertions(+), 417 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/7e659af1/assemble/pom.xml -- diff --git a/assemble/pom.xml b/assemble/pom.xml index 388a956..2a7156b 100644 --- a/assemble/pom.xml +++ b/assemble/pom.xml @@ -129,6 +129,10 @@ artifactIdcommons-vfs2/artifactId /dependency dependency + groupIdorg.apache.htrace/groupId + artifactIdhtrace-core/artifactId +/dependency +dependency groupIdorg.apache.thrift/groupId artifactIdlibthrift/artifactId /dependency @@ -157,10 +161,6 @@ artifactIdjetty-util/artifactId /dependency dependency - groupIdorg.htrace/groupId - artifactIdhtrace-core/artifactId -/dependency -dependency groupIdorg.slf4j/groupId artifactIdslf4j-api/artifactId /dependency http://git-wip-us.apache.org/repos/asf/accumulo/blob/7e659af1/assemble/src/main/assemblies/component.xml -- diff --git a/assemble/src/main/assemblies/component.xml b/assemble/src/main/assemblies/component.xml index 8dfce4d..a48023e 100644 --- a/assemble/src/main/assemblies/component.xml +++ b/assemble/src/main/assemblies/component.xml @@ -42,7 +42,7 @@ includeorg.eclipse.jetty:jetty-server/include includeorg.eclipse.jetty:jetty-servlet/include includeorg.eclipse.jetty:jetty-util/include -includeorg.htrace:htrace-core/include +includeorg.apache.htrace:htrace-core/include includeorg.slf4j:slf4j-api/include includeorg.slf4j:slf4j-log4j12/include /includes http://git-wip-us.apache.org/repos/asf/accumulo/blob/7e659af1/core/pom.xml -- diff --git a/core/pom.xml b/core/pom.xml index 67b486e..a736a3d 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -96,6 +96,10 @@ artifactIdhadoop-client/artifactId /dependency
accumulo git commit: ACCUMULO-3469 separate env setup needed for start and stop scripts
Repository: accumulo Updated Branches: refs/heads/master 30fad5a63 - e6145486d ACCUMULO-3469 separate env setup needed for start and stop scripts Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/e6145486 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/e6145486 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/e6145486 Branch: refs/heads/master Commit: e6145486dd27cc6cf6b484caf9efe675e152a455 Parents: 30fad5a Author: Billie Rinaldi bil...@apache.org Authored: Mon Jan 12 10:13:54 2015 -0800 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Jan 29 06:04:16 2015 -0800 -- assemble/bin/accumulo | 8 - assemble/bin/config-server.sh | 72 ++ assemble/bin/config.sh| 44 --- assemble/bin/start-all.sh | 1 + assemble/bin/start-here.sh| 1 + assemble/bin/start-server.sh | 1 + assemble/bin/stop-all.sh | 1 + 7 files changed, 76 insertions(+), 52 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/e6145486/assemble/bin/accumulo -- diff --git a/assemble/bin/accumulo b/assemble/bin/accumulo index e21e2fc..64b2f99 100755 --- a/assemble/bin/accumulo +++ b/assemble/bin/accumulo @@ -127,14 +127,6 @@ if [ -z ${ZOOKEEPER_HOME} -o ! -d ${ZOOKEEPER_HOME} ]; then echo ZOOKEEPER_HOME is not set or is not a directory. Please make sure it's set globally or in conf/accumulo-env.sh exit 1 fi -if [ -z ${ACCUMULO_LOG_DIR} ]; then - echo ACCUMULO_LOG_DIR is not set. Please make sure it's set globally or in conf/accumulo-env.sh - exit 1 -fi - -if [ ! -d ${ACCUMULO_LOG_DIR} ]; then - mkdir -p $ACCUMULO_LOG_DIR -fi # This is default for hadoop 2.x; # for another distribution, specify (DY)LD_LIBRARY_PATH http://git-wip-us.apache.org/repos/asf/accumulo/blob/e6145486/assemble/bin/config-server.sh -- diff --git a/assemble/bin/config-server.sh b/assemble/bin/config-server.sh new file mode 100755 index 000..eb3fd55 --- /dev/null +++ b/assemble/bin/config-server.sh @@ -0,0 +1,72 @@ +#! /usr/bin/env bash + +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the License); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an AS IS BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Guarantees that Accumulo and its environment variables are set. +# +# Parameters checked by script +# ACCUMULO_VERIFY_ONLY set to skip actions that would alter the local filesystem +# +# Values set by script that can be user provided. If not provided script attempts to infer. +# MONITORMachine to run monitor daemon on. Used by start-here.sh script +# +# Iff ACCUMULO_VERIFY_ONLY is not set, this script will +# * Check for standalone mode (lack of masters and slaves files) +# - Do appropriate set up +# * Ensure the presense of local role files (masters, slaves, gc, tracers) +# +# Values always set by script. +# SSHDefault ssh parameters used to start daemons +# + +unset MASTER1 +if [[ -f $ACCUMULO_CONF_DIR/masters ]]; then + MASTER1=$(egrep -v '(^#|^\s*$)' $ACCUMULO_CONF_DIR/masters | head -1) +fi + +if [[ -z ${MONITOR} ]] ; then + MONITOR=$MASTER1 + if [[ -f $ACCUMULO_CONF_DIR/monitor ]]; then + MONITOR=$(egrep -v '(^#|^\s*$)' $ACCUMULO_CONF_DIR/monitor | head -1) + fi + if [[ -z ${MONITOR} ]] ; then +echo Could not infer a Monitor role. You need to either define the MONITOR env variable, define \${ACCUMULO_CONF_DIR}/monitor\, or make sure \${ACCUMULO_CONF_DIR}/masters\ is non-empty. +exit 1 + fi +fi +if [[ ! -f $ACCUMULO_CONF_DIR/tracers -z ${ACCUMULO_VERIFY_ONLY} ]]; then + if [[ -z ${MASTER1} ]] ; then +echo Could not find a master node to use as a default for the tracer role. Either set up \${ACCUMULO_CONF_DIR}/tracers\ or make sure \${ACCUMULO_CONF_DIR}/masters\ is non-empty. +exit 1 + else +echo $MASTER1 $ACCUMULO_CONF_DIR/tracers + fi + +fi + +if [[ ! -f $ACCUMULO_CONF_DIR/gc -z ${ACCUMULO_VERIFY_ONLY} ]]; then + if [[ -z ${MASTER1} ]] ; then +echo
accumulo git commit: ACCUMULO-3507 remove TraceRunnable wrapper from thread pool threads and remove uses of the endThread method of stopping tracing
Repository: accumulo Updated Branches: refs/heads/master 2b2df4bd0 - 46d3fa9df ACCUMULO-3507 remove TraceRunnable wrapper from thread pool threads and remove uses of the endThread method of stopping tracing Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/46d3fa9d Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/46d3fa9d Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/46d3fa9d Branch: refs/heads/master Commit: 46d3fa9dfb90f615e53fdf4184760212b56cc7f4 Parents: 2b2df4b Author: Billie Rinaldi billie.rina...@gmail.com Authored: Fri Jan 23 09:50:26 2015 -0800 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Tue Jan 27 10:05:35 2015 -0800 -- .../accumulo/core/trace/wrappers/TraceCallable.java| 2 +- .../core/trace/wrappers/TraceExecutorService.java | 13 - .../accumulo/core/trace/wrappers/TraceRunnable.java| 2 +- .../apache/accumulo/core/util/NamingThreadFactory.java | 3 +-- .../org/apache/accumulo/trace/instrument/Trace.java| 6 -- 5 files changed, 7 insertions(+), 19 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/46d3fa9d/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceCallable.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceCallable.java b/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceCallable.java index d8b9b78..f6a2841 100644 --- a/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceCallable.java +++ b/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceCallable.java @@ -52,7 +52,7 @@ public class TraceCallableV implements CallableV { try { return impl.call(); } finally { -TraceExecutorService.endThread(chunk.getSpan()); +chunk.close(); } } else { return impl.call(); http://git-wip-us.apache.org/repos/asf/accumulo/blob/46d3fa9d/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceExecutorService.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceExecutorService.java b/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceExecutorService.java index 0ff5751..f2cf926 100644 --- a/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceExecutorService.java +++ b/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceExecutorService.java @@ -26,9 +26,6 @@ import java.util.concurrent.Future; import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeoutException; -import org.htrace.Span; -import org.htrace.Tracer; - public class TraceExecutorService implements ExecutorService { private final ExecutorService impl; @@ -110,14 +107,4 @@ public class TraceExecutorService implements ExecutorService { TimeoutException { return impl.invokeAny(wrapCollection(tasks), timeout, unit); } - - /** - * Finish a given trace and set the span for the current thread to null. - */ - public static void endThread(Span span) { -if (span != null) { - span.stop(); - Tracer.getInstance().continueSpan(null); -} - } } http://git-wip-us.apache.org/repos/asf/accumulo/blob/46d3fa9d/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceRunnable.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceRunnable.java b/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceRunnable.java index a870cbf..629db06 100644 --- a/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceRunnable.java +++ b/core/src/main/java/org/apache/accumulo/core/trace/wrappers/TraceRunnable.java @@ -51,7 +51,7 @@ public class TraceRunnable implements Runnable, ComparableTraceRunnable { try { runnable.run(); } finally { -TraceExecutorService.endThread(chunk.getSpan()); +chunk.close(); } } else { runnable.run(); http://git-wip-us.apache.org/repos/asf/accumulo/blob/46d3fa9d/core/src/main/java/org/apache/accumulo/core/util/NamingThreadFactory.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/util/NamingThreadFactory.java b/core/src/main/java/org/apache/accumulo/core/util/NamingThreadFactory.java index ebe9002..7a0b6a2 100644 --- a/core/src/main/java/org/apache/accumulo/core/util/NamingThreadFactory.java +++ b/core/src/main/java/org/apache/accumulo/core/util/NamingThreadFactory.java @@ -19,7 +19,6 @@ package org.apache.accumulo.core.util; import
accumulo git commit: ACCUMULO-3381 add accumulo-trace back into assembly
Repository: accumulo Updated Branches: refs/heads/master 5953f7845 - ca4e15b5d ACCUMULO-3381 add accumulo-trace back into assembly Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/ca4e15b5 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/ca4e15b5 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/ca4e15b5 Branch: refs/heads/master Commit: ca4e15b5d11c18459795ed36f9de222ed70193d7 Parents: 5953f78 Author: Billie Rinaldi billie.rina...@gmail.com Authored: Mon Dec 8 13:32:50 2014 -0800 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Mon Dec 8 13:32:50 2014 -0800 -- assemble/pom.xml | 4 1 file changed, 4 insertions(+) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/ca4e15b5/assemble/pom.xml -- diff --git a/assemble/pom.xml b/assemble/pom.xml index 0999555..c764091 100644 --- a/assemble/pom.xml +++ b/assemble/pom.xml @@ -114,6 +114,10 @@ /dependency dependency groupIdorg.apache.accumulo/groupId + artifactIdaccumulo-trace/artifactId +/dependency +dependency + groupIdorg.apache.accumulo/groupId artifactIdaccumulo-tracer/artifactId /dependency dependency
[14/15] accumulo git commit: ACCUMULO-898 refactoring and deprecation of old tracing
http://git-wip-us.apache.org/repos/asf/accumulo/blob/6b5f5ef4/core/src/main/java/org/apache/accumulo/core/client/impl/thrift/ClientService.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/client/impl/thrift/ClientService.java b/core/src/main/java/org/apache/accumulo/core/client/impl/thrift/ClientService.java index 638a8b7..d5d5b40 100644 --- a/core/src/main/java/org/apache/accumulo/core/client/impl/thrift/ClientService.java +++ b/core/src/main/java/org/apache/accumulo/core/client/impl/thrift/ClientService.java @@ -58,59 +58,59 @@ import org.slf4j.LoggerFactory; public String getZooKeepers() throws org.apache.thrift.TException; -public ListString bulkImportFiles(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, long tid, String tableId, ListString files, String errorDir, boolean setTime) throws ThriftSecurityException, ThriftTableOperationException, org.apache.thrift.TException; +public ListString bulkImportFiles(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, long tid, String tableId, ListString files, String errorDir, boolean setTime) throws ThriftSecurityException, ThriftTableOperationException, org.apache.thrift.TException; -public boolean isActive(org.apache.accumulo.trace.thrift.TInfo tinfo, long tid) throws org.apache.thrift.TException; +public boolean isActive(org.apache.accumulo.core.trace.thrift.TInfo tinfo, long tid) throws org.apache.thrift.TException; public void ping(org.apache.accumulo.core.security.thrift.TCredentials credentials) throws ThriftSecurityException, org.apache.thrift.TException; public ListTDiskUsage getDiskUsage(SetString tables, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws ThriftSecurityException, ThriftTableOperationException, org.apache.thrift.TException; -public SetString listLocalUsers(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws ThriftSecurityException, org.apache.thrift.TException; +public SetString listLocalUsers(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws ThriftSecurityException, org.apache.thrift.TException; -public void createLocalUser(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, String principal, ByteBuffer password) throws ThriftSecurityException, org.apache.thrift.TException; +public void createLocalUser(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, String principal, ByteBuffer password) throws ThriftSecurityException, org.apache.thrift.TException; -public void dropLocalUser(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, String principal) throws ThriftSecurityException, org.apache.thrift.TException; +public void dropLocalUser(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, String principal) throws ThriftSecurityException, org.apache.thrift.TException; -public void changeLocalUserPassword(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, String principal, ByteBuffer password) throws ThriftSecurityException, org.apache.thrift.TException; +public void changeLocalUserPassword(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, String principal, ByteBuffer password) throws ThriftSecurityException, org.apache.thrift.TException; -public boolean authenticate(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws ThriftSecurityException, org.apache.thrift.TException; +public boolean authenticate(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws ThriftSecurityException, org.apache.thrift.TException; -public boolean authenticateUser(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, org.apache.accumulo.core.security.thrift.TCredentials toAuth) throws ThriftSecurityException, org.apache.thrift.TException; +public boolean authenticateUser(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, org.apache.accumulo.core.security.thrift.TCredentials toAuth) throws ThriftSecurityException, org.apache.thrift.TException; -public void
[05/15] accumulo git commit: ACCUMULO-898 refactoring and deprecation of old tracing
http://git-wip-us.apache.org/repos/asf/accumulo/blob/6b5f5ef4/trace/src/main/java/org/apache/accumulo/trace/thrift/RemoteSpan.java -- diff --git a/trace/src/main/java/org/apache/accumulo/trace/thrift/RemoteSpan.java b/trace/src/main/java/org/apache/accumulo/trace/thrift/RemoteSpan.java deleted file mode 100644 index bfe183d..000 --- a/trace/src/main/java/org/apache/accumulo/trace/thrift/RemoteSpan.java +++ /dev/null @@ -1,1384 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the License); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an AS IS BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -/** - * Autogenerated by Thrift Compiler (0.9.1) - * - * DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING - * @generated - */ -package org.apache.accumulo.trace.thrift; - -import org.apache.thrift.scheme.IScheme; -import org.apache.thrift.scheme.SchemeFactory; -import org.apache.thrift.scheme.StandardScheme; - -import org.apache.thrift.scheme.TupleScheme; -import org.apache.thrift.protocol.TTupleProtocol; -import org.apache.thrift.protocol.TProtocolException; -import org.apache.thrift.EncodingUtils; -import org.apache.thrift.TException; -import org.apache.thrift.async.AsyncMethodCallback; -import org.apache.thrift.server.AbstractNonblockingServer.*; -import java.util.List; -import java.util.ArrayList; -import java.util.Map; -import java.util.HashMap; -import java.util.EnumMap; -import java.util.Set; -import java.util.HashSet; -import java.util.EnumSet; -import java.util.Collections; -import java.util.BitSet; -import java.nio.ByteBuffer; -import java.util.Arrays; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -@SuppressWarnings(all) public class RemoteSpan implements org.apache.thrift.TBaseRemoteSpan, RemoteSpan._Fields, java.io.Serializable, Cloneable, ComparableRemoteSpan { - private static final org.apache.thrift.protocol.TStruct STRUCT_DESC = new org.apache.thrift.protocol.TStruct(RemoteSpan); - - private static final org.apache.thrift.protocol.TField SENDER_FIELD_DESC = new org.apache.thrift.protocol.TField(sender, org.apache.thrift.protocol.TType.STRING, (short)1); - private static final org.apache.thrift.protocol.TField SVC_FIELD_DESC = new org.apache.thrift.protocol.TField(svc, org.apache.thrift.protocol.TType.STRING, (short)2); - private static final org.apache.thrift.protocol.TField TRACE_ID_FIELD_DESC = new org.apache.thrift.protocol.TField(traceId, org.apache.thrift.protocol.TType.I64, (short)3); - private static final org.apache.thrift.protocol.TField SPAN_ID_FIELD_DESC = new org.apache.thrift.protocol.TField(spanId, org.apache.thrift.protocol.TType.I64, (short)4); - private static final org.apache.thrift.protocol.TField PARENT_ID_FIELD_DESC = new org.apache.thrift.protocol.TField(parentId, org.apache.thrift.protocol.TType.I64, (short)5); - private static final org.apache.thrift.protocol.TField START_FIELD_DESC = new org.apache.thrift.protocol.TField(start, org.apache.thrift.protocol.TType.I64, (short)6); - private static final org.apache.thrift.protocol.TField STOP_FIELD_DESC = new org.apache.thrift.protocol.TField(stop, org.apache.thrift.protocol.TType.I64, (short)7); - private static final org.apache.thrift.protocol.TField DESCRIPTION_FIELD_DESC = new org.apache.thrift.protocol.TField(description, org.apache.thrift.protocol.TType.STRING, (short)8); - private static final org.apache.thrift.protocol.TField DATA_FIELD_DESC = new org.apache.thrift.protocol.TField(data, org.apache.thrift.protocol.TType.MAP, (short)10); - private static final org.apache.thrift.protocol.TField ANNOTATIONS_FIELD_DESC = new org.apache.thrift.protocol.TField(annotations, org.apache.thrift.protocol.TType.LIST, (short)11); - - private static final MapClass? extends IScheme, SchemeFactory schemes = new HashMapClass? extends IScheme, SchemeFactory(); - static { -schemes.put(StandardScheme.class, new RemoteSpanStandardSchemeFactory()); -schemes.put(TupleScheme.class, new RemoteSpanTupleSchemeFactory()); - } - - public String sender; // required - public String svc; // required - public long traceId; // required - public long spanId; // required - public long parentId; // required - public long start; // required - public long stop; //
[02/15] accumulo git commit: ACCUMULO-898 convert accumulo to use htrace
http://git-wip-us.apache.org/repos/asf/accumulo/blob/db2dda1b/server/master/src/main/java/org/apache/accumulo/master/replication/ReplicationDriver.java -- diff --git a/server/master/src/main/java/org/apache/accumulo/master/replication/ReplicationDriver.java b/server/master/src/main/java/org/apache/accumulo/master/replication/ReplicationDriver.java index a52f743..e3bbafa 100644 --- a/server/master/src/main/java/org/apache/accumulo/master/replication/ReplicationDriver.java +++ b/server/master/src/main/java/org/apache/accumulo/master/replication/ReplicationDriver.java @@ -25,7 +25,6 @@ import org.apache.accumulo.core.util.Daemon; import org.apache.accumulo.fate.util.UtilWaitThread; import org.apache.accumulo.master.Master; import org.apache.accumulo.trace.instrument.CountSampler; -import org.apache.accumulo.trace.instrument.Sampler; import org.apache.accumulo.trace.instrument.Trace; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -54,7 +53,7 @@ public class ReplicationDriver extends Daemon { @Override public void run() { -Sampler sampler = new CountSampler(10); +CountSampler sampler = new CountSampler(10); while (master.stillMaster()) { if (null == workMaker) { @@ -73,9 +72,7 @@ public class ReplicationDriver extends Daemon { rcrr = new RemoveCompleteReplicationRecords(conn); } - if (sampler.next()) { -Trace.on(masterReplicationDriver); - } + Trace.on(masterReplicationDriver, sampler); // Make status markers from replication records in metadata, removing entries in // metadata which are no longer needed (closed records) @@ -109,7 +106,7 @@ public class ReplicationDriver extends Daemon { log.error(Caught Exception trying to remove finished Replication records, e); } - Trace.offNoFlush(); + Trace.off(); // Sleep for a bit long sleepMillis = conf.getTimeInMillis(Property.MASTER_REPLICATION_SCAN_INTERVAL); http://git-wip-us.apache.org/repos/asf/accumulo/blob/db2dda1b/server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java -- diff --git a/server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java b/server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java index 49bb56d..7fe1af7 100644 --- a/server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java +++ b/server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java @@ -54,6 +54,7 @@ import org.apache.accumulo.core.util.ServerServices; import org.apache.accumulo.core.util.ServerServices.Service; import org.apache.accumulo.core.util.ThriftUtil; import org.apache.accumulo.core.util.UtilWaitThread; +import org.apache.accumulo.core.trace.DistributedTrace; import org.apache.accumulo.core.zookeeper.ZooUtil; import org.apache.accumulo.fate.zookeeper.ZooLock.LockLossReason; import org.apache.accumulo.fate.zookeeper.ZooUtil.NodeExistsPolicy; @@ -413,8 +414,12 @@ public class Monitor { config = new ServerConfigurationFactory(instance); Accumulo.init(fs, config, app); Monitor monitor = new Monitor(); -Accumulo.enableTracing(hostname, app); -monitor.run(hostname); +DistributedTrace.enable(hostname, app, config.getConfiguration()); +try { + monitor.run(hostname); +} finally { + DistributedTrace.disable(); +} } private static long START_TIME; http://git-wip-us.apache.org/repos/asf/accumulo/blob/db2dda1b/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java -- diff --git a/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java b/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java index a476201..8968088 100644 --- a/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java +++ b/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/ShowTrace.java @@ -17,7 +17,9 @@ package org.apache.accumulo.monitor.servlets.trace; import static java.lang.Math.min; +import static java.nio.charset.StandardCharsets.UTF_8; +import java.nio.ByteBuffer; import java.util.Collection; import java.util.Map.Entry; import java.util.Set; @@ -34,6 +36,7 @@ import org.apache.accumulo.core.trace.SpanTreeVisitor; import org.apache.accumulo.core.trace.TraceDump; import org.apache.accumulo.core.trace.TraceFormatter; import org.apache.accumulo.monitor.servlets.BasicServlet; +import org.apache.accumulo.trace.thrift.Annotation; import org.apache.accumulo.trace.thrift.RemoteSpan; import org.apache.hadoop.io.Text; @@ -115,7 +118,8 @@ public class ShowTrace extends Basic { sb.append(String.format(td style='text-indent: %dpx'%s@%s/td%n, level * 5, node.svc, node.sender));
[09/15] accumulo git commit: ACCUMULO-898 refactoring and deprecation of old tracing
http://git-wip-us.apache.org/repos/asf/accumulo/blob/6b5f5ef4/server/base/pom.xml -- diff --git a/server/base/pom.xml b/server/base/pom.xml index 60762be..c21a168 100644 --- a/server/base/pom.xml +++ b/server/base/pom.xml @@ -76,10 +76,6 @@ artifactIdaccumulo-start/artifactId /dependency dependency - groupIdorg.apache.accumulo/groupId - artifactIdaccumulo-trace/artifactId -/dependency -dependency groupIdorg.apache.hadoop/groupId artifactIdhadoop-client/artifactId /dependency http://git-wip-us.apache.org/repos/asf/accumulo/blob/6b5f5ef4/server/base/src/main/java/org/apache/accumulo/server/client/BulkImporter.java -- diff --git a/server/base/src/main/java/org/apache/accumulo/server/client/BulkImporter.java b/server/base/src/main/java/org/apache/accumulo/server/client/BulkImporter.java index 756d79b..a2ff172 100644 --- a/server/base/src/main/java/org/apache/accumulo/server/client/BulkImporter.java +++ b/server/base/src/main/java/org/apache/accumulo/server/client/BulkImporter.java @@ -54,6 +54,8 @@ import org.apache.accumulo.core.file.FileSKVIterator; import org.apache.accumulo.core.metadata.MetadataTable; import org.apache.accumulo.core.security.Credentials; import org.apache.accumulo.core.tabletserver.thrift.TabletClientService; +import org.apache.accumulo.core.trace.wrappers.TraceRunnable; +import org.apache.accumulo.core.trace.Tracer; import org.apache.accumulo.core.util.CachedConfiguration; import org.apache.accumulo.core.util.LoggingRunnable; import org.apache.accumulo.core.util.NamingThreadFactory; @@ -64,8 +66,6 @@ import org.apache.accumulo.server.conf.ServerConfigurationFactory; import org.apache.accumulo.server.fs.VolumeManager; import org.apache.accumulo.server.fs.VolumeManagerImpl; import org.apache.accumulo.server.util.FileUtil; -import org.apache.accumulo.trace.instrument.TraceRunnable; -import org.apache.accumulo.trace.instrument.Tracer; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; http://git-wip-us.apache.org/repos/asf/accumulo/blob/6b5f5ef4/server/base/src/main/java/org/apache/accumulo/server/client/ClientServiceHandler.java -- diff --git a/server/base/src/main/java/org/apache/accumulo/server/client/ClientServiceHandler.java b/server/base/src/main/java/org/apache/accumulo/server/client/ClientServiceHandler.java index 83d02db..d5349ba 100644 --- a/server/base/src/main/java/org/apache/accumulo/server/client/ClientServiceHandler.java +++ b/server/base/src/main/java/org/apache/accumulo/server/client/ClientServiceHandler.java @@ -54,6 +54,7 @@ import org.apache.accumulo.core.security.NamespacePermission; import org.apache.accumulo.core.security.SystemPermission; import org.apache.accumulo.core.security.TablePermission; import org.apache.accumulo.core.security.thrift.TCredentials; +import org.apache.accumulo.core.trace.thrift.TInfo; import org.apache.accumulo.server.conf.ServerConfigurationFactory; import org.apache.accumulo.server.fs.VolumeManager; import org.apache.accumulo.server.security.AuditedSecurityOperation; @@ -61,7 +62,6 @@ import org.apache.accumulo.server.security.SecurityOperation; import org.apache.accumulo.server.util.TableDiskUsage; import org.apache.accumulo.server.zookeeper.TransactionWatcher; import org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader; -import org.apache.accumulo.trace.thrift.TInfo; import org.apache.log4j.Logger; import org.apache.thrift.TException; http://git-wip-us.apache.org/repos/asf/accumulo/blob/6b5f5ef4/server/base/src/main/java/org/apache/accumulo/server/master/LiveTServerSet.java -- diff --git a/server/base/src/main/java/org/apache/accumulo/server/master/LiveTServerSet.java b/server/base/src/main/java/org/apache/accumulo/server/master/LiveTServerSet.java index e577153..3388b5b 100644 --- a/server/base/src/main/java/org/apache/accumulo/server/master/LiveTServerSet.java +++ b/server/base/src/main/java/org/apache/accumulo/server/master/LiveTServerSet.java @@ -34,6 +34,7 @@ import org.apache.accumulo.core.data.KeyExtent; import org.apache.accumulo.core.master.thrift.TabletServerStatus; import org.apache.accumulo.core.tabletserver.thrift.NotServingTabletException; import org.apache.accumulo.core.tabletserver.thrift.TabletClientService; +import org.apache.accumulo.core.trace.Tracer; import org.apache.accumulo.core.util.AddressUtil; import org.apache.accumulo.core.util.ServerServices; import org.apache.accumulo.core.util.ThriftUtil; @@ -45,7 +46,6 @@ import org.apache.accumulo.server.util.time.SimpleTimer; import org.apache.accumulo.server.zookeeper.ZooCache; import
[03/15] accumulo git commit: ACCUMULO-898 convert accumulo to use htrace
ACCUMULO-898 convert accumulo to use htrace Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/db2dda1b Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/db2dda1b Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/db2dda1b Branch: refs/heads/master Commit: db2dda1b7b6431a2ee57148db2de74c0432b9480 Parents: 34ca056 Author: Billie Rinaldi bil...@apache.org Authored: Tue Nov 4 10:16:11 2014 -0800 Committer: Billie Rinaldi bil...@apache.org Committed: Fri Nov 7 14:32:04 2014 -0800 -- assemble/bin/stop-all.sh| 2 +- assemble/pom.xml| 4 + assemble/src/main/assemblies/component.xml | 1 + core/pom.xml| 4 + .../core/client/ClientConfiguration.java| 34 +- .../org/apache/accumulo/core/conf/Property.java | 4 + .../apache/accumulo/core/conf/PropertyType.java | 3 + .../accumulo/core/trace/AsyncSpanReceiver.java | 173 .../accumulo/core/trace/DistributedTrace.java | 208 - .../accumulo/core/trace/SendSpansViaThrift.java | 97 +++ .../apache/accumulo/core/trace/TraceDump.java | 3 +- .../accumulo/core/trace/TraceFormatter.java | 17 +- .../accumulo/core/trace/ZooTraceClient.java | 78 +- .../apache/accumulo/core/util/ThriftUtil.java | 7 +- .../main/asciidoc/chapters/administration.txt | 106 ++- docs/src/main/resources/distributedTracing.html | 13 +- .../examples/simple/client/TracingExample.java | 29 +- .../minicluster/MiniAccumuloInstance.java | 2 +- pom.xml | 6 + .../org/apache/accumulo/server/Accumulo.java| 9 - .../apache/accumulo/server/init/Initialize.java | 1 - .../server/trace/TraceFSDataInputStream.java| 90 -- .../accumulo/server/trace/TraceFileSystem.java | 818 --- .../accumulo/server/util/AccumuloStatus.java| 3 +- .../org/apache/accumulo/server/util/ZooZap.java | 7 +- .../accumulo/gc/SimpleGarbageCollector.java | 17 +- .../java/org/apache/accumulo/master/Master.java | 5 +- .../master/replication/ReplicationDriver.java | 9 +- .../org/apache/accumulo/monitor/Monitor.java| 9 +- .../monitor/servlets/trace/ShowTrace.java | 26 +- .../accumulo/monitor/ShowTraceLinkTypeTest.java | 60 +- .../org/apache/accumulo/tracer/TraceServer.java | 21 +- .../tserver/BulkFailedCopyProcessor.java| 7 +- .../apache/accumulo/tserver/InMemoryMap.java| 5 +- .../apache/accumulo/tserver/TabletServer.java | 5 +- .../replication/AccumuloReplicaSystem.java | 2 +- .../apache/accumulo/tserver/tablet/Tablet.java | 7 +- .../java/org/apache/accumulo/shell/Shell.java | 5 +- .../accumulo/shell/commands/TraceCommand.java | 2 +- .../org/apache/accumulo/test/TestIngest.java| 12 +- .../org/apache/accumulo/test/VerifyIngest.java | 9 +- .../accumulo/test/ConditionalWriterIT.java | 3 +- .../accumulo/test/functional/BulkFileIT.java| 3 +- .../accumulo/test/functional/ExamplesIT.java| 2 +- .../accumulo/test/functional/SimpleMacIT.java | 18 +- trace/pom.xml | 4 + .../trace/instrument/CloudtraceSpan.java| 71 ++ .../accumulo/trace/instrument/CountSampler.java | 18 +- .../accumulo/trace/instrument/Sampler.java | 6 +- .../apache/accumulo/trace/instrument/Span.java | 213 +++-- .../apache/accumulo/trace/instrument/Trace.java | 133 ++- .../trace/instrument/TraceCallable.java | 24 +- .../trace/instrument/TraceExecutorService.java | 12 + .../accumulo/trace/instrument/TraceProxy.java | 72 -- .../trace/instrument/TraceRunnable.java | 20 +- .../accumulo/trace/instrument/Tracer.java | 127 +-- .../trace/instrument/impl/MilliSpan.java| 141 .../trace/instrument/impl/NullSpan.java | 102 --- .../trace/instrument/impl/RootMilliSpan.java| 43 - .../instrument/receivers/AsyncSpanReceiver.java | 132 --- .../trace/instrument/receivers/LogSpans.java| 63 -- .../receivers/SendSpansViaThrift.java | 89 -- .../instrument/receivers/SpanReceiver.java | 28 - .../instrument/receivers/ZooSpanClient.java | 122 --- .../accumulo/trace/thrift/Annotation.java | 502 .../accumulo/trace/thrift/RemoteSpan.java | 224 - trace/src/main/thrift/trace.thrift | 8 +- .../accumulo/trace/instrument/TracerTest.java | 63 +- 68 files changed, 1983 insertions(+), 2180 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/db2dda1b/assemble/bin/stop-all.sh -- diff --git a/assemble/bin/stop-all.sh b/assemble/bin/stop-all.sh index 4bf06c0
[13/15] accumulo git commit: ACCUMULO-898 refactoring and deprecation of old tracing
http://git-wip-us.apache.org/repos/asf/accumulo/blob/6b5f5ef4/core/src/main/java/org/apache/accumulo/core/conf/Property.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/Property.java b/core/src/main/java/org/apache/accumulo/core/conf/Property.java index fe313c1..025a233 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/Property.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/Property.java @@ -330,7 +330,7 @@ public enum Property { + the date shown on the 'Recent Logs' monitor page), TRACE_PREFIX(trace., null, PropertyType.PREFIX, Properties in this category affect the behavior of distributed tracing.), - TRACE_SPAN_RECEIVERS(trace.span.receivers, org.apache.accumulo.core.trace.ZooTraceClient, PropertyType.CLASSNAMELIST, A list of span receiver classes to send trace spans), + TRACE_SPAN_RECEIVERS(trace.span.receivers, org.apache.accumulo.tracer.ZooTraceClient, PropertyType.CLASSNAMELIST, A list of span receiver classes to send trace spans), TRACE_SPAN_RECEIVER_PREFIX(trace.span.receiver., null, PropertyType.PREFIX, Prefix for span receiver configuration properties), TRACE_ZK_PATH(trace.span.receiver.zookeeper.path, Constants.ZTRACERS, PropertyType.STRING, The zookeeper node where tracers are registered), TRACE_PORT(trace.port.client, 12234, PropertyType.PORT, The listening port for the trace server), http://git-wip-us.apache.org/repos/asf/accumulo/blob/6b5f5ef4/core/src/main/java/org/apache/accumulo/core/gc/thrift/GCMonitorService.java -- diff --git a/core/src/main/java/org/apache/accumulo/core/gc/thrift/GCMonitorService.java b/core/src/main/java/org/apache/accumulo/core/gc/thrift/GCMonitorService.java index 9d7ba57..a744038 100644 --- a/core/src/main/java/org/apache/accumulo/core/gc/thrift/GCMonitorService.java +++ b/core/src/main/java/org/apache/accumulo/core/gc/thrift/GCMonitorService.java @@ -52,13 +52,13 @@ import org.slf4j.LoggerFactory; public interface Iface { -public GCStatus getStatus(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws org.apache.accumulo.core.client.impl.thrift.ThriftSecurityException, org.apache.thrift.TException; +public GCStatus getStatus(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws org.apache.accumulo.core.client.impl.thrift.ThriftSecurityException, org.apache.thrift.TException; } public interface AsyncIface { -public void getStatus(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, org.apache.thrift.async.AsyncMethodCallback resultHandler) throws org.apache.thrift.TException; +public void getStatus(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, org.apache.thrift.async.AsyncMethodCallback resultHandler) throws org.apache.thrift.TException; } @@ -82,13 +82,13 @@ import org.slf4j.LoggerFactory; super(iprot, oprot); } -public GCStatus getStatus(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws org.apache.accumulo.core.client.impl.thrift.ThriftSecurityException, org.apache.thrift.TException +public GCStatus getStatus(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws org.apache.accumulo.core.client.impl.thrift.ThriftSecurityException, org.apache.thrift.TException { send_getStatus(tinfo, credentials); return recv_getStatus(); } -public void send_getStatus(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws org.apache.thrift.TException +public void send_getStatus(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials) throws org.apache.thrift.TException { getStatus_args args = new getStatus_args(); args.setTinfo(tinfo); @@ -127,7 +127,7 @@ import org.slf4j.LoggerFactory; super(protocolFactory, clientManager, transport); } -public void getStatus(org.apache.accumulo.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, org.apache.thrift.async.AsyncMethodCallback resultHandler) throws org.apache.thrift.TException { +public void getStatus(org.apache.accumulo.core.trace.thrift.TInfo tinfo, org.apache.accumulo.core.security.thrift.TCredentials credentials, org.apache.thrift.async.AsyncMethodCallback resultHandler) throws org.apache.thrift.TException { checkReady(); getStatus_call method_call = new
[08/15] accumulo git commit: ACCUMULO-898 refactoring and deprecation of old tracing
http://git-wip-us.apache.org/repos/asf/accumulo/blob/6b5f5ef4/server/tracer/src/main/java/org/apache/accumulo/tracer/thrift/Annotation.java -- diff --git a/server/tracer/src/main/java/org/apache/accumulo/tracer/thrift/Annotation.java b/server/tracer/src/main/java/org/apache/accumulo/tracer/thrift/Annotation.java new file mode 100644 index 000..607 --- /dev/null +++ b/server/tracer/src/main/java/org/apache/accumulo/tracer/thrift/Annotation.java @@ -0,0 +1,502 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the License); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/** + * Autogenerated by Thrift Compiler (0.9.1) + * + * DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING + * @generated + */ +package org.apache.accumulo.tracer.thrift; + +import org.apache.thrift.scheme.IScheme; +import org.apache.thrift.scheme.SchemeFactory; +import org.apache.thrift.scheme.StandardScheme; + +import org.apache.thrift.scheme.TupleScheme; +import org.apache.thrift.protocol.TTupleProtocol; +import org.apache.thrift.protocol.TProtocolException; +import org.apache.thrift.EncodingUtils; +import org.apache.thrift.TException; +import org.apache.thrift.async.AsyncMethodCallback; +import org.apache.thrift.server.AbstractNonblockingServer.*; +import java.util.List; +import java.util.ArrayList; +import java.util.Map; +import java.util.HashMap; +import java.util.EnumMap; +import java.util.Set; +import java.util.HashSet; +import java.util.EnumSet; +import java.util.Collections; +import java.util.BitSet; +import java.nio.ByteBuffer; +import java.util.Arrays; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +@SuppressWarnings(all) public class Annotation implements org.apache.thrift.TBaseAnnotation, Annotation._Fields, java.io.Serializable, Cloneable, ComparableAnnotation { + private static final org.apache.thrift.protocol.TStruct STRUCT_DESC = new org.apache.thrift.protocol.TStruct(Annotation); + + private static final org.apache.thrift.protocol.TField TIME_FIELD_DESC = new org.apache.thrift.protocol.TField(time, org.apache.thrift.protocol.TType.I64, (short)1); + private static final org.apache.thrift.protocol.TField MSG_FIELD_DESC = new org.apache.thrift.protocol.TField(msg, org.apache.thrift.protocol.TType.STRING, (short)2); + + private static final MapClass? extends IScheme, SchemeFactory schemes = new HashMapClass? extends IScheme, SchemeFactory(); + static { +schemes.put(StandardScheme.class, new AnnotationStandardSchemeFactory()); +schemes.put(TupleScheme.class, new AnnotationTupleSchemeFactory()); + } + + public long time; // required + public String msg; // required + + /** The set of fields this struct contains, along with convenience methods for finding and manipulating them. */ + @SuppressWarnings(all) public enum _Fields implements org.apache.thrift.TFieldIdEnum { +TIME((short)1, time), +MSG((short)2, msg); + +private static final MapString, _Fields byName = new HashMapString, _Fields(); + +static { + for (_Fields field : EnumSet.allOf(_Fields.class)) { +byName.put(field.getFieldName(), field); + } +} + +/** + * Find the _Fields constant that matches fieldId, or null if its not found. + */ +public static _Fields findByThriftId(int fieldId) { + switch(fieldId) { +case 1: // TIME + return TIME; +case 2: // MSG + return MSG; +default: + return null; + } +} + +/** + * Find the _Fields constant that matches fieldId, throwing an exception + * if it is not found. + */ +public static _Fields findByThriftIdOrThrow(int fieldId) { + _Fields fields = findByThriftId(fieldId); + if (fields == null) throw new IllegalArgumentException(Field + fieldId + doesn't exist!); + return fields; +} + +/** + * Find the _Fields constant that matches name, or null if its not found. + */ +public static _Fields findByName(String name) { + return byName.get(name); +} + +private final short _thriftId; +private final String _fieldName; + +_Fields(short thriftId, String fieldName) { + _thriftId = thriftId; + _fieldName =
svn commit: r1637487 - /accumulo/site/trunk/content/people.mdtext
Author: billie Date: Fri Nov 7 23:54:51 2014 New Revision: 1637487 URL: http://svn.apache.org/r1637487 Log: added jim fiori to contributors Modified: accumulo/site/trunk/content/people.mdtext Modified: accumulo/site/trunk/content/people.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/people.mdtext?rev=1637487r1=1637486r2=1637487view=diff == --- accumulo/site/trunk/content/people.mdtext (original) +++ accumulo/site/trunk/content/people.mdtext Fri Nov 7 23:54:51 2014 @@ -68,6 +68,7 @@ Contributors trtdedwardyoon/tdtdEdward Yoon/tdtd/tdtd/td/tr trtd/tdtdGary Singh/tdtda href=http://www.sabre-eng.com;Sabre Engineering/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtd/tdtdHung Pham/tdtda href=http://www.cloudera.com/;Cloudera/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr + trtd/tdtdJames Fiori/tdtda href=http://flywheeldata.com/;Flywheel Data/a/tdtdET (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr trtdjaredwinick/tdtdJared Winick/tdtda href=http://www.koverse.com;Koverse/a/tdtdMT (a href=http://www.timeanddate.com/library/abbreviations/timezones/na/mst.html-7/a / a href=http://www.timeanddate.com/library/abbreviations/timezones/na/mdt.html-6/a)/td/tr trtd/tdtdJason Then/tdtd/tdtd/td/tr trtd/tdtdJay Shipper/tdtd/tdtd/td/tr
svn commit: r928334 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/ 1.6/apidocs/
Author: billie Date: Fri Nov 7 23:55:32 2014 New Revision: 928334 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r928333, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r928333, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r928333, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r928333, websites/production/accumulo/content/1.6/apidocs/
git commit: ACCUMULO-3110 converted ConditionalWriterIT to use defaultTimeoutSeconds
Repository: accumulo Updated Branches: refs/heads/1.6.1-SNAPSHOT 8a1389211 - 279614529 ACCUMULO-3110 converted ConditionalWriterIT to use defaultTimeoutSeconds Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/27961452 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/27961452 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/27961452 Branch: refs/heads/1.6.1-SNAPSHOT Commit: 27961452991f25854890db142f22a49bd10c8eec Parents: 8a13892 Author: Billie Rinaldi bil...@apache.org Authored: Thu Sep 11 15:37:16 2014 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Sep 11 15:59:42 2014 -0700 -- .../accumulo/test/ConditionalWriterIT.java | 41 +++- 1 file changed, 23 insertions(+), 18 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/27961452/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java b/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java index 6d10f10..477f492 100644 --- a/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java +++ b/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java @@ -89,7 +89,12 @@ import org.junit.Test; * */ public class ConditionalWriterIT extends SimpleMacIT { - + + @Override + protected int defaultTimeoutSeconds() { +return 60; + } + public static long abs(long l) { l = Math.abs(l); // abs(Long.MIN_VALUE) == Long.MIN_VALUE... if (l 0) @@ -97,7 +102,7 @@ public class ConditionalWriterIT extends SimpleMacIT { return l; } - @Test(timeout = 60 * 1000) + @Test public void testBasic() throws Exception { Connector conn = getConnector(); @@ -175,7 +180,7 @@ public class ConditionalWriterIT extends SimpleMacIT { Assert.assertEquals(doe, scanner.iterator().next().getValue().toString()); } - @Test(timeout = 60 * 1000) + @Test public void testFields() throws Exception { Connector conn = getConnector(); @@ -257,7 +262,7 @@ public class ConditionalWriterIT extends SimpleMacIT { } - @Test(timeout = 60 * 1000) + @Test public void testBadColVis() throws Exception { // test when a user sets a col vis in a condition that can never be seen @@ -354,7 +359,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw2.close(); } - @Test(timeout = 60 * 1000) + @Test public void testConstraints() throws Exception { // ensure constraint violations are properly reported @@ -384,7 +389,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testIterators() throws Exception { Connector conn = getConnector(); @@ -480,7 +485,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testBatch() throws Exception { Connector conn = getConnector(); @@ -585,7 +590,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testBigBatch() throws Exception { Connector conn = getConnector(); @@ -658,7 +663,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testBatchErrors() throws Exception { Connector conn = getConnector(); @@ -738,7 +743,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testSameRow() throws Exception { // test multiple mutations for same row in same batch @@ -923,7 +928,7 @@ public class ConditionalWriterIT extends SimpleMacIT { } } - @Test(timeout = 60 * 1000) + @Test public void testThreads() throws Exception { // test multiple threads using a single conditional writer @@ -1002,7 +1007,7 @@ public class ConditionalWriterIT extends SimpleMacIT { return ret; } - @Test(timeout = 60 * 1000) + @Test public void testSecurity() throws Exception { // test against table user does not have read and/or write permissions for Connector conn = getConnector(); @@ -1045,7 +1050,7 @@ public class ConditionalWriterIT extends SimpleMacIT { } } - @Test(timeout = 60 * 1000) + @Test public void testTimeout() throws Exception { Connector conn = getConnector(); @@ -1094,7 +1099,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testDeleteTable() throws Exception { String table = getUniqueNames(1)[0
git commit: ACCUMULO-3111 added some retries when checking the number of splits after merging in MetaSplitIT
Repository: accumulo Updated Branches: refs/heads/1.6.1-SNAPSHOT 279614529 - 0d76cd520 ACCUMULO-3111 added some retries when checking the number of splits after merging in MetaSplitIT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/0d76cd52 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/0d76cd52 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/0d76cd52 Branch: refs/heads/1.6.1-SNAPSHOT Commit: 0d76cd5205250469d30cb19f3af73090c295fe70 Parents: 2796145 Author: Billie Rinaldi bil...@apache.org Authored: Thu Sep 11 16:07:29 2014 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Sep 11 16:07:29 2014 -0700 -- .../org/apache/accumulo/test/MetaSplitIT.java | 28 +++- 1 file changed, 21 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/0d76cd52/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java b/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java index 4ea2663..8a655dd 100644 --- a/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java +++ b/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java @@ -22,6 +22,8 @@ import java.util.SortedSet; import java.util.TreeSet; import org.apache.accumulo.core.client.AccumuloException; +import org.apache.accumulo.core.client.AccumuloSecurityException; +import org.apache.accumulo.core.client.TableNotFoundException; import org.apache.accumulo.core.client.admin.TableOperations; import org.apache.accumulo.core.metadata.MetadataTable; import org.apache.accumulo.core.metadata.RootTable; @@ -53,24 +55,36 @@ public class MetaSplitIT extends SimpleMacIT { opts.addSplits(MetadataTable.NAME, splits); } - @Test(timeout = 6) + @Test(timeout = 18) public void testMetadataTableSplit() throws Exception { TableOperations opts = getConnector().tableOperations(); for (int i = 1; i = 10; i++) { opts.create( + i); } opts.merge(MetadataTable.NAME, new Text(01), new Text(02)); -assertEquals(1, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(1, opts); addSplits(opts, 4 5 6 7 8.split( )); -assertEquals(6, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(6, opts); opts.merge(MetadataTable.NAME, new Text(6), new Text(9)); -assertEquals(4, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(4, opts); addSplits(opts, 44 55 66 77 88.split( )); -assertEquals(9, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(9, opts); opts.merge(MetadataTable.NAME, new Text(5), new Text(7)); -assertEquals(6, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(6, opts); opts.merge(MetadataTable.NAME, null, null); -assertEquals(0, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(0, opts); + } + + private static void checkMetadataSplits(int numSplits, TableOperations opts) + throws AccumuloSecurityException, TableNotFoundException, + AccumuloException, InterruptedException { +for (int i = 0; i 10; i++) { + if (opts.listSplits(MetadataTable.NAME).size() == numSplits) { +break; + } + Thread.sleep(2000); +} +assertEquals(numSplits, opts.listSplits(MetadataTable.NAME).size()); } }
[3/3] git commit: Merge branch '1.6.1-SNAPSHOT'
Merge branch '1.6.1-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/abf966e1 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/abf966e1 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/abf966e1 Branch: refs/heads/master Commit: abf966e19649d75c9774aa8d0bf6fcb4cd545005 Parents: c196a26 0d76cd5 Author: Billie Rinaldi bil...@apache.org Authored: Thu Sep 11 16:08:39 2014 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Sep 11 16:08:39 2014 -0700 -- .../accumulo/test/ConditionalWriterIT.java | 41 +++- .../org/apache/accumulo/test/MetaSplitIT.java | 28 + 2 files changed, 44 insertions(+), 25 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/abf966e1/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java -- diff --cc test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java index 6951884,477f492..3bf9284 --- a/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java +++ b/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java @@@ -1199,9 -1204,10 +1204,9 @@@ public class ConditionalWriterIT extend cw.write(cm1); } - @Test(timeout = 60 * 1000) + @Test public void testTrace() throws Exception { -Process tracer = null; Connector conn = getConnector(); if (!conn.tableOperations().exists(trace)) { getStaticCluster().exec(TraceServer.class);
[1/3] git commit: ACCUMULO-3110 converted ConditionalWriterIT to use defaultTimeoutSeconds
Repository: accumulo Updated Branches: refs/heads/master c196a267c - abf966e19 ACCUMULO-3110 converted ConditionalWriterIT to use defaultTimeoutSeconds Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/27961452 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/27961452 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/27961452 Branch: refs/heads/master Commit: 27961452991f25854890db142f22a49bd10c8eec Parents: 8a13892 Author: Billie Rinaldi bil...@apache.org Authored: Thu Sep 11 15:37:16 2014 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Sep 11 15:59:42 2014 -0700 -- .../accumulo/test/ConditionalWriterIT.java | 41 +++- 1 file changed, 23 insertions(+), 18 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/27961452/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java b/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java index 6d10f10..477f492 100644 --- a/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java +++ b/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java @@ -89,7 +89,12 @@ import org.junit.Test; * */ public class ConditionalWriterIT extends SimpleMacIT { - + + @Override + protected int defaultTimeoutSeconds() { +return 60; + } + public static long abs(long l) { l = Math.abs(l); // abs(Long.MIN_VALUE) == Long.MIN_VALUE... if (l 0) @@ -97,7 +102,7 @@ public class ConditionalWriterIT extends SimpleMacIT { return l; } - @Test(timeout = 60 * 1000) + @Test public void testBasic() throws Exception { Connector conn = getConnector(); @@ -175,7 +180,7 @@ public class ConditionalWriterIT extends SimpleMacIT { Assert.assertEquals(doe, scanner.iterator().next().getValue().toString()); } - @Test(timeout = 60 * 1000) + @Test public void testFields() throws Exception { Connector conn = getConnector(); @@ -257,7 +262,7 @@ public class ConditionalWriterIT extends SimpleMacIT { } - @Test(timeout = 60 * 1000) + @Test public void testBadColVis() throws Exception { // test when a user sets a col vis in a condition that can never be seen @@ -354,7 +359,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw2.close(); } - @Test(timeout = 60 * 1000) + @Test public void testConstraints() throws Exception { // ensure constraint violations are properly reported @@ -384,7 +389,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testIterators() throws Exception { Connector conn = getConnector(); @@ -480,7 +485,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testBatch() throws Exception { Connector conn = getConnector(); @@ -585,7 +590,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testBigBatch() throws Exception { Connector conn = getConnector(); @@ -658,7 +663,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testBatchErrors() throws Exception { Connector conn = getConnector(); @@ -738,7 +743,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testSameRow() throws Exception { // test multiple mutations for same row in same batch @@ -923,7 +928,7 @@ public class ConditionalWriterIT extends SimpleMacIT { } } - @Test(timeout = 60 * 1000) + @Test public void testThreads() throws Exception { // test multiple threads using a single conditional writer @@ -1002,7 +1007,7 @@ public class ConditionalWriterIT extends SimpleMacIT { return ret; } - @Test(timeout = 60 * 1000) + @Test public void testSecurity() throws Exception { // test against table user does not have read and/or write permissions for Connector conn = getConnector(); @@ -1045,7 +1050,7 @@ public class ConditionalWriterIT extends SimpleMacIT { } } - @Test(timeout = 60 * 1000) + @Test public void testTimeout() throws Exception { Connector conn = getConnector(); @@ -1094,7 +1099,7 @@ public class ConditionalWriterIT extends SimpleMacIT { cw.close(); } - @Test(timeout = 60 * 1000) + @Test public void testDeleteTable() throws Exception { String table = getUniqueNames(1)[0]; Connector
[2/3] git commit: ACCUMULO-3111 added some retries when checking the number of splits after merging in MetaSplitIT
ACCUMULO-3111 added some retries when checking the number of splits after merging in MetaSplitIT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/0d76cd52 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/0d76cd52 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/0d76cd52 Branch: refs/heads/master Commit: 0d76cd5205250469d30cb19f3af73090c295fe70 Parents: 2796145 Author: Billie Rinaldi bil...@apache.org Authored: Thu Sep 11 16:07:29 2014 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Sep 11 16:07:29 2014 -0700 -- .../org/apache/accumulo/test/MetaSplitIT.java | 28 +++- 1 file changed, 21 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/0d76cd52/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java b/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java index 4ea2663..8a655dd 100644 --- a/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java +++ b/test/src/test/java/org/apache/accumulo/test/MetaSplitIT.java @@ -22,6 +22,8 @@ import java.util.SortedSet; import java.util.TreeSet; import org.apache.accumulo.core.client.AccumuloException; +import org.apache.accumulo.core.client.AccumuloSecurityException; +import org.apache.accumulo.core.client.TableNotFoundException; import org.apache.accumulo.core.client.admin.TableOperations; import org.apache.accumulo.core.metadata.MetadataTable; import org.apache.accumulo.core.metadata.RootTable; @@ -53,24 +55,36 @@ public class MetaSplitIT extends SimpleMacIT { opts.addSplits(MetadataTable.NAME, splits); } - @Test(timeout = 6) + @Test(timeout = 18) public void testMetadataTableSplit() throws Exception { TableOperations opts = getConnector().tableOperations(); for (int i = 1; i = 10; i++) { opts.create( + i); } opts.merge(MetadataTable.NAME, new Text(01), new Text(02)); -assertEquals(1, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(1, opts); addSplits(opts, 4 5 6 7 8.split( )); -assertEquals(6, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(6, opts); opts.merge(MetadataTable.NAME, new Text(6), new Text(9)); -assertEquals(4, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(4, opts); addSplits(opts, 44 55 66 77 88.split( )); -assertEquals(9, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(9, opts); opts.merge(MetadataTable.NAME, new Text(5), new Text(7)); -assertEquals(6, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(6, opts); opts.merge(MetadataTable.NAME, null, null); -assertEquals(0, opts.listSplits(MetadataTable.NAME).size()); +checkMetadataSplits(0, opts); + } + + private static void checkMetadataSplits(int numSplits, TableOperations opts) + throws AccumuloSecurityException, TableNotFoundException, + AccumuloException, InterruptedException { +for (int i = 0; i 10; i++) { + if (opts.listSplits(MetadataTable.NAME).size() == numSplits) { +break; + } + Thread.sleep(2000); +} +assertEquals(numSplits, opts.listSplits(MetadataTable.NAME).size()); } }
git commit: ACCUMULO-3078 made some more ITs rely on a Connector where possible instead of a MAC instance
Repository: accumulo Updated Branches: refs/heads/1.6.1-SNAPSHOT 8147b196a - 4135e4f87 ACCUMULO-3078 made some more ITs rely on a Connector where possible instead of a MAC instance Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/4135e4f8 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/4135e4f8 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/4135e4f8 Branch: refs/heads/1.6.1-SNAPSHOT Commit: 4135e4f87c0fa4a967517f54ce67b77344ef735f Parents: 8147b19 Author: Billie Rinaldi billie.rina...@gmail.com Authored: Thu Aug 28 07:59:09 2014 -0700 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Thu Aug 28 07:59:09 2014 -0700 -- .../apache/accumulo/test/ConditionalWriterIT.java| 9 +++-- .../accumulo/test/MultiTableBatchWriterIT.java | 2 +- .../test/functional/AccumuloInputFormatIT.java | 15 +-- .../apache/accumulo/test/functional/BinaryIT.java| 15 ++- .../accumulo/test/functional/LogicalTimeIT.java | 2 +- .../apache/accumulo/test/functional/SimpleMacIT.java | 1 + 6 files changed, 29 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/4135e4f8/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java b/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java index 15c1fd3..6d10f10 100644 --- a/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java +++ b/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java @@ -1202,8 +1202,11 @@ public class ConditionalWriterIT extends SimpleMacIT { @Test(timeout = 60 * 1000) public void testTrace() throws Exception { -Process tracer = getStaticCluster().exec(TraceServer.class); +Process tracer = null; Connector conn = getConnector(); +if (!conn.tableOperations().exists(trace)) { + getStaticCluster().exec(TraceServer.class); +} while (!conn.tableOperations().exists(trace)) { UtilWaitThread.sleep(1000); } @@ -1250,6 +1253,8 @@ public class ConditionalWriterIT extends SimpleMacIT { break; } } -tracer.destroy(); +if (tracer != null) { + tracer.destroy(); +} } } http://git-wip-us.apache.org/repos/asf/accumulo/blob/4135e4f8/test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java b/test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java index 427b12f..f1eba61 100644 --- a/test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java +++ b/test/src/test/java/org/apache/accumulo/test/MultiTableBatchWriterIT.java @@ -60,7 +60,7 @@ public class MultiTableBatchWriterIT extends SimpleMacIT { } public MultiTableBatchWriter getMultiTableBatchWriter(long cacheTimeoutInSeconds) { -return new MultiTableBatchWriterImpl(connector.getInstance(), new Credentials(root, new PasswordToken(getStaticCluster().getConfig().getRootPassword())), +return new MultiTableBatchWriterImpl(connector.getInstance(), new Credentials(root, new PasswordToken(ROOT_PASSWORD)), new BatchWriterConfig(), cacheTimeoutInSeconds, TimeUnit.SECONDS); } http://git-wip-us.apache.org/repos/asf/accumulo/blob/4135e4f8/test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java -- diff --git a/test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java b/test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java index 17e319c..150222f 100644 --- a/test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java +++ b/test/src/test/java/org/apache/accumulo/test/functional/AccumuloInputFormatIT.java @@ -29,6 +29,8 @@ import java.util.TreeSet; import org.apache.accumulo.core.client.AccumuloException; import org.apache.accumulo.core.client.AccumuloSecurityException; import org.apache.accumulo.core.client.BatchWriter; +import org.apache.accumulo.core.client.ClientConfiguration; +import org.apache.accumulo.core.client.Connector; import org.apache.accumulo.core.client.TableExistsException; import org.apache.accumulo.core.client.TableNotFoundException; import org.apache.accumulo.core.client.mapreduce.AccumuloInputFormat; @@ -57,25 +59,26 @@ public class AccumuloInputFormatIT extends SimpleMacIT { */ @Test public void testGetSplits() throws IOException, AccumuloSecurityException, AccumuloException
[2/2] git commit: Merge branch '1.6.1-SNAPSHOT'
Merge branch '1.6.1-SNAPSHOT' Conflicts: test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/4a4f5463 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/4a4f5463 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/4a4f5463 Branch: refs/heads/master Commit: 4a4f5463cf2859773326c0db35386f7f63e16c53 Parents: 91186bf 4135e4f Author: Billie Rinaldi billie.rina...@gmail.com Authored: Thu Aug 28 08:03:06 2014 -0700 Committer: Billie Rinaldi billie.rina...@gmail.com Committed: Thu Aug 28 08:03:06 2014 -0700 -- .../apache/accumulo/test/ConditionalWriterIT.java| 9 +++-- .../accumulo/test/MultiTableBatchWriterIT.java | 2 +- .../test/functional/AccumuloInputFormatIT.java | 15 +-- .../apache/accumulo/test/functional/BinaryIT.java| 15 ++- .../accumulo/test/functional/LogicalTimeIT.java | 2 +- .../apache/accumulo/test/functional/SimpleMacIT.java | 1 + 6 files changed, 29 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/4a4f5463/test/src/test/java/org/apache/accumulo/test/ConditionalWriterIT.java -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/4a4f5463/test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java -- diff --cc test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java index ef9034b,9c0edaa..2f79dcb --- a/test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java +++ b/test/src/test/java/org/apache/accumulo/test/functional/BinaryIT.java @@@ -55,7 -60,7 +60,7 @@@ public class BinaryIT extends SimpleMac BatchWriterOpts bwOpts = new BatchWriterOpts(); ScannerOpts scanOpts = new ScannerOpts(); TestBinaryRows.Opts opts = new TestBinaryRows.Opts(); - opts.setTableName(bt); -opts.tableName = tableName; ++opts.setTableName(tableName); opts.start = 0; opts.num = 10; opts.mode = ingest;
svn commit: r1615836 - in /accumulo/site/trunk/content: ./ 1.4/apidocs/ 1.4/apidocs/org/ 1.4/apidocs/org/apache/ 1.4/apidocs/org/apache/accumulo/ 1.4/apidocs/org/apache/accumulo/cloudtrace/ 1.4/apidoc
Author: billie Date: Tue Aug 5 01:45:26 2014 New Revision: 1615836 URL: http://svn.apache.org/r1615836 Log: ACCUMULO-1532 repaired 1.4 javadocs [This commit notification would consist of 978 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]
svn commit: r918424 - in /websites/production/accumulo/content: ./ 1.5/apidocs/ 1.6/apidocs/
Author: billie Date: Tue Aug 5 01:57:57 2014 New Revision: 918424 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r918423, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.5/apidocs/ - copied from r918423, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r918423, websites/production/accumulo/content/1.6/apidocs/
svn commit: r1615839 - in /accumulo/site/trunk/content: 1.4/apidocs/ extpaths.txt
Author: billie Date: Tue Aug 5 02:00:16 2014 New Revision: 1615839 URL: http://svn.apache.org/r1615839 Log: ACCUMULO-1532 updated 1.4/apidocs extpaths Removed: accumulo/site/trunk/content/1.4/apidocs/ Modified: accumulo/site/trunk/content/extpaths.txt Modified: accumulo/site/trunk/content/extpaths.txt URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/extpaths.txt?rev=1615839r1=1615838r2=1615839view=diff == --- accumulo/site/trunk/content/extpaths.txt (original) +++ accumulo/site/trunk/content/extpaths.txt Tue Aug 5 02:00:16 2014 @@ -1,2 +1,3 @@ +1.4/apidocs 1.5/apidocs 1.6/apidocs
svn commit: r918426 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/ 1.6/apidocs/
Author: billie Date: Tue Aug 5 02:01:24 2014 New Revision: 918426 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r918425, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r918425, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r918425, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r918425, websites/production/accumulo/content/1.6/apidocs/
git commit: ACCUMULO-612 added information on monitor page to user manual
Repository: accumulo Updated Branches: refs/heads/master f783b4b5b - 6c7650327 ACCUMULO-612 added information on monitor page to user manual Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/6c765032 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/6c765032 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/6c765032 Branch: refs/heads/master Commit: 6c76503279096780e6b9b6ef974e7eec181cae29 Parents: f783b4b Author: Billie Rinaldi bil...@apache.org Authored: Thu Jun 12 17:25:52 2014 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Thu Jun 12 17:25:52 2014 -0700 -- .../main/asciidoc/chapters/administration.txt | 49 ++-- docs/src/main/asciidoc/chapters/design.txt | 2 +- 2 files changed, 46 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/6c765032/docs/src/main/asciidoc/chapters/administration.txt -- diff --git a/docs/src/main/asciidoc/chapters/administration.txt b/docs/src/main/asciidoc/chapters/administration.txt index b4c1a71..28030cb 100644 --- a/docs/src/main/asciidoc/chapters/administration.txt +++ b/docs/src/main/asciidoc/chapters/administration.txt @@ -214,8 +214,8 @@ Make sure ZooKeeper is configured and running on at least one machine in the cluster. Start Accumulo using the +bin/start-all.sh+ script. -To verify that Accumulo is running, check the Status page as described under -_Monitoring_. In addition, the Shell can provide some information about the status of +To verify that Accumulo is running, check the Status page as described in +monitoring. In addition, the Shell can provide some information about the status of tables via reading the metadata tables. Stopping Accumulo @@ -254,12 +254,53 @@ account for the removal of these hosts. Bear in mind that the monitor will not r slaves file automatically, so it will report the decomissioned servers as down; it's recommended that you restart the monitor so that the node list is up to date. +[[monitoring]] === Monitoring The Accumulo Master provides an interface for monitoring the status and health of -Accumulo components. This interface can be accessed by pointing a web browser to -+http://accumulomaster:50095/status+ +Accumulo components. The Accumulo Monitor provides a web UI for accessing this information at ++http://_monitorhost_:50095/+. +Things highlighted in yellow may be in need of attention. +If anything is highlighted in red on the monitor page, it is something that definitely needs attention. + +The Overview page contains some summary information about the Accumulo instance, including the version, instance name, and instance ID. +There is a table labeled Accumulo Master with current status, a table listing the active Zookeeper servers, and graphs displaying various metrics over time. +These include ingest and scan performance and other useful measurements. + +The Master Server, Tablet Servers, and Tables pages display metrics grouped in different ways (e.g. by tablet server or by table). +Metrics typically include number of entries (key/value pairs), ingest and query rates. +The number of running scans, major and minor compactions are in the form _number_running_ (_number_queued_). +Another important metric is hold time, which is the amount of time a tablet has been waiting but unable to flush its memory in a minor compaction. + +The Server Activity page graphically displays tablet server status, with each server represented as a circle or square. +Different metrics may be assigned to the nodes' color and speed of oscillation. +The Overall Avg metric is only used on the Server Activity page, and represents the average of all the other metrics (after normalization). +Similarly, the Overall Max metric picks the metric with the maximum normalized value. + +The Garbage Collector page displays a list of garbage collection cycles, the number of files found of each type (including deletion candidates in use and files actually deleted), and the length of the deletion cycle. +The Traces page displays data for recent traces performed (see the following section for information on tracing). +The Recent Logs page displays warning and error logs forwarded to the monitor from all Accumulo processes. +Also, the XML and JSON links provide metrics in XML and JSON formats, respectively. + + SSL +SSL may be enabled for the monitor page by setting the following properties in the +accumulo-site.xml+ file: + + monitor.ssl.keyStore + monitor.ssl.keyStorePassword + monitor.ssl.trustStore + monitor.ssl.trustStorePassword + +If the Accumulo conf directory has been configured (in particular the +accumulo-env.sh+ file must be set up
svn commit: r1598943 - /accumulo/site/trunk/content/source.mdtext
Author: billie Date: Sun Jun 1 02:33:49 2014 New Revision: 1598943 URL: http://svn.apache.org/r1598943 Log: removed jenkins icons Modified: accumulo/site/trunk/content/source.mdtext Modified: accumulo/site/trunk/content/source.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/source.mdtext?rev=1598943r1=1598942r2=1598943view=diff == --- accumulo/site/trunk/content/source.mdtext (original) +++ accumulo/site/trunk/content/source.mdtext Sun Jun 1 02:33:49 2014 @@ -90,13 +90,10 @@ Note that this git clean command will de Accumulo uses [Jenkins][jenkins] for automatic builds. -img src=https://builds.apache.org/job/Accumulo-Master/lastBuild/buildStatus; style=height: 1.1em [Master][masterbuild] - -img src=https://builds.apache.org/job/Accumulo-1.6/lastBuild/buildStatus; style=height: 1.1em [1.6 Branch][16build] - -img src=https://builds.apache.org/job/Accumulo-1.5-Hadoop-2/lastBuild/buildStatus; style=height: 1.1em [1.5 Branch (Hadoop 2)][15buildh2] - -img src=https://builds.apache.org/job/Accumulo-1.5/lastBuild/buildStatus; style=height: 1.1em [1.5 Branch (Hadoop 1)][15build] +* [Master][masterbuild] +* [1.6 Branch][16build] +* [1.5 Branch (Hadoop 2)][15buildh2] +* [1.5 Branch (Hadoop 1)][15build] ### Issue Tracking
svn commit: r910737 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/ 1.6/apidocs/
Author: billie Date: Sun Jun 1 02:34:39 2014 New Revision: 910737 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r910736, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r910736, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r910736, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r910736, websites/production/accumulo/content/1.6/apidocs/
svn commit: r908320 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/ 1.6/apidocs/
Author: billie Date: Wed May 7 17:14:28 2014 New Revision: 908320 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r908319, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r908319, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r908319, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r908319, websites/production/accumulo/content/1.6/apidocs/
git commit: ACCUMULO-1327 removed commented style info from manual header
Repository: accumulo Updated Branches: refs/heads/master 900d6abb0 - 4c0f3590c ACCUMULO-1327 removed commented style info from manual header Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/4c0f3590 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/4c0f3590 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/4c0f3590 Branch: refs/heads/master Commit: 4c0f3590c29ecbedaf534ae18d8590bc9b352c2a Parents: 900d6ab Author: Billie Rinaldi bil...@apache.org Authored: Fri May 9 10:57:35 2014 -0700 Committer: Billie Rinaldi bil...@apache.org Committed: Fri May 9 10:57:35 2014 -0700 -- docs/src/main/asciidoc/accumulo_user_manual.asciidoc | 2 -- 1 file changed, 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/4c0f3590/docs/src/main/asciidoc/accumulo_user_manual.asciidoc -- diff --git a/docs/src/main/asciidoc/accumulo_user_manual.asciidoc b/docs/src/main/asciidoc/accumulo_user_manual.asciidoc index e3f9d4b..fec40ca 100644 --- a/docs/src/main/asciidoc/accumulo_user_manual.asciidoc +++ b/docs/src/main/asciidoc/accumulo_user_manual.asciidoc @@ -17,12 +17,10 @@ Apache Accumulo User Manual Version 1.6 === :author: Apache Accumulo Project :email: d...@accumulo.apache.org -//:theme: flask :toc2: :toclevels: 4 :toc-title: Apache Accumulo 1.6 :numbered: -//:max-width: 50em :website: http://accumulo.apache.org/ image::accumulo-logo.png[]
svn commit: r908327 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/ 1.6/apidocs/
Author: billie Date: Wed May 7 18:16:48 2014 New Revision: 908327 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r908326, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r908326, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r908326, websites/production/accumulo/content/1.5/apidocs/ websites/production/accumulo/content/1.6/apidocs/ - copied from r908326, websites/production/accumulo/content/1.6/apidocs/
svn commit: r1593098 - /accumulo/site/trunk/content/1.6/accumulo_user_manual.html
Author: billie Date: Wed May 7 18:15:51 2014 New Revision: 1593098 URL: http://svn.apache.org/r1593098 Log: removed refs to nonexistent appendix Modified: accumulo/site/trunk/content/1.6/accumulo_user_manual.html Modified: accumulo/site/trunk/content/1.6/accumulo_user_manual.html URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.6/accumulo_user_manual.html?rev=1593098r1=1593097r2=1593098view=diff == --- accumulo/site/trunk/content/1.6/accumulo_user_manual.html (original) +++ accumulo/site/trunk/content/1.6/accumulo_user_manual.html Wed May 7 18:15:51 2014 @@ -1535,8 +1535,7 @@ Connector conn = instance.getConnector( pAccumulo tables have a few options that can be configured to alter the default behavior of Accumulo as well as improve performance based on the data stored. These include locality groups, constraints, bloom filters, iterators, and block -cache. For a complete list of available configuration options, see -Appendix~\ref{app:config}./p +cache./p /div div class=sect2 h3 id=_locality_groups6.1. Locality Groups/h3 @@ -3780,7 +3779,7 @@ secret and make sure that the codeaccu div class=paragraph pSome settings can be modified via the Accumulo shell and take effect immediately, but some settings require a process restart to take effect. See the configuration documentation -(available in the docs directory of the tarball and in Appendix~\ref{app:config}) for details./p +(available in the docs directory of the tarball) for details./p /div /div div class=sect3 @@ -5217,4 +5216,4 @@ Last updated 2014-05-07 09:29:30 PDT /div /div /body -/html \ No newline at end of file +/html
svn commit: r1587229 - /accumulo/site/trunk/content/bylaws.mdtext
Author: billie Date: Mon Apr 14 15:54:53 2014 New Revision: 1587229 URL: http://svn.apache.org/r1587229 Log: committed CTR definition to bylaws Modified: accumulo/site/trunk/content/bylaws.mdtext Modified: accumulo/site/trunk/content/bylaws.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/bylaws.mdtext?rev=1587229r1=1587228r2=1587229view=diff == --- accumulo/site/trunk/content/bylaws.mdtext (original) +++ accumulo/site/trunk/content/bylaws.mdtext Mon Apr 14 15:54:53 2014 @@ -125,7 +125,14 @@ Decisions regarding the project are made All participants in the Accumulo project are encouraged to vote. For technical decisions, only the votes of active committers are binding. Non-binding votes are still useful for those with binding votes to understand the perception of an action across the wider Accumulo community. For PMC decisions, only the votes of active PMC members are binding. -Voting can also be applied to changes to the Accumulo codebase. Please refer to the Accumulo commit and review standard for details. +See the [voting page](http://accumulo.apache.org/governance/voting.html) for more details on the mechanics of voting. + +a name=CTR/a +## Commit Then Review (CTR) + +Voting can also be applied to changes to the Accumulo codebase. Under the Commit Then Review policy, committers can make changes to the codebase without seeking approval beforehand, and the changes are assumed to be approved unless an objection is raised. Only if an objection is raised must a vote take place on the code change. + +For some code changes, committers may wish to get feedback from the community before making the change. It is acceptable for a committer to seek approval before making a change if they so desire. ## Approvals @@ -139,7 +146,7 @@ These are the types of approvals that ca trtdMajority Approval/td tdA majority approval vote passes with 3 binding +1 votes and more binding +1 votes than -1 votes./td trtdLazy Approval (or Lazy Consensus)/td -tdAn action with lazy approval is implicitly allowed unless a -1 vote is received, at which time, depending on the type of action, either majority approval or consensus approval must be obtained./td +tdAn action with lazy approval is implicitly allowed unless a -1 vote is received, at which time, depending on the type of action, either majority approval or consensus approval must be obtained. Lazy Approval can be either emstated/em or emassumed/em, as detailed on the [lazy consensus page](http://accumulo.apache.org/governance/lazyConsensus.html)./td /table ## Vetoes @@ -152,6 +159,8 @@ If you disagree with a valid veto, you m This section describes the various actions which are undertaken within the project, the corresponding approval required for that action and those who have binding votes over the action. It also specifies the minimum length of time that a vote must remain open, measured in days. In general, votes should not be called at times when it is known that interested members of the project will be unavailable. +For Code Change actions, a committer may choose to employ assumed or stated Lazy Approval under the [CTR](#CTR) policy. Assumed Lazy Approval has no minimum length of time before the change can be made. + table trthAction/th thDescription/th
svn commit: r1587239 - /accumulo/site/trunk/content/bylaws.mdtext
Author: billie Date: Mon Apr 14 16:05:40 2014 New Revision: 1587239 URL: http://svn.apache.org/r1587239 Log: fixed lazy consensus link formatting in table Modified: accumulo/site/trunk/content/bylaws.mdtext Modified: accumulo/site/trunk/content/bylaws.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/bylaws.mdtext?rev=1587239r1=1587238r2=1587239view=diff == --- accumulo/site/trunk/content/bylaws.mdtext (original) +++ accumulo/site/trunk/content/bylaws.mdtext Mon Apr 14 16:05:40 2014 @@ -146,7 +146,7 @@ These are the types of approvals that ca trtdMajority Approval/td tdA majority approval vote passes with 3 binding +1 votes and more binding +1 votes than -1 votes./td trtdLazy Approval (or Lazy Consensus)/td -tdAn action with lazy approval is implicitly allowed unless a -1 vote is received, at which time, depending on the type of action, either majority approval or consensus approval must be obtained. Lazy Approval can be either emstated/em or emassumed/em, as detailed on the [lazy consensus page](http://accumulo.apache.org/governance/lazyConsensus.html)./td +tdAn action with lazy approval is implicitly allowed unless a -1 vote is received, at which time, depending on the type of action, either majority approval or consensus approval must be obtained. Lazy Approval can be either emstated/em or emassumed/em, as detailed on the a href=http://accumulo.apache.org/governance/lazyConsensus.html;lazy consensus page/a./td /table ## Vetoes
svn commit: r906155 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/
Author: billie Date: Mon Apr 14 16:06:42 2014 New Revision: 906155 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r906154, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r906154, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r906154, websites/production/accumulo/content/1.5/apidocs/
svn commit: r1587258 - /accumulo/site/trunk/content/bylaws.mdtext
Author: billie Date: Mon Apr 14 17:12:41 2014 New Revision: 1587258 URL: http://svn.apache.org/r1587258 Log: updated bylaws version Modified: accumulo/site/trunk/content/bylaws.mdtext Modified: accumulo/site/trunk/content/bylaws.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/bylaws.mdtext?rev=1587258r1=1587257r2=1587258view=diff == --- accumulo/site/trunk/content/bylaws.mdtext (original) +++ accumulo/site/trunk/content/bylaws.mdtext Mon Apr 14 17:12:41 2014 @@ -16,7 +16,7 @@ Notice:Licensed to the Apache Softwa specific language governing permissions and limitations under the License. -This is version 1 of the bylaws. Community work actively continues on the bylaws, and so key segments of them are subject to change. +This is version 2 of the bylaws. Community work actively continues on the bylaws, and so key segments of them are subject to change. # Introduction
svn commit: r906162 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/
Author: billie Date: Mon Apr 14 17:13:02 2014 New Revision: 906162 Log: Publishing svnmucc operation to accumulo site by billie Added: websites/production/accumulo/content/ - copied from r906161, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r906161, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r906161, websites/production/accumulo/content/1.5/apidocs/