svn commit: r1581267 - /accumulo/site/trunk/content/governance/releasing.mdtext
Author: busbey Date: Tue Mar 25 07:29:05 2014 New Revision: 1581267 URL: http://svn.apache.org/r1581267 Log: per dev-list DISCUSS thread, clarifying language around release tests. Modified: accumulo/site/trunk/content/governance/releasing.mdtext Modified: accumulo/site/trunk/content/governance/releasing.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/governance/releasing.mdtext?rev=1581267r1=1581266r2=1581267view=diff == --- accumulo/site/trunk/content/governance/releasing.mdtext (original) +++ accumulo/site/trunk/content/governance/releasing.mdtext Tue Mar 25 07:29:05 2014 @@ -62,15 +62,15 @@ The following steps can be taken without - Use mvn -Dhadoop.profile=1 verify to run the functional tests against the Hadoop 1 profile ### Cluster based -The following tests require a Hadoop cluster running a minimum of HDFS, MapReduce, and ZooKeeper. The cluster MAY have any number of worker nodes; it can even be a single node in psuedo-distributed mode. A cluster with multiple tablet servers SHOULD be used so that more of the code base will be exercised. For the purposes of release testing, you should note the number of nodes and versions used. See the Releasing section for more details. +The following tests require a Hadoop cluster running a minimum of HDFS, MapReduce, and ZooKeeper. The cluster MAY have any number of worker nodes; it can even be a single node in pseudo-distributed mode. A cluster with multiple tablet servers SHOULD be used so that more of the code base will be exercised. For the purposes of release testing, you should note the number of nodes and versions used. See the Releasing section for more details. 1. For versions prior to 1.6, all functional tests must complete successfully. - See $ACCUMULO_HOME/test/system/auto/README for details on running the functional tests. - - Two 24-hour periods of the randomwalk LongClean test with and without agitation need to be run successfully. + - Two 24-hour periods of the LongClean module of the RandomWalk test need to be run successfully. One of them must use agitation and the other should not. - See $ACCUMULO_HOME/test/system/randomwalk/README for details on running the LongClean module. - - Two 24-hour periods of continuous ingest with and without agitation need to be validated successfully. -- See $ACCUMULO_HOME/test/system/continuous/README for details on running and verifying the continous ingest test. - - Two 72-hour periods of continuous ingest with and without agitation. No validation is necessary but the cluster should be checked to ensure it is still functional. + - Two 24-hour periods of the continuous ingest test must be validated successfully. One test period must use agitation and the other should not. +- See $ACCUMULO_HOME/test/system/continuous/README for details on running and verifying the continuous ingest test. + - Two 72-hour periods of continuous ingest must run. One test period must use agitation and the other should not. No validation is necessary but the cluster should be checked to ensure it is still functional. ## Releasing
svn commit: r903472 - in /websites/staging/accumulo/trunk/content: ./ governance/releasing.html
Author: buildbot Date: Tue Mar 25 07:29:21 2014 New Revision: 903472 Log: Staging update by buildbot for accumulo Modified: websites/staging/accumulo/trunk/content/ (props changed) websites/staging/accumulo/trunk/content/governance/releasing.html Propchange: websites/staging/accumulo/trunk/content/ -- --- cms:source-revision (original) +++ cms:source-revision Tue Mar 25 07:29:21 2014 @@ -1 +1 @@ -1579959 +1581267 Modified: websites/staging/accumulo/trunk/content/governance/releasing.html == --- websites/staging/accumulo/trunk/content/governance/releasing.html (original) +++ websites/staging/accumulo/trunk/content/governance/releasing.html Tue Mar 25 07:29:21 2014 @@ -140,21 +140,21 @@ /li /ol h3 id=cluster-basedCluster based/h3 -pThe following tests require a Hadoop cluster running a minimum of HDFS, MapReduce, and ZooKeeper. The cluster MAY have any number of worker nodes; it can even be a single node in psuedo-distributed mode. A cluster with multiple tablet servers SHOULD be used so that more of the code base will be exercised. For the purposes of release testing, you should note the number of nodes and versions used. See the Releasing section for more details./p +pThe following tests require a Hadoop cluster running a minimum of HDFS, MapReduce, and ZooKeeper. The cluster MAY have any number of worker nodes; it can even be a single node in pseudo-distributed mode. A cluster with multiple tablet servers SHOULD be used so that more of the code base will be exercised. For the purposes of release testing, you should note the number of nodes and versions used. See the Releasing section for more details./p ol liFor versions prior to 1.6, all functional tests must complete successfully.ul liSee $ACCUMULO_HOME/test/system/auto/README for details on running the functional tests./li /ul /li -liTwo 24-hour periods of the randomwalk LongClean test with and without agitation need to be run successfully.ul +liTwo 24-hour periods of the LongClean module of the RandomWalk test need to be run successfully. One of them must use agitation and the other should not.ul liSee $ACCUMULO_HOME/test/system/randomwalk/README for details on running the LongClean module./li /ul /li -liTwo 24-hour periods of continuous ingest with and without agitation need to be validated successfully.ul -liSee $ACCUMULO_HOME/test/system/continuous/README for details on running and verifying the continous ingest test./li +liTwo 24-hour periods of the continuous ingest test must be validated successfully. One test period must use agitation and the other should not.ul +liSee $ACCUMULO_HOME/test/system/continuous/README for details on running and verifying the continuous ingest test./li /ul /li -liTwo 72-hour periods of continuous ingest with and without agitation. No validation is necessary but the cluster should be checked to ensure it is still functional./li +liTwo 72-hour periods of continuous ingest must run. One test period must use agitation and the other should not. No validation is necessary but the cluster should be checked to ensure it is still functional./li /ol h2 id=releasingReleasing/h2 ol
svn commit: r903473 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/
Author: busbey Date: Tue Mar 25 07:56:01 2014 New Revision: 903473 Log: Publishing svnmucc operation to accumulo site by busbey Added: websites/production/accumulo/content/ - copied from r903472, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r903472, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r903472, websites/production/accumulo/content/1.5/apidocs/
git commit: ACCUMULO-2494 add dependency on commons-math
Repository: accumulo Updated Branches: refs/heads/1.6.0-SNAPSHOT e366e7509 - 4bf28d99f ACCUMULO-2494 add dependency on commons-math Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/4bf28d99 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/4bf28d99 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/4bf28d99 Branch: refs/heads/1.6.0-SNAPSHOT Commit: 4bf28d99f21b6620ffd063fd82499fb2193cb508 Parents: e366e75 Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 09:10:35 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 09:11:09 2014 -0400 -- core/pom.xml | 4 1 file changed, 4 insertions(+) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/4bf28d99/core/pom.xml -- diff --git a/core/pom.xml b/core/pom.xml index d56ca2c..a6ac7f8 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -137,6 +137,10 @@ artifactIdslf4j-log4j12/artifactId scopetest/scope /dependency +dependency + groupIdorg.apache.commons/groupId + artifactIdcommons-math/artifactId +/dependency /dependencies build testResources
[1/2] git commit: ACCUMULO-2494 add dependency on commons-math
Repository: accumulo Updated Branches: refs/heads/master 3b80978ed - f7add1859 ACCUMULO-2494 add dependency on commons-math Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/4bf28d99 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/4bf28d99 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/4bf28d99 Branch: refs/heads/master Commit: 4bf28d99f21b6620ffd063fd82499fb2193cb508 Parents: e366e75 Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 09:10:35 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 09:11:09 2014 -0400 -- core/pom.xml | 4 1 file changed, 4 insertions(+) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/4bf28d99/core/pom.xml -- diff --git a/core/pom.xml b/core/pom.xml index d56ca2c..a6ac7f8 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -137,6 +137,10 @@ artifactIdslf4j-log4j12/artifactId scopetest/scope /dependency +dependency + groupIdorg.apache.commons/groupId + artifactIdcommons-math/artifactId +/dependency /dependencies build testResources
[2/2] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f7add185 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f7add185 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f7add185 Branch: refs/heads/master Commit: f7add185924425b2316a94acb11ee79c24756cf1 Parents: 3b80978 4bf28d9 Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 09:15:33 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 09:15:33 2014 -0400 -- core/pom.xml | 4 1 file changed, 4 insertions(+) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/f7add185/core/pom.xml --
[2/2] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/b8b982e7 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/b8b982e7 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/b8b982e7 Branch: refs/heads/master Commit: b8b982e77ac90a994b83c2dcca0ae044564f0038 Parents: f7add18 9f6cc39 Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 09:26:49 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 09:26:49 2014 -0400 -- core/pom.xml | 8 1 file changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/b8b982e7/core/pom.xml --
[1/2] git commit: ACCUMULO-2494 add dependency on commons-math, in sorted order
Repository: accumulo Updated Branches: refs/heads/master f7add1859 - b8b982e77 ACCUMULO-2494 add dependency on commons-math, in sorted order Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/9f6cc390 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/9f6cc390 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/9f6cc390 Branch: refs/heads/master Commit: 9f6cc39015a7533837f35f9af5f4666d52b3c6cb Parents: 4bf28d9 Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 09:26:41 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 09:26:41 2014 -0400 -- core/pom.xml | 8 1 file changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/9f6cc390/core/pom.xml -- diff --git a/core/pom.xml b/core/pom.xml index a6ac7f8..ab77f46 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -84,6 +84,10 @@ /dependency dependency groupIdorg.apache.commons/groupId + artifactIdcommons-math/artifactId +/dependency +dependency + groupIdorg.apache.commons/groupId artifactIdcommons-vfs2/artifactId /dependency dependency @@ -137,10 +141,6 @@ artifactIdslf4j-log4j12/artifactId scopetest/scope /dependency -dependency - groupIdorg.apache.commons/groupId - artifactIdcommons-math/artifactId -/dependency /dependencies build testResources
git commit: ACCUMULO-2494 add dependency on commons-math, in sorted order
Repository: accumulo Updated Branches: refs/heads/1.6.0-SNAPSHOT 4bf28d99f - 9f6cc3901 ACCUMULO-2494 add dependency on commons-math, in sorted order Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/9f6cc390 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/9f6cc390 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/9f6cc390 Branch: refs/heads/1.6.0-SNAPSHOT Commit: 9f6cc39015a7533837f35f9af5f4666d52b3c6cb Parents: 4bf28d9 Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 09:26:41 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 09:26:41 2014 -0400 -- core/pom.xml | 8 1 file changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/9f6cc390/core/pom.xml -- diff --git a/core/pom.xml b/core/pom.xml index a6ac7f8..ab77f46 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -84,6 +84,10 @@ /dependency dependency groupIdorg.apache.commons/groupId + artifactIdcommons-math/artifactId +/dependency +dependency + groupIdorg.apache.commons/groupId artifactIdcommons-vfs2/artifactId /dependency dependency @@ -137,10 +141,6 @@ artifactIdslf4j-log4j12/artifactId scopetest/scope /dependency -dependency - groupIdorg.apache.commons/groupId - artifactIdcommons-math/artifactId -/dependency /dependencies build testResources
svn commit: r1581322 - /accumulo/site/trunk/content/bylaws.mdtext
Author: bhavanki Date: Tue Mar 25 13:49:54 2014 New Revision: 1581322 URL: http://svn.apache.org/r1581322 Log: Add missing vote action for electing new PMC chair; fix couple of minor typos. Modified: accumulo/site/trunk/content/bylaws.mdtext Modified: accumulo/site/trunk/content/bylaws.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/bylaws.mdtext?rev=1581322r1=1581321r2=1581322view=diff == --- accumulo/site/trunk/content/bylaws.mdtext (original) +++ accumulo/site/trunk/content/bylaws.mdtext Tue Mar 25 13:49:54 2014 @@ -128,7 +128,7 @@ Voting can also be applied to changes to ## Approvals -There are the types of approvals that can be sought. Different actions require different types of approvals. +These are the types of approvals that can be sought. Different actions require different types of approvals. table trthApproval Type/th @@ -149,7 +149,7 @@ If you disagree with a valid veto, you m ## Actions -This section describes the various actions which are undertaken within the project, the corresponding approval required for that action and those who have binding votes over the action. It also specifies the minimum length of time that a vote must remain open, measured in days. In general votes should not be called at times when it is known that interested members of the project will be unavailable. +This section describes the various actions which are undertaken within the project, the corresponding approval required for that action and those who have binding votes over the action. It also specifies the minimum length of time that a vote must remain open, measured in days. In general, votes should not be called at times when it is known that interested members of the project will be unavailable. table trthAction/th @@ -192,6 +192,11 @@ This section describes the various actio tdConsensus approval/td tdActive PMC members/td td3/td +trtdNew PMC Chair/td +tdWhen a new PMC chair is chosen to succeed an outgoing chair./td +tdConsensus approval/td +tdActive PMC members/td +td3/td trtdModifying Bylaws/td tdModifying this document./td tdMajority approval/td
svn commit: r903507 - in /websites/staging/accumulo/trunk/content: ./ bylaws.html
Author: buildbot Date: Tue Mar 25 13:50:09 2014 New Revision: 903507 Log: Staging update by buildbot for accumulo Modified: websites/staging/accumulo/trunk/content/ (props changed) websites/staging/accumulo/trunk/content/bylaws.html Propchange: websites/staging/accumulo/trunk/content/ -- --- cms:source-revision (original) +++ cms:source-revision Tue Mar 25 13:50:09 2014 @@ -1 +1 @@ -1581267 +1581322 Modified: websites/staging/accumulo/trunk/content/bylaws.html == --- websites/staging/accumulo/trunk/content/bylaws.html (original) +++ websites/staging/accumulo/trunk/content/bylaws.html Tue Mar 25 13:50:09 2014 @@ -167,7 +167,7 @@ stability, and robustness of both code a pAll participants in the Accumulo project are encouraged to vote. For technical decisions, only the votes of active committers are binding. Non-binding votes are still useful for those with binding votes to understand the perception of an action across the wider Accumulo community. For PMC decisions, only the votes of active PMC members are binding./p pVoting can also be applied to changes to the Accumulo codebase. Please refer to the Accumulo commit and review standard for details./p h2 id=approvalsApprovals/h2 -pThere are the types of approvals that can be sought. Different actions require different types of approvals./p +pThese are the types of approvals that can be sought. Different actions require different types of approvals./p table trthApproval Type/th thDefinition/th @@ -183,7 +183,7 @@ stability, and robustness of both code a pA valid, binding veto cannot be overruled. If a veto is cast, it must be accompanied by a valid reason explaining the veto. The validity of a veto, if challenged, can be confirmed by anyone who has a binding vote. This does not necessarily signify agreement with the veto, but merely that the veto is valid./p pIf you disagree with a valid veto, you must lobby the person casting the veto to withdraw their veto. If a veto is not withdrawn, the action that has been vetoed must be reversed in a timely manner./p h2 id=actionsActions/h2 -pThis section describes the various actions which are undertaken within the project, the corresponding approval required for that action and those who have binding votes over the action. It also specifies the minimum length of time that a vote must remain open, measured in days. In general votes should not be called at times when it is known that interested members of the project will be unavailable./p +pThis section describes the various actions which are undertaken within the project, the corresponding approval required for that action and those who have binding votes over the action. It also specifies the minimum length of time that a vote must remain open, measured in days. In general, votes should not be called at times when it is known that interested members of the project will be unavailable./p table trthAction/th thDescription/th @@ -225,6 +225,11 @@ stability, and robustness of both code a tdConsensus approval/td tdActive PMC members/td td3/td +trtdNew PMC Chair/td +tdWhen a new PMC chair is chosen to succeed an outgoing chair./td +tdConsensus approval/td +tdActive PMC members/td +td3/td trtdModifying Bylaws/td tdModifying this document./td tdMajority approval/td
svn commit: r903508 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/
Author: bhavanki Date: Tue Mar 25 13:50:32 2014 New Revision: 903508 Log: Publishing svnmucc operation to accumulo site by bhavanki Added: websites/production/accumulo/content/ - copied from r903507, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r903507, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r903507, websites/production/accumulo/content/1.5/apidocs/
git commit: ACCUMULO-2419 remove warnings about accessing a static in a static way
Repository: accumulo
Updated Branches:
refs/heads/master b8b982e77 - 2412419af
ACCUMULO-2419 remove warnings about accessing a static in a static way
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2412419a
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2412419a
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2412419a
Branch: refs/heads/master
Commit: 2412419afe54d797af59ff9cf1cf182484c14b3c
Parents: b8b982e
Author: Eric C. Newton eric.new...@gmail.com
Authored: Tue Mar 25 10:02:40 2014 -0400
Committer: Eric C. Newton eric.new...@gmail.com
Committed: Tue Mar 25 10:02:40 2014 -0400
--
.../java/org/apache/accumulo/server/util/time/SimpleTimer.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/2412419a/server/base/src/main/java/org/apache/accumulo/server/util/time/SimpleTimer.java
--
diff --git
a/server/base/src/main/java/org/apache/accumulo/server/util/time/SimpleTimer.java
b/server/base/src/main/java/org/apache/accumulo/server/util/time/SimpleTimer.java
index b8f7f6d..bc1d652 100644
---
a/server/base/src/main/java/org/apache/accumulo/server/util/time/SimpleTimer.java
+++
b/server/base/src/main/java/org/apache/accumulo/server/util/time/SimpleTimer.java
@@ -65,9 +65,9 @@ public class SimpleTimer {
public static synchronized SimpleTimer getInstance(int threadPoolSize) {
if (instance == null) {
instance = new SimpleTimer(threadPoolSize);
- instance.instanceThreadPoolSize = threadPoolSize;
+ SimpleTimer.instanceThreadPoolSize = threadPoolSize;
} else {
- if (instance.instanceThreadPoolSize != threadPoolSize) {
+ if (SimpleTimer.instanceThreadPoolSize != threadPoolSize) {
log.warn(Asked to create SimpleTimer with thread pool size +
threadPoolSize + , existing instance has +
instanceThreadPoolSize);
[1/3] git commit: ACCUMULO-2543 close the cacheblock
Repository: accumulo
Updated Branches:
refs/heads/1.6.0-SNAPSHOT 9f6cc3901 - fd99e24dc
ACCUMULO-2543 close the cacheblock
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/1970b58d
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/1970b58d
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/1970b58d
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: 1970b58dbf0877cc425097e5895ac680432e0a6d
Parents: 9f6cc39
Author: Eric C. Newton eric.new...@gmail.com
Authored: Tue Mar 25 10:37:52 2014 -0400
Committer: Eric C. Newton eric.new...@gmail.com
Committed: Tue Mar 25 10:37:52 2014 -0400
--
.../java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/1970b58d/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
--
diff --git
a/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
b/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
index 4a78fb7..f293718 100644
--- a/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
+++ b/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
@@ -112,6 +112,7 @@ public class BlockIndexTest {
Assert.assertEquals(rk.getKey(), new Key(RFileTest.nf(, row + 1),
cf1, cq1));
}
+cacheBlock.close();
}
@Test
@@ -170,6 +171,6 @@ public class BlockIndexTest {
Assert.assertTrue(rk.getKey().compareTo(seekKey) = 0);
}
-
+cacheBlock.close();
}
}
[2/3] git commit: ACCUMULO-2543 close the cacheblock
ACCUMULO-2543 close the cacheblock
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/ba48f030
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/ba48f030
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/ba48f030
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: ba48f030b1c3f8b249b32f55e9a6b952da28c178
Parents: e6748a8
Author: Eric C. Newton eric.new...@gmail.com
Authored: Tue Mar 25 10:37:52 2014 -0400
Committer: Eric C. Newton eric.new...@gmail.com
Committed: Tue Mar 25 10:41:06 2014 -0400
--
.../java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/ba48f030/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
--
diff --git
a/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
b/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
index 4a78fb7..f293718 100644
--- a/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
+++ b/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
@@ -112,6 +112,7 @@ public class BlockIndexTest {
Assert.assertEquals(rk.getKey(), new Key(RFileTest.nf(, row + 1),
cf1, cq1));
}
+cacheBlock.close();
}
@Test
@@ -170,6 +171,6 @@ public class BlockIndexTest {
Assert.assertTrue(rk.getKey().compareTo(seekKey) = 0);
}
-
+cacheBlock.close();
}
}
[3/5] git commit: ACCUMULO-2543 close the cacheblock
ACCUMULO-2543 close the cacheblock
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/ba48f030
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/ba48f030
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/ba48f030
Branch: refs/heads/master
Commit: ba48f030b1c3f8b249b32f55e9a6b952da28c178
Parents: e6748a8
Author: Eric C. Newton eric.new...@gmail.com
Authored: Tue Mar 25 10:37:52 2014 -0400
Committer: Eric C. Newton eric.new...@gmail.com
Committed: Tue Mar 25 10:41:06 2014 -0400
--
.../java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/ba48f030/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
--
diff --git
a/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
b/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
index 4a78fb7..f293718 100644
--- a/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
+++ b/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
@@ -112,6 +112,7 @@ public class BlockIndexTest {
Assert.assertEquals(rk.getKey(), new Key(RFileTest.nf(, row + 1),
cf1, cq1));
}
+cacheBlock.close();
}
@Test
@@ -170,6 +171,6 @@ public class BlockIndexTest {
Assert.assertTrue(rk.getKey().compareTo(seekKey) = 0);
}
-
+cacheBlock.close();
}
}
[4/5] git commit: Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT
Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/fd99e24d Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/fd99e24d Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/fd99e24d Branch: refs/heads/master Commit: fd99e24dc32be49433476be85893138296ec5cbf Parents: 1970b58 ba48f03 Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 10:41:22 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 10:41:22 2014 -0400 -- --
[3/3] git commit: Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT
Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/fd99e24d Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/fd99e24d Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/fd99e24d Branch: refs/heads/1.6.0-SNAPSHOT Commit: fd99e24dc32be49433476be85893138296ec5cbf Parents: 1970b58 ba48f03 Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 10:41:22 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 10:41:22 2014 -0400 -- --
[3/3] git commit: Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT
Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/76b3fd04 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/76b3fd04 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/76b3fd04 Branch: refs/heads/1.6.0-SNAPSHOT Commit: 76b3fd0499cffc2e40e7b8b2059ed64ad0c701f1 Parents: fd99e24 00355d0 Author: Mike Drob md...@cloudera.com Authored: Tue Mar 25 11:02:56 2014 -0400 Committer: Mike Drob md...@cloudera.com Committed: Tue Mar 25 11:02:56 2014 -0400 -- --
[2/3] git commit: ACCUMULO-2494 add commons-math to 1.5
ACCUMULO-2494 add commons-math to 1.5 Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/00355d0e Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/00355d0e Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/00355d0e Branch: refs/heads/1.6.0-SNAPSHOT Commit: 00355d0e02f3877558d9668bb75db6d703e860ef Parents: ba48f03 Author: Mike Drob md...@cloudera.com Authored: Tue Mar 25 10:56:25 2014 -0400 Committer: Mike Drob md...@cloudera.com Committed: Tue Mar 25 11:02:07 2014 -0400 -- core/pom.xml | 4 1 file changed, 4 insertions(+) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/00355d0e/core/pom.xml -- diff --git a/core/pom.xml b/core/pom.xml index 6e3d917..39de12a 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -51,6 +51,10 @@ /dependency dependency groupIdorg.apache.commons/groupId + artifactIdcommons-math/artifactId +/dependency +dependency + groupIdorg.apache.commons/groupId artifactIdcommons-vfs2/artifactId /dependency dependency
[1/5] git commit: ACCUMULO-2543 close the cacheblock
Repository: accumulo
Updated Branches:
refs/heads/master 2412419af - b40ce8e58
ACCUMULO-2543 close the cacheblock
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/1970b58d
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/1970b58d
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/1970b58d
Branch: refs/heads/master
Commit: 1970b58dbf0877cc425097e5895ac680432e0a6d
Parents: 9f6cc39
Author: Eric C. Newton eric.new...@gmail.com
Authored: Tue Mar 25 10:37:52 2014 -0400
Committer: Eric C. Newton eric.new...@gmail.com
Committed: Tue Mar 25 10:37:52 2014 -0400
--
.../java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/1970b58d/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
--
diff --git
a/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
b/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
index 4a78fb7..f293718 100644
--- a/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
+++ b/core/src/test/java/org/apache/accumulo/core/file/rfile/BlockIndexTest.java
@@ -112,6 +112,7 @@ public class BlockIndexTest {
Assert.assertEquals(rk.getKey(), new Key(RFileTest.nf(, row + 1),
cf1, cq1));
}
+cacheBlock.close();
}
@Test
@@ -170,6 +171,6 @@ public class BlockIndexTest {
Assert.assertTrue(rk.getKey().compareTo(seekKey) = 0);
}
-
+cacheBlock.close();
}
}
[5/5] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/b40ce8e5 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/b40ce8e5 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/b40ce8e5 Branch: refs/heads/master Commit: b40ce8e58c66012b02f4b27ab755ce1d4e769adc Parents: 0f9eb88 fd99e24 Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 10:41:37 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 10:41:37 2014 -0400 -- --
svn commit: r1581380 - /accumulo/site/trunk/content/bylaws.mdtext
Author: busbey Date: Tue Mar 25 15:45:35 2014 New Revision: 1581380 URL: http://svn.apache.org/r1581380 Log: Made clear we can vote on other things if we so choose. Modified: accumulo/site/trunk/content/bylaws.mdtext Modified: accumulo/site/trunk/content/bylaws.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/bylaws.mdtext?rev=1581380r1=1581379r2=1581380view=diff == --- accumulo/site/trunk/content/bylaws.mdtext (original) +++ accumulo/site/trunk/content/bylaws.mdtext Tue Mar 25 15:45:35 2014 @@ -204,7 +204,7 @@ This section describes the various actio td7/td /table -No other voting actions are defined. If a new one is found to be needed, it should be added to these bylaws. +No other voting actions are defined; all other actions should presume Lazy Approval (defaulting to Consensus Approval upon veto). If an action is voted on multiple times, or if a different approval type is desired, these bylaws should be amended to include the action. For the purposes of the Adoption of New Codebase action, the Accumulo codebase is defined as the Accumulo site content, primary project code, and all contributed code (contribs) as they exist in their respective repositories. Adoption of a new codebase generally refers to the creation of a new contrib repository, but could cover, for example, a rework of the project site, or merging a contrib project into the primary codebase.
svn commit: r903533 - in /websites/staging/accumulo/trunk/content: ./ bylaws.html
Author: buildbot Date: Tue Mar 25 15:45:47 2014 New Revision: 903533 Log: Staging update by buildbot for accumulo Modified: websites/staging/accumulo/trunk/content/ (props changed) websites/staging/accumulo/trunk/content/bylaws.html Propchange: websites/staging/accumulo/trunk/content/ -- --- cms:source-revision (original) +++ cms:source-revision Tue Mar 25 15:45:47 2014 @@ -1 +1 @@ -1581322 +1581380 Modified: websites/staging/accumulo/trunk/content/bylaws.html == --- websites/staging/accumulo/trunk/content/bylaws.html (original) +++ websites/staging/accumulo/trunk/content/bylaws.html Tue Mar 25 15:45:47 2014 @@ -237,7 +237,7 @@ stability, and robustness of both code a td7/td /table -pNo other voting actions are defined. If a new one is found to be needed, it should be added to these bylaws./p +pNo other voting actions are defined; all other actions should presume Lazy Approval (defaulting to Consensus Approval upon veto). If an action is voted on multiple times, or if a different approval type is desired, these bylaws should be amended to include the action./p pFor the purposes of the Adoption of New Codebase action, the Accumulo codebase is defined as the Accumulo site content, primary project code, and all contributed code (contribs) as they exist in their respective repositories. Adoption of a new codebase generally refers to the creation of a new contrib repository, but could cover, for example, a rework of the project site, or merging a contrib project into the primary codebase./p pVoting actions for the removal of a committer or PMC member are intentionally not defined. According to ASF rules, a href=http://www.apache.org/dev/committers.html#committer-set-term;committer status never expires/a and a href=http://www.apache.org/dev/pmc.html#pmc-removal;the ASF board removes PMC members/a./p h1 id=release-plansRelease Plans/h1
svn commit: r903534 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/
Author: busbey Date: Tue Mar 25 15:47:13 2014 New Revision: 903534 Log: Publishing svnmucc operation to accumulo site by busbey Added: websites/production/accumulo/content/ - copied from r903533, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r903533, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r903533, websites/production/accumulo/content/1.5/apidocs/
svn commit: r903512 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/
Author: bhavanki Date: Tue Mar 25 13:56:35 2014 New Revision: 903512 Log: Publishing svnmucc operation to accumulo site by bhavanki Added: websites/production/accumulo/content/ - copied from r903511, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r903511, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r903511, websites/production/accumulo/content/1.5/apidocs/
[1/3] git commit: ACCUMULO-2169 tighten up rat check in build script
Repository: accumulo
Updated Branches:
refs/heads/1.4.5-SNAPSHOT edfbd2fff - 2ab011c3a
refs/heads/1.5.2-SNAPSHOT 00355d0e0 - 22f955900
ACCUMULO-2169 tighten up rat check in build script
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2ab011c3
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2ab011c3
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2ab011c3
Branch: refs/heads/1.4.5-SNAPSHOT
Commit: 2ab011c3a7dc2eaa52bd8845533af331e28ef7b5
Parents: edfbd2f
Author: Mike Drob md...@cloudera.com
Authored: Tue Mar 25 12:19:34 2014 -0400
Committer: Mike Drob md...@cloudera.com
Committed: Tue Mar 25 12:19:34 2014 -0400
--
src/assemble/build.sh | 31 +++
1 file changed, 23 insertions(+), 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/2ab011c3/src/assemble/build.sh
--
diff --git a/src/assemble/build.sh b/src/assemble/build.sh
index 867595b..6904535 100755
--- a/src/assemble/build.sh
+++ b/src/assemble/build.sh
@@ -15,6 +15,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+set -e
+
loc=`dirname $0`
loc=`cd $loc/../..; pwd`
@@ -38,16 +40,29 @@ runAt() {
( cd $1 ; echo in `pwd`; shift ; run $@ ) || fail
}
+verifyMissingLicenses() {
+ COUNT=$(grep -c '!' $1)
+ EXPECTED=$2
+ if [[ $COUNT != $EXPECTED ]]; then
+fail expected $EXPECTED files missing licenses, but saw $COUNT
+ fi
+}
+
+mvn -version | grep -q Java version: 1.6 || fail Need to use Java 6
+
run mvn -U -P distclean clean
-mvn org.apache.rat:apache-rat-plugin:0.10:check
-COUNT=`grep '!' target/rat.txt | wc -l`
-EXPECTED=51
-if [ $COUNT -ne $EXPECTED ]
-then
- fail expected $EXPECTED files missing licenses, but saw $COUNT
-fi
+
+mvn org.apache.rat:apache-rat-plugin:0.10:check || :
+verifyMissingLicenses target/rat.txt 51
+
+find src -name pom.xml -not -path 'src/server/*' \
+ -exec mvn org.apache.rat:apache-rat-plugin:0.10:check -f {} \;
+
+mvn org.apache.rat:apache-rat-plugin:0.10:check -f src/server/pom.xml || :
+verifyMissingLicenses src/server/target/rat.txt 27
+
#need to run mvn package twice to properly build docs/config.html
run mvn package
-run mvn package javadoc:aggregate javadoc:jar source:jar
+run mvn package javadoc:aggregate javadoc:jar source:jar -DskipTests
runAt ./src/server/src/main/c++ make
run mvn assembly:single -N
[2/3] git commit: ACCUMULO-2169 tighten up rat check in build script
ACCUMULO-2169 tighten up rat check in build script
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2ab011c3
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2ab011c3
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2ab011c3
Branch: refs/heads/1.5.2-SNAPSHOT
Commit: 2ab011c3a7dc2eaa52bd8845533af331e28ef7b5
Parents: edfbd2f
Author: Mike Drob md...@cloudera.com
Authored: Tue Mar 25 12:19:34 2014 -0400
Committer: Mike Drob md...@cloudera.com
Committed: Tue Mar 25 12:19:34 2014 -0400
--
src/assemble/build.sh | 31 +++
1 file changed, 23 insertions(+), 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/2ab011c3/src/assemble/build.sh
--
diff --git a/src/assemble/build.sh b/src/assemble/build.sh
index 867595b..6904535 100755
--- a/src/assemble/build.sh
+++ b/src/assemble/build.sh
@@ -15,6 +15,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+set -e
+
loc=`dirname $0`
loc=`cd $loc/../..; pwd`
@@ -38,16 +40,29 @@ runAt() {
( cd $1 ; echo in `pwd`; shift ; run $@ ) || fail
}
+verifyMissingLicenses() {
+ COUNT=$(grep -c '!' $1)
+ EXPECTED=$2
+ if [[ $COUNT != $EXPECTED ]]; then
+fail expected $EXPECTED files missing licenses, but saw $COUNT
+ fi
+}
+
+mvn -version | grep -q Java version: 1.6 || fail Need to use Java 6
+
run mvn -U -P distclean clean
-mvn org.apache.rat:apache-rat-plugin:0.10:check
-COUNT=`grep '!' target/rat.txt | wc -l`
-EXPECTED=51
-if [ $COUNT -ne $EXPECTED ]
-then
- fail expected $EXPECTED files missing licenses, but saw $COUNT
-fi
+
+mvn org.apache.rat:apache-rat-plugin:0.10:check || :
+verifyMissingLicenses target/rat.txt 51
+
+find src -name pom.xml -not -path 'src/server/*' \
+ -exec mvn org.apache.rat:apache-rat-plugin:0.10:check -f {} \;
+
+mvn org.apache.rat:apache-rat-plugin:0.10:check -f src/server/pom.xml || :
+verifyMissingLicenses src/server/target/rat.txt 27
+
#need to run mvn package twice to properly build docs/config.html
run mvn package
-run mvn package javadoc:aggregate javadoc:jar source:jar
+run mvn package javadoc:aggregate javadoc:jar source:jar -DskipTests
runAt ./src/server/src/main/c++ make
run mvn assembly:single -N
[3/3] git commit: Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT
Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/22f95590 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/22f95590 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/22f95590 Branch: refs/heads/1.5.2-SNAPSHOT Commit: 22f9559002df977fb18819c5935bfd1841be0e5c Parents: 00355d0 2ab011c Author: Mike Drob md...@cloudera.com Authored: Tue Mar 25 12:22:01 2014 -0400 Committer: Mike Drob md...@cloudera.com Committed: Tue Mar 25 12:22:01 2014 -0400 -- --
svn commit: r903536 - in /websites/staging/accumulo/trunk/content: ./ people.html
Author: buildbot
Date: Tue Mar 25 16:24:58 2014
New Revision: 903536
Log:
Staging update by buildbot for accumulo
Modified:
websites/staging/accumulo/trunk/content/ (props changed)
websites/staging/accumulo/trunk/content/people.html
Propchange: websites/staging/accumulo/trunk/content/
--
--- cms:source-revision (original)
+++ cms:source-revision Tue Mar 25 16:24:58 2014
@@ -1 +1 @@
-1581380
+1581405
Modified: websites/staging/accumulo/trunk/content/people.html
==
--- websites/staging/accumulo/trunk/content/people.html (original)
+++ websites/staging/accumulo/trunk/content/people.html Tue Mar 25 16:24:58 2014
@@ -177,6 +177,7 @@ tr:nth-child(2n+1) {
trtd/tdtdMichael Berman/tdtda
href=http://sqrrl.com/sqrrl/a/tdtdET (a
href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a
/ a
href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr
trtd/tdtdMichael Wall/tdtd/tdtd/td/tr
trtd/tdtdMiguel Pereira/tdtda href=http://sra.com;SRA
International, Inc/a/tdtdET (a
href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a
/ a
href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr
+ trtdfaganm/tdtdMike Fagan/tdtda
href=http://www.arcus-research.com;Arcus Research/a/tdtdMT (a
href=http://www.timeanddate.com/library/abbreviations/timezones/na/mst.html;-7/a
/ a
href=http://www.timeanddate.com/library/abbreviations/timezones/na/mdt.html;-6/a)/td/tr
trtd/tdtdMorgan Haskel/tdtd/tdtd/td/tr
trtd/tdtdNguessan Kouame/tdtd/tdtd/td/tr
trtd/tdtdOren Falkowitz/tdtda
href=http://sqrrl.com/sqrrl/a/tdtdET (a
href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html-5/a
/ a
href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html-4/a)/td/tr
svn commit: r903537 - in /websites/production/accumulo/content: ./ 1.4/apidocs/ 1.5/apidocs/
Author: mdrob Date: Tue Mar 25 16:25:16 2014 New Revision: 903537 Log: Publishing svnmucc operation to accumulo site by mdrob Added: websites/production/accumulo/content/ - copied from r903536, websites/staging/accumulo/trunk/content/ websites/production/accumulo/content/1.4/apidocs/ - copied from r903536, websites/production/accumulo/content/1.4/apidocs/ websites/production/accumulo/content/1.5/apidocs/ - copied from r903536, websites/production/accumulo/content/1.5/apidocs/
svn commit: r903541 - in /websites/staging/accumulo/trunk/content: ./ bylaws.html
Author: buildbot Date: Tue Mar 25 16:55:27 2014 New Revision: 903541 Log: Staging update by buildbot for accumulo Modified: websites/staging/accumulo/trunk/content/ (props changed) websites/staging/accumulo/trunk/content/bylaws.html Propchange: websites/staging/accumulo/trunk/content/ -- --- cms:source-revision (original) +++ cms:source-revision Tue Mar 25 16:55:27 2014 @@ -1 +1 @@ -1581405 +1581416 Modified: websites/staging/accumulo/trunk/content/bylaws.html == --- websites/staging/accumulo/trunk/content/bylaws.html (original) +++ websites/staging/accumulo/trunk/content/bylaws.html Tue Mar 25 16:55:27 2014 @@ -125,12 +125,14 @@ stability, and robustness of both code a ul liDeciding what is distributed as products of the Apache Accumulo project./li liMaintaining the project's shared resources, including the code repository, mailing lists, and websites./li +liProtecting and ensuring proper use of Apache trademarks by the project and by other parties./li liSpeaking on behalf of the project./li liResolving license disputes regarding products of the project./li liNominating new PMC members and committers./li liMaintaining these bylaws and other guidelines of the project./li /ul -pIn particular, PMC members must understand both our project's criteria and ASF criteria for voting on a a href=http://www.apache.org/dev/release.html#management;release/a./p +pIn particular, PMC members must understand both our project's criteria and ASF criteria for voting on a a href=http://www.apache.org/dev/release.html#management;release/a. +See the a href=http://www.apache.org/dev/pmc.html;PMC Guide/a for more information on PMC responsibilities./p pMembership of the PMC is by invitation only and must be approved by a consensus approval of active PMC members. Upon acceptance of the invitation to become a PMC member, it is the accepting member’s responsibility to update their status on the Accumulo web page accordingly./p pA PMC member is considered emeritus, meaning inactive, by their own declaration or by not contributing in any form to the project for over six months. Emeritus members will be recognized by the PMC on the Accumulo web page, in honor of their past contributions. Emeritus members retain all voting and commit rights associated with their former designation and can move themselves out of emeritus status by sending an announcement of their return to the developer mailing list. It will be the returning member's responsibility to update their status on the web page accordingly./p pThe chair of the PMC is appointed by the ASF board. The chair is an office holder of the Apache Software Foundation (Vice President, Apache Accumulo) and has primary responsibility to the board for the management of the projects within the scope of the Accumulo PMC. The chair reports to the board quarterly on developments within the Accumulo project./p @@ -239,7 +241,7 @@ stability, and robustness of both code a pNo other voting actions are defined; all other actions should presume Lazy Approval (defaulting to Consensus Approval upon veto). If an action is voted on multiple times, or if a different approval type is desired, these bylaws should be amended to include the action./p pFor the purposes of the Adoption of New Codebase action, the Accumulo codebase is defined as the Accumulo site content, primary project code, and all contributed code (contribs) as they exist in their respective repositories. Adoption of a new codebase generally refers to the creation of a new contrib repository, but could cover, for example, a rework of the project site, or merging a contrib project into the primary codebase./p -pVoting actions for the removal of a committer or PMC member are intentionally not defined. According to ASF rules, a href=http://www.apache.org/dev/committers.html#committer-set-term;committer status never expires/a and a href=http://www.apache.org/dev/pmc.html#pmc-removal;the ASF board removes PMC members/a./p +pVoting actions for the removal of a committer or PMC member are intentionally not defined. According to ASF rules, a href=http://www.apache.org/dev/committers.html#committer-set-term;committer status never expires/a and a href=http://www.apache.org/dev/pmc.html#pmc-removal;PMC members can only be removed with approval from the ASF Board/a./p h1 id=release-plansRelease Plans/h1 pThe approval of a release plan begins the process of creating a new project release. The process ends when a release candidate is approved./p pAn Accumulo release plan consists of at least the following:/p
svn commit: r1581416 - /accumulo/site/trunk/content/bylaws.mdtext
Author: billie Date: Tue Mar 25 16:55:20 2014 New Revision: 1581416 URL: http://svn.apache.org/r1581416 Log: some clarifications Modified: accumulo/site/trunk/content/bylaws.mdtext Modified: accumulo/site/trunk/content/bylaws.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/trunk/content/bylaws.mdtext?rev=1581416r1=1581415r2=1581416view=diff == --- accumulo/site/trunk/content/bylaws.mdtext (original) +++ accumulo/site/trunk/content/bylaws.mdtext Tue Mar 25 16:55:20 2014 @@ -71,12 +71,14 @@ The responsibilities of the PMC include: * Deciding what is distributed as products of the Apache Accumulo project. * Maintaining the project's shared resources, including the code repository, mailing lists, and websites. +* Protecting and ensuring proper use of Apache trademarks by the project and by other parties. * Speaking on behalf of the project. * Resolving license disputes regarding products of the project. * Nominating new PMC members and committers. * Maintaining these bylaws and other guidelines of the project. In particular, PMC members must understand both our project's criteria and ASF criteria for voting on a [release](http://www.apache.org/dev/release.html#management). +See the [PMC Guide](http://www.apache.org/dev/pmc.html) for more information on PMC responsibilities. Membership of the PMC is by invitation only and must be approved by a consensus approval of active PMC members. Upon acceptance of the invitation to become a PMC member, it is the accepting member’s responsibility to update their status on the Accumulo web page accordingly. @@ -208,7 +210,7 @@ No other voting actions are defined; all For the purposes of the Adoption of New Codebase action, the Accumulo codebase is defined as the Accumulo site content, primary project code, and all contributed code (contribs) as they exist in their respective repositories. Adoption of a new codebase generally refers to the creation of a new contrib repository, but could cover, for example, a rework of the project site, or merging a contrib project into the primary codebase. -Voting actions for the removal of a committer or PMC member are intentionally not defined. According to ASF rules, [committer status never expires](http://www.apache.org/dev/committers.html#committer-set-term) and [the ASF board removes PMC members](http://www.apache.org/dev/pmc.html#pmc-removal). +Voting actions for the removal of a committer or PMC member are intentionally not defined. According to ASF rules, [committer status never expires](http://www.apache.org/dev/committers.html#committer-set-term) and [PMC members can only be removed with approval from the ASF Board](http://www.apache.org/dev/pmc.html#pmc-removal). # Release Plans
git commit: ACCUMULO-2547 run the tservers with more memory
Repository: accumulo
Updated Branches:
refs/heads/1.6.0-SNAPSHOT 76b3fd049 - d77fd6fd2
ACCUMULO-2547 run the tservers with more memory
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/d77fd6fd
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/d77fd6fd
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/d77fd6fd
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: d77fd6fd29d2c769991b44b6f98be14b108169aa
Parents: 76b3fd0
Author: Eric C. Newton eric.new...@gmail.com
Authored: Tue Mar 25 13:00:08 2014 -0400
Committer: Eric C. Newton eric.new...@gmail.com
Committed: Tue Mar 25 13:00:22 2014 -0400
--
.../test/java/org/apache/accumulo/test/functional/LargeRowIT.java | 3 +++
1 file changed, 3 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/d77fd6fd/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
--
diff --git
a/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
b/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
index 2f3ce04..31783c4 100644
--- a/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
+++ b/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
@@ -33,6 +33,8 @@ import org.apache.accumulo.core.data.Range;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.security.Authorizations;
import org.apache.accumulo.core.util.UtilWaitThread;
+import org.apache.accumulo.minicluster.MemoryUnit;
+import org.apache.accumulo.minicluster.ServerType;
import org.apache.accumulo.minicluster.impl.MiniAccumuloConfigImpl;
import org.apache.accumulo.test.TestIngest;
import org.apache.hadoop.conf.Configuration;
@@ -44,6 +46,7 @@ public class LargeRowIT extends ConfigurableMacIT {
@Override
public void configure(MiniAccumuloConfigImpl cfg, Configuration
hadoopCoreSite) {
+cfg.setMemory(ServerType.TABLET_SERVER,
cfg.getMemory(ServerType.TABLET_SERVER) * 2, MemoryUnit.BYTE);
cfg.setSiteConfig(Collections.singletonMap(Property.TSERV_MAJC_DELAY.getKey(),
10ms));
}
[2/4] git commit: Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT
Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/76b3fd04 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/76b3fd04 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/76b3fd04 Branch: refs/heads/master Commit: 76b3fd0499cffc2e40e7b8b2059ed64ad0c701f1 Parents: fd99e24 00355d0 Author: Mike Drob md...@cloudera.com Authored: Tue Mar 25 11:02:56 2014 -0400 Committer: Mike Drob md...@cloudera.com Committed: Tue Mar 25 11:02:56 2014 -0400 -- --
[4/4] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/4eb5237e Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/4eb5237e Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/4eb5237e Branch: refs/heads/master Commit: 4eb5237e27b655d41e7e513751341671955dc034 Parents: b40ce8e d77fd6f Author: Eric C. Newton eric.new...@gmail.com Authored: Tue Mar 25 13:01:54 2014 -0400 Committer: Eric C. Newton eric.new...@gmail.com Committed: Tue Mar 25 13:01:54 2014 -0400 -- .../test/java/org/apache/accumulo/test/functional/LargeRowIT.java | 3 +++ 1 file changed, 3 insertions(+) --
[3/4] git commit: ACCUMULO-2547 run the tservers with more memory
ACCUMULO-2547 run the tservers with more memory
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/d77fd6fd
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/d77fd6fd
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/d77fd6fd
Branch: refs/heads/master
Commit: d77fd6fd29d2c769991b44b6f98be14b108169aa
Parents: 76b3fd0
Author: Eric C. Newton eric.new...@gmail.com
Authored: Tue Mar 25 13:00:08 2014 -0400
Committer: Eric C. Newton eric.new...@gmail.com
Committed: Tue Mar 25 13:00:22 2014 -0400
--
.../test/java/org/apache/accumulo/test/functional/LargeRowIT.java | 3 +++
1 file changed, 3 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/d77fd6fd/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
--
diff --git
a/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
b/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
index 2f3ce04..31783c4 100644
--- a/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
+++ b/test/src/test/java/org/apache/accumulo/test/functional/LargeRowIT.java
@@ -33,6 +33,8 @@ import org.apache.accumulo.core.data.Range;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.security.Authorizations;
import org.apache.accumulo.core.util.UtilWaitThread;
+import org.apache.accumulo.minicluster.MemoryUnit;
+import org.apache.accumulo.minicluster.ServerType;
import org.apache.accumulo.minicluster.impl.MiniAccumuloConfigImpl;
import org.apache.accumulo.test.TestIngest;
import org.apache.hadoop.conf.Configuration;
@@ -44,6 +46,7 @@ public class LargeRowIT extends ConfigurableMacIT {
@Override
public void configure(MiniAccumuloConfigImpl cfg, Configuration
hadoopCoreSite) {
+cfg.setMemory(ServerType.TABLET_SERVER,
cfg.getMemory(ServerType.TABLET_SERVER) * 2, MemoryUnit.BYTE);
cfg.setSiteConfig(Collections.singletonMap(Property.TSERV_MAJC_DELAY.getKey(),
10ms));
}
[1/4] git commit: ACCUMULO-2494 add commons-math to 1.5
Repository: accumulo Updated Branches: refs/heads/master b40ce8e58 - 4eb5237e2 ACCUMULO-2494 add commons-math to 1.5 Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/00355d0e Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/00355d0e Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/00355d0e Branch: refs/heads/master Commit: 00355d0e02f3877558d9668bb75db6d703e860ef Parents: ba48f03 Author: Mike Drob md...@cloudera.com Authored: Tue Mar 25 10:56:25 2014 -0400 Committer: Mike Drob md...@cloudera.com Committed: Tue Mar 25 11:02:07 2014 -0400 -- core/pom.xml | 4 1 file changed, 4 insertions(+) -- http://git-wip-us.apache.org/repos/asf/accumulo/blob/00355d0e/core/pom.xml -- diff --git a/core/pom.xml b/core/pom.xml index 6e3d917..39de12a 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -51,6 +51,10 @@ /dependency dependency groupIdorg.apache.commons/groupId + artifactIdcommons-math/artifactId +/dependency +dependency + groupIdorg.apache.commons/groupId artifactIdcommons-vfs2/artifactId /dependency dependency
svn commit: r1581434 - in /accumulo/site/branches/redesign14: content/ content/downloads/ templates/
Author: bhavanki Date: Tue Mar 25 18:02:09 2014 New Revision: 1581434 URL: http://svn.apache.org/r1581434 Log: Fixed nav menu so current page is indicated correctly. Modified: accumulo/site/branches/redesign14/content/contrib.mdtext accumulo/site/branches/redesign14/content/downloads/index.mdtext accumulo/site/branches/redesign14/content/get_involved.mdtext accumulo/site/branches/redesign14/content/git.mdtext accumulo/site/branches/redesign14/content/glossary.mdtext accumulo/site/branches/redesign14/content/index.mdtext accumulo/site/branches/redesign14/content/mailing_list.mdtext accumulo/site/branches/redesign14/content/notable_features.mdtext accumulo/site/branches/redesign14/content/papers.mdtext accumulo/site/branches/redesign14/content/people.mdtext accumulo/site/branches/redesign14/content/releasing.mdtext accumulo/site/branches/redesign14/content/screenshots.mdtext accumulo/site/branches/redesign14/content/source.mdtext accumulo/site/branches/redesign14/templates/nav.html accumulo/site/branches/redesign14/templates/skeleton.html Modified: accumulo/site/branches/redesign14/content/contrib.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/contrib.mdtext?rev=1581434r1=1581433r2=1581434view=diff == --- accumulo/site/branches/redesign14/content/contrib.mdtext (original) +++ accumulo/site/branches/redesign14/content/contrib.mdtext Tue Mar 25 18:02:09 2014 @@ -1,4 +1,5 @@ Title: Apache Accumulo Contrib Projects +Nav: nav_contrib Notice:Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information Modified: accumulo/site/branches/redesign14/content/downloads/index.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/downloads/index.mdtext?rev=1581434r1=1581433r2=1581434view=diff == --- accumulo/site/branches/redesign14/content/downloads/index.mdtext (original) +++ accumulo/site/branches/redesign14/content/downloads/index.mdtext Tue Mar 25 18:02:09 2014 @@ -1,4 +1,5 @@ Title: Apache Accumulo Downloads +Nav: nav_downloads Notice:Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information Modified: accumulo/site/branches/redesign14/content/get_involved.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/get_involved.mdtext?rev=1581434r1=1581433r2=1581434view=diff == --- accumulo/site/branches/redesign14/content/get_involved.mdtext (original) +++ accumulo/site/branches/redesign14/content/get_involved.mdtext Tue Mar 25 18:02:09 2014 @@ -1,4 +1,5 @@ Title: Get Involved +Nav: nav_getinvolved Notice:Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information Modified: accumulo/site/branches/redesign14/content/git.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/git.mdtext?rev=1581434r1=1581433r2=1581434view=diff == --- accumulo/site/branches/redesign14/content/git.mdtext (original) +++ accumulo/site/branches/redesign14/content/git.mdtext Tue Mar 25 18:02:09 2014 @@ -1,4 +1,5 @@ Title: Apache Accumulo Git WIP +Nav: nav_git Notice:Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information Modified: accumulo/site/branches/redesign14/content/glossary.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/glossary.mdtext?rev=1581434r1=1581433r2=1581434view=diff == --- accumulo/site/branches/redesign14/content/glossary.mdtext (original) +++ accumulo/site/branches/redesign14/content/glossary.mdtext Tue Mar 25 18:02:09 2014 @@ -1,4 +1,5 @@ Title: Apache Accumulo Glossary +Nav: nav_glossary Notice:Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information Modified: accumulo/site/branches/redesign14/content/index.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/index.mdtext?rev=1581434r1=1581433r2=1581434view=diff
[2/3] git commit: ACCUMULO-2503 add formatter tests
ACCUMULO-2503 add formatter tests
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/c56ef2ee
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/c56ef2ee
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/c56ef2ee
Branch: refs/heads/master
Commit: c56ef2eec1f62e940cb9f137bfd0b2821aa478e7
Parents: d77fd6f
Author: Mike Drob md...@cloudera.com
Authored: Wed Mar 19 19:12:47 2014 -0400
Committer: Mike Drob md...@cloudera.com
Committed: Tue Mar 25 14:00:08 2014 -0400
--
.../core/util/format/AggregatingFormatter.java | 52 ++
.../core/util/format/BinaryFormatter.java | 89 ++---
.../core/util/format/DefaultFormatter.java | 42 +++--
.../core/util/format/DeleterFormatter.java | 3 +
.../core/util/format/FormatterFactory.java | 4 +
.../accumulo/core/util/format/HexFormatter.java | 8 +-
.../ShardedTableDistributionFormatter.java | 16 +-
.../util/format/StatisticsDisplayFormatter.java | 16 +-
.../util/format/DateStringFormatterTest.java| 61 +++
.../core/util/format/DefaultFormatterTest.java | 62 +++
.../core/util/format/DeleterFormatterTest.java | 179 +++
.../core/util/format/FormatterFactoryTest.java | 45 +
.../core/util/format/HexFormatterTest.java | 88 +
.../ShardedTableDistributionFormatterTest.java | 66 +++
.../format/StatisticsDisplayFormatterTest.java | 64 +++
core/src/test/resources/log4j.properties| 1 +
16 files changed, 681 insertions(+), 115 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/c56ef2ee/core/src/main/java/org/apache/accumulo/core/util/format/AggregatingFormatter.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/util/format/AggregatingFormatter.java
b/core/src/main/java/org/apache/accumulo/core/util/format/AggregatingFormatter.java
new file mode 100644
index 000..e0791eb
--- /dev/null
+++
b/core/src/main/java/org/apache/accumulo/core/util/format/AggregatingFormatter.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.util.format;
+
+import java.util.Iterator;
+import java.util.Map.Entry;
+
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Value;
+
+/**
+ * Formatter that will aggregate entries for various display purposes.
+ */
+public abstract class AggregatingFormatter extends DefaultFormatter {
+ @Override
+ public String next() {
+IteratorEntryKey,Value si = super.getScannerIterator();
+checkState(true);
+while (si.hasNext())
+ aggregateStats(si.next());
+return getStats();
+ }
+
+ /**
+ * Generate statistics from each {@link Entry}, called for each entry to be
iterated over.
+ *
+ * @param next
+ * the next entry to aggregate
+ */
+ protected abstract void aggregateStats(EntryKey,Value next);
+
+ /**
+ * Finalize the aggregation and return the result. Called once at the end.
+ *
+ * @return the aggregation results, suitable for printing to the console
+ */
+ protected abstract String getStats();
+}
http://git-wip-us.apache.org/repos/asf/accumulo/blob/c56ef2ee/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
b/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
index 5021d66..b31df18 100644
---
a/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
+++
b/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
@@ -16,7 +16,6 @@
*/
package org.apache.accumulo.core.util.format;
-import java.util.Iterator;
import java.util.Map.Entry;
import org.apache.accumulo.core.data.Key;
@@ -24,66 +23,43 @@ import org.apache.accumulo.core.data.Value;
import
[3/3] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/a749fad0 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/a749fad0 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/a749fad0 Branch: refs/heads/master Commit: a749fad02d0290361a311fae63600746005d5f26 Parents: 4eb5237 c56ef2e Author: Mike Drob md...@cloudera.com Authored: Tue Mar 25 14:14:03 2014 -0400 Committer: Mike Drob md...@cloudera.com Committed: Tue Mar 25 14:14:03 2014 -0400 -- .../core/util/format/AggregatingFormatter.java | 52 ++ .../core/util/format/BinaryFormatter.java | 89 ++--- .../core/util/format/DefaultFormatter.java | 42 +++-- .../core/util/format/DeleterFormatter.java | 3 + .../core/util/format/FormatterFactory.java | 4 + .../accumulo/core/util/format/HexFormatter.java | 8 +- .../ShardedTableDistributionFormatter.java | 16 +- .../util/format/StatisticsDisplayFormatter.java | 16 +- .../util/format/DateStringFormatterTest.java| 61 +++ .../core/util/format/DefaultFormatterTest.java | 62 +++ .../core/util/format/DeleterFormatterTest.java | 179 +++ .../core/util/format/FormatterFactoryTest.java | 45 + .../core/util/format/HexFormatterTest.java | 88 + .../ShardedTableDistributionFormatterTest.java | 66 +++ .../format/StatisticsDisplayFormatterTest.java | 64 +++ core/src/test/resources/log4j.properties| 1 + 16 files changed, 681 insertions(+), 115 deletions(-) --
[1/3] git commit: ACCUMULO-2503 add formatter tests
Repository: accumulo
Updated Branches:
refs/heads/1.6.0-SNAPSHOT d77fd6fd2 - c56ef2eec
refs/heads/master 4eb5237e2 - a749fad02
ACCUMULO-2503 add formatter tests
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/c56ef2ee
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/c56ef2ee
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/c56ef2ee
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: c56ef2eec1f62e940cb9f137bfd0b2821aa478e7
Parents: d77fd6f
Author: Mike Drob md...@cloudera.com
Authored: Wed Mar 19 19:12:47 2014 -0400
Committer: Mike Drob md...@cloudera.com
Committed: Tue Mar 25 14:00:08 2014 -0400
--
.../core/util/format/AggregatingFormatter.java | 52 ++
.../core/util/format/BinaryFormatter.java | 89 ++---
.../core/util/format/DefaultFormatter.java | 42 +++--
.../core/util/format/DeleterFormatter.java | 3 +
.../core/util/format/FormatterFactory.java | 4 +
.../accumulo/core/util/format/HexFormatter.java | 8 +-
.../ShardedTableDistributionFormatter.java | 16 +-
.../util/format/StatisticsDisplayFormatter.java | 16 +-
.../util/format/DateStringFormatterTest.java| 61 +++
.../core/util/format/DefaultFormatterTest.java | 62 +++
.../core/util/format/DeleterFormatterTest.java | 179 +++
.../core/util/format/FormatterFactoryTest.java | 45 +
.../core/util/format/HexFormatterTest.java | 88 +
.../ShardedTableDistributionFormatterTest.java | 66 +++
.../format/StatisticsDisplayFormatterTest.java | 64 +++
core/src/test/resources/log4j.properties| 1 +
16 files changed, 681 insertions(+), 115 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/c56ef2ee/core/src/main/java/org/apache/accumulo/core/util/format/AggregatingFormatter.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/util/format/AggregatingFormatter.java
b/core/src/main/java/org/apache/accumulo/core/util/format/AggregatingFormatter.java
new file mode 100644
index 000..e0791eb
--- /dev/null
+++
b/core/src/main/java/org/apache/accumulo/core/util/format/AggregatingFormatter.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.util.format;
+
+import java.util.Iterator;
+import java.util.Map.Entry;
+
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Value;
+
+/**
+ * Formatter that will aggregate entries for various display purposes.
+ */
+public abstract class AggregatingFormatter extends DefaultFormatter {
+ @Override
+ public String next() {
+IteratorEntryKey,Value si = super.getScannerIterator();
+checkState(true);
+while (si.hasNext())
+ aggregateStats(si.next());
+return getStats();
+ }
+
+ /**
+ * Generate statistics from each {@link Entry}, called for each entry to be
iterated over.
+ *
+ * @param next
+ * the next entry to aggregate
+ */
+ protected abstract void aggregateStats(EntryKey,Value next);
+
+ /**
+ * Finalize the aggregation and return the result. Called once at the end.
+ *
+ * @return the aggregation results, suitable for printing to the console
+ */
+ protected abstract String getStats();
+}
http://git-wip-us.apache.org/repos/asf/accumulo/blob/c56ef2ee/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
b/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
index 5021d66..b31df18 100644
---
a/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
+++
b/core/src/main/java/org/apache/accumulo/core/util/format/BinaryFormatter.java
@@ -16,7 +16,6 @@
*/
package org.apache.accumulo.core.util.format;
-import java.util.Iterator;
import java.util.Map.Entry;
svn commit: r1581455 - in /accumulo/site/branches/redesign14: content/index.mdtext templates/skeleton.html
Author: bhavanki
Date: Tue Mar 25 18:32:37 2014
New Revision: 1581455
URL: http://svn.apache.org/r1581455
Log:
Allow pages to not include h1 with title (like the index page).
Modified:
accumulo/site/branches/redesign14/content/index.mdtext
accumulo/site/branches/redesign14/templates/skeleton.html
Modified: accumulo/site/branches/redesign14/content/index.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/index.mdtext?rev=1581455r1=1581454r2=1581455view=diff
==
--- accumulo/site/branches/redesign14/content/index.mdtext (original)
+++ accumulo/site/branches/redesign14/content/index.mdtext Tue Mar 25 18:32:37
2014
@@ -1,4 +1,5 @@
Title: Apache Accumulo
+Skiph1fortitle: true
Nav: nav_index
Notice:Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
@@ -17,6 +18,7 @@ Notice:Licensed to the Apache Softwa
specific language governing permissions and limitations
under the License.
+br
div class=jumbotron style=text-align: center
p
The Apache Accumulotrade; sorted, distributed key/value store is a robust,
scalable, high performance data storage and retrieval system.
Modified: accumulo/site/branches/redesign14/templates/skeleton.html
URL:
http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/templates/skeleton.html?rev=1581455r1=1581454r2=1581455view=diff
==
--- accumulo/site/branches/redesign14/templates/skeleton.html (original)
+++ accumulo/site/branches/redesign14/templates/skeleton.html Tue Mar 25
18:32:37 2014
@@ -53,11 +53,6 @@
body
{% include nav.html %}
-!--
- div id=navigation
- {% filter markdown %}{% include sidenav.mdtext %}{% endfilter %}
- /div
---
div class=container-fluid
div class=row
@@ -67,7 +62,9 @@
/div
div id=content
+{% if not headers.skiph1fortitle %}
h1 class=title{% block title %}{{ headers.title }}{% endblock %}/h1
+{% endif %}
{% block content %}{{ content|markdown }}{% endblock %}
/div
svn commit: r903562 - /websites/staging/accumulo/trunk/content/
Author: buildbot Date: Tue Mar 25 18:32:42 2014 New Revision: 903562 Log: Staging update by buildbot for accumulo Modified: websites/staging/accumulo/trunk/content/ (props changed) Propchange: websites/staging/accumulo/trunk/content/ -- --- cms:source-revision (original) +++ cms:source-revision Tue Mar 25 18:32:42 2014 @@ -1 +1 @@ -1581434 +1581455
svn commit: r903571 - /websites/staging/accumulo/trunk/content/
Author: buildbot Date: Tue Mar 25 18:43:26 2014 New Revision: 903571 Log: Staging update by buildbot for accumulo Modified: websites/staging/accumulo/trunk/content/ (props changed) Propchange: websites/staging/accumulo/trunk/content/ -- --- cms:source-revision (original) +++ cms:source-revision Tue Mar 25 18:43:26 2014 @@ -1 +1 @@ -1581455 +1581459
svn commit: r1581459 - in /accumulo/site/branches/redesign14: content/1.4/examples/index.mdtext content/1.5/examples/index.mdtext templates/nav.html templates/skeleton.html
Author: bhavanki
Date: Tue Mar 25 18:43:21 2014
New Revision: 1581459
URL: http://svn.apache.org/r1581459
Log:
Add active nav indicators for example index pages, and properly support pages
without a Nav header.
Modified:
accumulo/site/branches/redesign14/content/1.4/examples/index.mdtext
accumulo/site/branches/redesign14/content/1.5/examples/index.mdtext
accumulo/site/branches/redesign14/templates/nav.html
accumulo/site/branches/redesign14/templates/skeleton.html
Modified: accumulo/site/branches/redesign14/content/1.4/examples/index.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/1.4/examples/index.mdtext?rev=1581459r1=1581458r2=1581459view=diff
==
--- accumulo/site/branches/redesign14/content/1.4/examples/index.mdtext
(original)
+++ accumulo/site/branches/redesign14/content/1.4/examples/index.mdtext Tue Mar
25 18:43:21 2014
@@ -1,4 +1,5 @@
Title: Apache Accumulo Examples
+Nav: nav_examples_1_4
Notice:Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
Modified: accumulo/site/branches/redesign14/content/1.5/examples/index.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/1.5/examples/index.mdtext?rev=1581459r1=1581458r2=1581459view=diff
==
--- accumulo/site/branches/redesign14/content/1.5/examples/index.mdtext
(original)
+++ accumulo/site/branches/redesign14/content/1.5/examples/index.mdtext Tue Mar
25 18:43:21 2014
@@ -1,4 +1,5 @@
Title: Apache Accumulo Examples
+Nav: nav_examples_1_5
Notice:Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
Modified: accumulo/site/branches/redesign14/templates/nav.html
URL:
http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/templates/nav.html?rev=1581459r1=1581458r2=1581459view=diff
==
--- accumulo/site/branches/redesign14/templates/nav.html (original)
+++ accumulo/site/branches/redesign14/templates/nav.html Tue Mar 25 18:43:21
2014
@@ -56,8 +56,8 @@
lia href=/1.5/apidocs1.5/a/li
li class=divider/li
li class=dropdown-headerExamples/li
-lia href=/1.4/examples1.4/a/li
-lia href=/1.5/examples1.5/a/li
+li id=nav_examples_1_4a href=/1.4/examples1.4/a/li
+li id=nav_examples_1_5a href=/1.5/examples1.5/a/li
li class=divider/li
li id=nav_screenshotsa href=/screenshots.htmlScreenshots/a/li
li id=nav_papersa href=/papers.htmlPapers amp; Other Links/a/li
Modified: accumulo/site/branches/redesign14/templates/skeleton.html
URL:
http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/templates/skeleton.html?rev=1581459r1=1581458r2=1581459view=diff
==
--- accumulo/site/branches/redesign14/templates/skeleton.html (original)
+++ accumulo/site/branches/redesign14/templates/skeleton.html Tue Mar 25
18:43:21 2014
@@ -89,7 +89,9 @@
/div
script type=text/javascript
+{% if headers.nav %}
$(#{{headers.nav}}).addClass(active);
+{% endif %}
/script
/body
/html
svn commit: r903580 - /websites/staging/accumulo/trunk/content/
Author: buildbot Date: Tue Mar 25 19:01:29 2014 New Revision: 903580 Log: Staging update by buildbot for accumulo Modified: websites/staging/accumulo/trunk/content/ (props changed) Propchange: websites/staging/accumulo/trunk/content/ -- --- cms:source-revision (original) +++ cms:source-revision Tue Mar 25 19:01:29 2014 @@ -1 +1 @@ -1581459 +1581464
svn commit: r1581464 - /accumulo/site/branches/redesign14/content/screenshots.mdtext
Author: bhavanki Date: Tue Mar 25 19:01:22 2014 New Revision: 1581464 URL: http://svn.apache.org/r1581464 Log: Fix img links in screenshots.mdtext. Modified: accumulo/site/branches/redesign14/content/screenshots.mdtext Modified: accumulo/site/branches/redesign14/content/screenshots.mdtext URL: http://svn.apache.org/viewvc/accumulo/site/branches/redesign14/content/screenshots.mdtext?rev=1581464r1=1581463r2=1581464view=diff == --- accumulo/site/branches/redesign14/content/screenshots.mdtext (original) +++ accumulo/site/branches/redesign14/content/screenshots.mdtext Tue Mar 25 19:01:22 2014 @@ -19,11 +19,11 @@ Notice:Licensed to the Apache Softwa The following is a screenshot of the Apache Accumulotrade; monitor overview web page. Accumulo was running on an eleven node cluster. The continuous ingest test suite was running to generate load. Ten continuous ingest clients were running. -img src=/images/overview2.png / +img src=/images/overview2.png / The following screenshot shows the monitor master page. This page gives information about individual tables within accumulo. Continuous ingest is running against the table ci. -img src=/images/master2.png / +img src=/images/master2.png / The data generated by the continuous ingest test suite looks like the following. The data is random, but forms many giant linked list. Each ingest client continually generates linked list containing 25 million nodes. In the shell session below a few scans are issued to start following the linked list that the first node in the table is part of.
[3/3] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/8291b6ce Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/8291b6ce Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/8291b6ce Branch: refs/heads/master Commit: 8291b6ce1ac765a56ebbafdc6cc4aff03b9b6f57 Parents: a749fad 1c51619 Author: Josh Elser els...@apache.org Authored: Tue Mar 25 12:52:46 2014 -0700 Committer: Josh Elser els...@apache.org Committed: Tue Mar 25 12:52:46 2014 -0700 -- .../chapters/troubleshooting.tex| 64 1 file changed, 64 insertions(+) --
[1/3] git commit: ACCUMULO-1218 Overview on how to recover an instance from failed zookeepers
Repository: accumulo
Updated Branches:
refs/heads/1.6.0-SNAPSHOT c56ef2eec - 1c5161933
refs/heads/master a749fad02 - 8291b6ce1
ACCUMULO-1218 Overview on how to recover an instance from failed zookeepers
Ample warning given to the reintroduction of stale data (from files
that should be deleted but have not yet been deleted) or omission
of new data only present in WALs.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/1c516193
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/1c516193
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/1c516193
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: 1c516193342acfa838df25bc880e3c594a659282
Parents: c56ef2e
Author: Josh Elser els...@apache.org
Authored: Mon Mar 24 18:42:00 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 12:49:31 2014 -0700
--
.../chapters/troubleshooting.tex| 64
1 file changed, 64 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/1c516193/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
--
diff --git
a/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
b/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
index 98cf549..a6a86dc 100644
--- a/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
+++ b/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
@@ -561,6 +561,7 @@ processes should be stable on the order of months and not
require frequent resta
\section{Advanced System Recovery}
+\subsection{HDFS Failure}
Q. I had disasterous HDFS failure. After bringing everything back up, several
tablets refuse to go online.
Data written to tablets is written into memory before being written into
indexed files. In case the server
@@ -641,6 +642,69 @@ but the basic approach is:
\item Import the directories under \texttt{/corrupt/tables/id} into the new
instance
\end{itemize}
+
+\subsection{ZooKeeper Failure}
+Q. I lost my ZooKeeper quorum (hardware failure), but HDFS is still intact.
How can I recover my Accumulo instance?
+
+ZooKeeper, in addition to its lock-service capabilities, also serves to
bootstrap an Accumulo
+instance from some location in HDFS. It contains the pointers to the root
tablet in HDFS which
+is then used to load the Accumulo metadata tablets, which then loads all user
tables. ZooKeeper
+also stores all namespace and table configuration, the user database, the
mapping of table IDs to
+table names, and more across Accumulo restarts.
+
+Presently, the only way to recover such an instance is to initialize a new
instance and import all
+of the old data into the new instance. The easiest way to tackle this problem
is to first recreate
+the mapping of table ID to table name and then recreate each of those tables
in the new instance.
+Set any necessary configuration on the new tables and add some split points to
the tables to close
+the gap between how many splits the old table had and no splits.
+
+The directory structure in HDFS for tables will follow the general structure:
+
+\small
+\begin{verbatim}
+ /accumulo
+ /accumulo/tables/
+ /accumulo/tables/1
+ /accumulo/tables/1/default_tablet/A01.rf
+ /accumulo/tables/1/t-1/A02.rf
+ /accumulo/tables/1/t-1/A03.rf
+ /accumulo/tables/2/default_tablet/A04.rf
+ /accumulo/tables/2/t-1/A05.rf
+\end{verbatim}
+\normalsize
+
+For each table, make a new directory that you can move (or copy if you have
the HDFS space to do so)
+all of the rfiles for a given table into. For example, to process the table
with an ID of ``1``, make a new directory,
+say ``/new-table-1`` and then copy all files from
``/accumulo/tables/1/*/*.rf`` into that directory. Additionally,
+make a directory, ``/new-table-1-failures``, for any failures during the
import process. Then, issue the import
+command using the Accumulo shell into the new table, telling Accumulo to not
re-set the timestamp:
+
+\small
+\begin{verbatim}
+user@instance new_table importdirectory /new-table-1 /new-table-1-failures
false
+\end{verbatim}
+\normalsize
+
+Any RFiles which were failed to be loaded will be placed in
``/new-table-1-failures``. Rfiles that were successfully
+imported will no longer exist in ``/new-table-1``. For failures, move them
back to the import directory and retry
+the ``importdirectory`` command.
+
+It is \textbf{extremely} important to note that this approach may introduce
stale data back into
+the tables. For a few reasons, RFiles may exist in the table directory which
are candidates for deletion but have
+not yet been deleted. Additionally, deleted data which was not compacted away,
[2/3] git commit: ACCUMULO-1218 Overview on how to recover an instance from failed zookeepers
ACCUMULO-1218 Overview on how to recover an instance from failed zookeepers
Ample warning given to the reintroduction of stale data (from files
that should be deleted but have not yet been deleted) or omission
of new data only present in WALs.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/1c516193
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/1c516193
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/1c516193
Branch: refs/heads/master
Commit: 1c516193342acfa838df25bc880e3c594a659282
Parents: c56ef2e
Author: Josh Elser els...@apache.org
Authored: Mon Mar 24 18:42:00 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 12:49:31 2014 -0700
--
.../chapters/troubleshooting.tex| 64
1 file changed, 64 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/1c516193/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
--
diff --git
a/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
b/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
index 98cf549..a6a86dc 100644
--- a/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
+++ b/docs/src/main/latex/accumulo_user_manual/chapters/troubleshooting.tex
@@ -561,6 +561,7 @@ processes should be stable on the order of months and not
require frequent resta
\section{Advanced System Recovery}
+\subsection{HDFS Failure}
Q. I had disasterous HDFS failure. After bringing everything back up, several
tablets refuse to go online.
Data written to tablets is written into memory before being written into
indexed files. In case the server
@@ -641,6 +642,69 @@ but the basic approach is:
\item Import the directories under \texttt{/corrupt/tables/id} into the new
instance
\end{itemize}
+
+\subsection{ZooKeeper Failure}
+Q. I lost my ZooKeeper quorum (hardware failure), but HDFS is still intact.
How can I recover my Accumulo instance?
+
+ZooKeeper, in addition to its lock-service capabilities, also serves to
bootstrap an Accumulo
+instance from some location in HDFS. It contains the pointers to the root
tablet in HDFS which
+is then used to load the Accumulo metadata tablets, which then loads all user
tables. ZooKeeper
+also stores all namespace and table configuration, the user database, the
mapping of table IDs to
+table names, and more across Accumulo restarts.
+
+Presently, the only way to recover such an instance is to initialize a new
instance and import all
+of the old data into the new instance. The easiest way to tackle this problem
is to first recreate
+the mapping of table ID to table name and then recreate each of those tables
in the new instance.
+Set any necessary configuration on the new tables and add some split points to
the tables to close
+the gap between how many splits the old table had and no splits.
+
+The directory structure in HDFS for tables will follow the general structure:
+
+\small
+\begin{verbatim}
+ /accumulo
+ /accumulo/tables/
+ /accumulo/tables/1
+ /accumulo/tables/1/default_tablet/A01.rf
+ /accumulo/tables/1/t-1/A02.rf
+ /accumulo/tables/1/t-1/A03.rf
+ /accumulo/tables/2/default_tablet/A04.rf
+ /accumulo/tables/2/t-1/A05.rf
+\end{verbatim}
+\normalsize
+
+For each table, make a new directory that you can move (or copy if you have
the HDFS space to do so)
+all of the rfiles for a given table into. For example, to process the table
with an ID of ``1``, make a new directory,
+say ``/new-table-1`` and then copy all files from
``/accumulo/tables/1/*/*.rf`` into that directory. Additionally,
+make a directory, ``/new-table-1-failures``, for any failures during the
import process. Then, issue the import
+command using the Accumulo shell into the new table, telling Accumulo to not
re-set the timestamp:
+
+\small
+\begin{verbatim}
+user@instance new_table importdirectory /new-table-1 /new-table-1-failures
false
+\end{verbatim}
+\normalsize
+
+Any RFiles which were failed to be loaded will be placed in
``/new-table-1-failures``. Rfiles that were successfully
+imported will no longer exist in ``/new-table-1``. For failures, move them
back to the import directory and retry
+the ``importdirectory`` command.
+
+It is \textbf{extremely} important to note that this approach may introduce
stale data back into
+the tables. For a few reasons, RFiles may exist in the table directory which
are candidates for deletion but have
+not yet been deleted. Additionally, deleted data which was not compacted away,
but still exists in write-ahead logs if
+the original instance was somehow recoverable, will be re-introduced in the
new instance. Table
[3/3] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/39e09cbc Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/39e09cbc Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/39e09cbc Branch: refs/heads/master Commit: 39e09cbc02cd8f7eb35d8f151aae7b1cfc642350 Parents: 8291b6c 55b17e4 Author: Mike Drob md...@cloudera.com Authored: Tue Mar 25 15:56:16 2014 -0400 Committer: Mike Drob md...@cloudera.com Committed: Tue Mar 25 15:56:16 2014 -0400 -- .../apache/accumulo/core/util/format/DateStringFormatter.java | 6 ++ .../accumulo/core/util/format/DateStringFormatterTest.java | 4 +++- 2 files changed, 9 insertions(+), 1 deletion(-) --
[1/3] git commit: ACCUMULO-2503 fix time zone assertion
Repository: accumulo
Updated Branches:
refs/heads/1.6.0-SNAPSHOT 1c5161933 - 55b17e476
refs/heads/master 8291b6ce1 - 39e09cbc0
ACCUMULO-2503 fix time zone assertion
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/55b17e47
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/55b17e47
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/55b17e47
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: 55b17e476686e846cdfd519393ef506e8a35a234
Parents: 1c51619
Author: Mike Drob md...@cloudera.com
Authored: Tue Mar 25 15:51:40 2014 -0400
Committer: Mike Drob md...@cloudera.com
Committed: Tue Mar 25 15:55:56 2014 -0400
--
.../apache/accumulo/core/util/format/DateStringFormatter.java | 6 ++
.../accumulo/core/util/format/DateStringFormatterTest.java | 4 +++-
2 files changed, 9 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/55b17e47/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
b/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
index 9a7febc..037ddb0 100644
---
a/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
+++
b/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
@@ -19,6 +19,7 @@ package org.apache.accumulo.core.util.format;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Map.Entry;
+import java.util.TimeZone;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Value;
@@ -26,6 +27,7 @@ import org.apache.accumulo.core.data.Value;
public class DateStringFormatter implements Formatter {
private boolean printTimestamps = false;
private DefaultFormatter defaultFormatter = new DefaultFormatter();
+
public static final String DATE_FORMAT = /MM/dd HH:mm:ss.SSS;
// SimpleDataFormat is not thread safe
private static final ThreadLocalDateFormat formatter = new
ThreadLocalDateFormat() {
@@ -58,4 +60,8 @@ public class DateStringFormatter implements Formatter {
public void remove() {
defaultFormatter.remove();
}
+
+ public void setTimeZone(TimeZone zone) {
+formatter.get().setTimeZone(zone);
+ }
}
http://git-wip-us.apache.org/repos/asf/accumulo/blob/55b17e47/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
--
diff --git
a/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
b/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
index 69b2e11..1b121f3 100644
---
a/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
+++
b/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
@@ -20,6 +20,7 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import java.util.Map;
+import java.util.TimeZone;
import java.util.TreeMap;
import org.apache.accumulo.core.data.Key;
@@ -42,9 +43,10 @@ public class DateStringFormatterTest {
@Test
public void testTimestamps() {
formatter.initialize(data.entrySet(), true);
+formatter.setTimeZone(TimeZone.getTimeZone(UTC));
assertTrue(formatter.hasNext());
-assertTrue(formatter.next().endsWith(1969/12/31 19:00:00.000));
+assertTrue(formatter.next().endsWith(1970/01/01 00:00:00.000));
}
@Test
[2/3] git commit: ACCUMULO-2503 fix time zone assertion
ACCUMULO-2503 fix time zone assertion
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/55b17e47
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/55b17e47
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/55b17e47
Branch: refs/heads/master
Commit: 55b17e476686e846cdfd519393ef506e8a35a234
Parents: 1c51619
Author: Mike Drob md...@cloudera.com
Authored: Tue Mar 25 15:51:40 2014 -0400
Committer: Mike Drob md...@cloudera.com
Committed: Tue Mar 25 15:55:56 2014 -0400
--
.../apache/accumulo/core/util/format/DateStringFormatter.java | 6 ++
.../accumulo/core/util/format/DateStringFormatterTest.java | 4 +++-
2 files changed, 9 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/55b17e47/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
b/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
index 9a7febc..037ddb0 100644
---
a/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
+++
b/core/src/main/java/org/apache/accumulo/core/util/format/DateStringFormatter.java
@@ -19,6 +19,7 @@ package org.apache.accumulo.core.util.format;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Map.Entry;
+import java.util.TimeZone;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Value;
@@ -26,6 +27,7 @@ import org.apache.accumulo.core.data.Value;
public class DateStringFormatter implements Formatter {
private boolean printTimestamps = false;
private DefaultFormatter defaultFormatter = new DefaultFormatter();
+
public static final String DATE_FORMAT = /MM/dd HH:mm:ss.SSS;
// SimpleDataFormat is not thread safe
private static final ThreadLocalDateFormat formatter = new
ThreadLocalDateFormat() {
@@ -58,4 +60,8 @@ public class DateStringFormatter implements Formatter {
public void remove() {
defaultFormatter.remove();
}
+
+ public void setTimeZone(TimeZone zone) {
+formatter.get().setTimeZone(zone);
+ }
}
http://git-wip-us.apache.org/repos/asf/accumulo/blob/55b17e47/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
--
diff --git
a/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
b/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
index 69b2e11..1b121f3 100644
---
a/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
+++
b/core/src/test/java/org/apache/accumulo/core/util/format/DateStringFormatterTest.java
@@ -20,6 +20,7 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import java.util.Map;
+import java.util.TimeZone;
import java.util.TreeMap;
import org.apache.accumulo.core.data.Key;
@@ -42,9 +43,10 @@ public class DateStringFormatterTest {
@Test
public void testTimestamps() {
formatter.initialize(data.entrySet(), true);
+formatter.setTimeZone(TimeZone.getTimeZone(UTC));
assertTrue(formatter.hasNext());
-assertTrue(formatter.next().endsWith(1969/12/31 19:00:00.000));
+assertTrue(formatter.next().endsWith(1970/01/01 00:00:00.000));
}
@Test
[13/14] git commit: Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT
Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/fa49e804 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/fa49e804 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/fa49e804 Branch: refs/heads/1.6.0-SNAPSHOT Commit: fa49e8041b3500320af5e376e3d948d8b31c8a26 Parents: 55b17e4 5413823 Author: Josh Elser els...@apache.org Authored: Tue Mar 25 14:23:13 2014 -0700 Committer: Josh Elser els...@apache.org Committed: Tue Mar 25 14:23:13 2014 -0700 -- .../accumulo/core/iterators/Combiner.java | 92 +++- 1 file changed, 51 insertions(+), 41 deletions(-) --
[04/14] git commit: Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT
Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/22f95590 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/22f95590 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/22f95590 Branch: refs/heads/1.6.0-SNAPSHOT Commit: 22f9559002df977fb18819c5935bfd1841be0e5c Parents: 00355d0 2ab011c Author: Mike Drob md...@cloudera.com Authored: Tue Mar 25 12:22:01 2014 -0400 Committer: Mike Drob md...@cloudera.com Committed: Tue Mar 25 12:22:01 2014 -0400 -- --
[12/14] git commit: Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT
Merge branch '1.5.2-SNAPSHOT' into 1.6.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/fa49e804 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/fa49e804 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/fa49e804 Branch: refs/heads/master Commit: fa49e8041b3500320af5e376e3d948d8b31c8a26 Parents: 55b17e4 5413823 Author: Josh Elser els...@apache.org Authored: Tue Mar 25 14:23:13 2014 -0700 Committer: Josh Elser els...@apache.org Committed: Tue Mar 25 14:23:13 2014 -0700 -- .../accumulo/core/iterators/Combiner.java | 92 +++- 1 file changed, 51 insertions(+), 41 deletions(-) --
[03/14] git commit: Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT
Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/22f95590 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/22f95590 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/22f95590 Branch: refs/heads/master Commit: 22f9559002df977fb18819c5935bfd1841be0e5c Parents: 00355d0 2ab011c Author: Mike Drob md...@cloudera.com Authored: Tue Mar 25 12:22:01 2014 -0400 Committer: Mike Drob md...@cloudera.com Committed: Tue Mar 25 12:22:01 2014 -0400 -- --
[10/14] git commit: Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT
Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT
Conflicts:
core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/5413823d
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/5413823d
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/5413823d
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: 5413823d6ca91454c81a48730fa75633bd463745
Parents: 22f9559 88761e0
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 13:55:54 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 13:55:54 2014 -0700
--
.../accumulo/core/iterators/Combiner.java | 92 +++-
1 file changed, 51 insertions(+), 41 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/5413823d/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
--
diff --cc core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
index 21cbfe5,000..58071cf
mode 100644,00..100644
--- a/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
+++ b/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
@@@ -1,313 -1,0 +1,323 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.iterators;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.NoSuchElementException;
+
+import org.apache.accumulo.core.client.IteratorSetting;
+import org.apache.accumulo.core.client.IteratorSetting.Column;
++import org.apache.accumulo.core.client.ScannerBase;
+import org.apache.accumulo.core.data.ByteSequence;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.PartialKey;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.iterators.conf.ColumnSet;
++import org.apache.hadoop.io.Text;
+import org.apache.log4j.Logger;
+
+/**
- * A SortedKeyValueIterator that combines the Values for different versions
(timestamps) of a Key into a single Value. Combiner will replace one or more
- * versions of a Key and their Values with the most recent Key and a Value
which is the result of the reduce method.
++ * A SortedKeyValueIterator that combines the Values for different versions
(timestamp) of a Key within a row into a single Value. Combiner will replace
one or
++ * more versions of a Key and their Values with the most recent Key and a
Value which is the result of the reduce method. An {@link
IteratorSetting.Column}
++ * which only specifies a column family will combine all Keys in that column
family individually. Similarly, a {@link IteratorSetting.Column} which
specifies a
++ * column family and column qualifier will combine all Keys in column family
and qualifier individually. Combination is only ever performed on multiple
versions
++ * and not across column qualifiers or column visibilities.
+ *
- * Subclasses must implement a reduce method: {@code public Value reduce(Key
key, IteratorValue iter)}.
++ * Implementations must provide a reduce method: {@code public Value
reduce(Key key, IteratorValue iter)}.
+ *
+ * This reduce method will be passed the most recent Key and an iterator over
the Values for all non-deleted versions of that Key. A combiner will not combine
+ * keys that differ by more than the timestamp.
++ *
++ * This class and its implementations do not automatically filter out
unwanted columns from those being combined, thus it is generally recommended to
use a
++ * {@link Combiner} implementation with the {@link
ScannerBase#fetchColumnFamily(Text)} or {@link ScannerBase#fetchColumn(Text,
Text)} methods.
+ */
+public abstract class Combiner extends WrappingIterator implements
OptionDescriber {
+ static final Logger log =
[07/14] git commit: ACCUMULO-2539 Expand javadoc for Combiner to be more specific about combination implementation
ACCUMULO-2539 Expand javadoc for Combiner to be more specific about combination
implementation
Explicitly note that, most times, fetchColumnFamily or fetchColumns is desired
and that columns
are only combined within a row. Only multiple versions for otherwise equal Keys
are combined.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/88761e05
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/88761e05
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/88761e05
Branch: refs/heads/1.4.5-SNAPSHOT
Commit: 88761e05bce7e71c89b1a6033ba74edfcd8db9a6
Parents: 2ab011c
Author: Josh Elser els...@apache.org
Authored: Mon Mar 24 19:10:31 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 13:34:48 2014 -0700
--
.../accumulo/core/iterators/Combiner.java | 94 +++-
1 file changed, 52 insertions(+), 42 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/88761e05/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
--
diff --git
a/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
b/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
index 584eb14..083e93e 100644
--- a/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
+++ b/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
@@ -26,27 +26,35 @@ import java.util.NoSuchElementException;
import org.apache.accumulo.core.client.IteratorSetting;
import org.apache.accumulo.core.client.IteratorSetting.Column;
+import org.apache.accumulo.core.client.ScannerBase;
import org.apache.accumulo.core.data.ByteSequence;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.PartialKey;
import org.apache.accumulo.core.data.Range;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.iterators.conf.ColumnSet;
+import org.apache.hadoop.io.Text;
import org.apache.log4j.Logger;
/**
- * A SortedKeyValueIterator that combines the Values for different versions of
a Key into a single Value. Combiner will replace one or more versions of a Key
- * and their Values with the most recent Key and a Value which is the result
of the reduce method.
+ * A SortedKeyValueIterator that combines the Values for different versions
(timestamp) of a Key within a row into a single Value. Combiner will replace
one or
+ * more versions of a Key and their Values with the most recent Key and a
Value which is the result of the reduce method. An {@link
IteratorSetting.Column}
+ * which only specifies a column family will combine all Keys in that column
family individually. Similarly, a {@link IteratorSetting.Column} which
specifies a
+ * column family and column qualifier will combine all Keys in column family
and qualifier individually. Combination is only ever performed on multiple
versions
+ * and not across column qualifiers or column visibilities.
*
- * Subclasses must implement a reduce method: {@code public Value reduce(Key
key, IteratorValue iter)}.
+ * Implementations must provide a reduce method: {@code public Value
reduce(Key key, IteratorValue iter)}.
*
* This reduce method will be passed the most recent Key and an iterator over
the Values for all non-deleted versions of that Key.
+ *
+ * This class and its implementations do not automatically filter out unwanted
columns from those being combined, thus it is generally recommended to use a
+ * {@link Combiner} implementation with the {@link
ScannerBase#fetchColumnFamily(Text)} or {@link ScannerBase#fetchColumn(Text,
Text)} methods.
*/
public abstract class Combiner extends WrappingIterator implements
OptionDescriber {
static final Logger log = Logger.getLogger(Combiner.class);
protected static final String COLUMNS_OPTION = columns;
protected static final String ALL_OPTION = all;
-
+
/**
* A Java Iterator that iterates over the Values for a given Key from a
source SortedKeyValueIterator.
*/
@@ -54,7 +62,7 @@ public abstract class Combiner extends WrappingIterator
implements OptionDescrib
Key topKey;
SortedKeyValueIteratorKey,Value source;
boolean hasNext;
-
+
/**
* Constructs an iterator over Values whose Keys are versions of the
current topKey of the source SortedKeyValueIterator.
*
@@ -66,11 +74,11 @@ public abstract class Combiner extends WrappingIterator
implements OptionDescrib
topKey = new Key(source.getTopKey());
hasNext = _hasNext();
}
-
+
private boolean _hasNext() {
return source.hasTop() !source.getTopKey().isDeleted()
topKey.equals(source.getTopKey(), PartialKey.ROW_COLFAM_COLQUAL_COLVIS);
}
-
+
[09/14] git commit: Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT
Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT
Conflicts:
core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/5413823d
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/5413823d
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/5413823d
Branch: refs/heads/master
Commit: 5413823d6ca91454c81a48730fa75633bd463745
Parents: 22f9559 88761e0
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 13:55:54 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 13:55:54 2014 -0700
--
.../accumulo/core/iterators/Combiner.java | 92 +++-
1 file changed, 51 insertions(+), 41 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/5413823d/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
--
diff --cc core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
index 21cbfe5,000..58071cf
mode 100644,00..100644
--- a/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
+++ b/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
@@@ -1,313 -1,0 +1,323 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.iterators;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.NoSuchElementException;
+
+import org.apache.accumulo.core.client.IteratorSetting;
+import org.apache.accumulo.core.client.IteratorSetting.Column;
++import org.apache.accumulo.core.client.ScannerBase;
+import org.apache.accumulo.core.data.ByteSequence;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.PartialKey;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.iterators.conf.ColumnSet;
++import org.apache.hadoop.io.Text;
+import org.apache.log4j.Logger;
+
+/**
- * A SortedKeyValueIterator that combines the Values for different versions
(timestamps) of a Key into a single Value. Combiner will replace one or more
- * versions of a Key and their Values with the most recent Key and a Value
which is the result of the reduce method.
++ * A SortedKeyValueIterator that combines the Values for different versions
(timestamp) of a Key within a row into a single Value. Combiner will replace
one or
++ * more versions of a Key and their Values with the most recent Key and a
Value which is the result of the reduce method. An {@link
IteratorSetting.Column}
++ * which only specifies a column family will combine all Keys in that column
family individually. Similarly, a {@link IteratorSetting.Column} which
specifies a
++ * column family and column qualifier will combine all Keys in column family
and qualifier individually. Combination is only ever performed on multiple
versions
++ * and not across column qualifiers or column visibilities.
+ *
- * Subclasses must implement a reduce method: {@code public Value reduce(Key
key, IteratorValue iter)}.
++ * Implementations must provide a reduce method: {@code public Value
reduce(Key key, IteratorValue iter)}.
+ *
+ * This reduce method will be passed the most recent Key and an iterator over
the Values for all non-deleted versions of that Key. A combiner will not combine
+ * keys that differ by more than the timestamp.
++ *
++ * This class and its implementations do not automatically filter out
unwanted columns from those being combined, thus it is generally recommended to
use a
++ * {@link Combiner} implementation with the {@link
ScannerBase#fetchColumnFamily(Text)} or {@link ScannerBase#fetchColumn(Text,
Text)} methods.
+ */
+public abstract class Combiner extends WrappingIterator implements
OptionDescriber {
+ static final Logger log =
[06/14] git commit: ACCUMULO-2539 Expand javadoc for Combiner to be more specific about combination implementation
ACCUMULO-2539 Expand javadoc for Combiner to be more specific about combination
implementation
Explicitly note that, most times, fetchColumnFamily or fetchColumns is desired
and that columns
are only combined within a row. Only multiple versions for otherwise equal Keys
are combined.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/88761e05
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/88761e05
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/88761e05
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: 88761e05bce7e71c89b1a6033ba74edfcd8db9a6
Parents: 2ab011c
Author: Josh Elser els...@apache.org
Authored: Mon Mar 24 19:10:31 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 13:34:48 2014 -0700
--
.../accumulo/core/iterators/Combiner.java | 94 +++-
1 file changed, 52 insertions(+), 42 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/88761e05/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
--
diff --git
a/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
b/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
index 584eb14..083e93e 100644
--- a/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
+++ b/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
@@ -26,27 +26,35 @@ import java.util.NoSuchElementException;
import org.apache.accumulo.core.client.IteratorSetting;
import org.apache.accumulo.core.client.IteratorSetting.Column;
+import org.apache.accumulo.core.client.ScannerBase;
import org.apache.accumulo.core.data.ByteSequence;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.PartialKey;
import org.apache.accumulo.core.data.Range;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.iterators.conf.ColumnSet;
+import org.apache.hadoop.io.Text;
import org.apache.log4j.Logger;
/**
- * A SortedKeyValueIterator that combines the Values for different versions of
a Key into a single Value. Combiner will replace one or more versions of a Key
- * and their Values with the most recent Key and a Value which is the result
of the reduce method.
+ * A SortedKeyValueIterator that combines the Values for different versions
(timestamp) of a Key within a row into a single Value. Combiner will replace
one or
+ * more versions of a Key and their Values with the most recent Key and a
Value which is the result of the reduce method. An {@link
IteratorSetting.Column}
+ * which only specifies a column family will combine all Keys in that column
family individually. Similarly, a {@link IteratorSetting.Column} which
specifies a
+ * column family and column qualifier will combine all Keys in column family
and qualifier individually. Combination is only ever performed on multiple
versions
+ * and not across column qualifiers or column visibilities.
*
- * Subclasses must implement a reduce method: {@code public Value reduce(Key
key, IteratorValue iter)}.
+ * Implementations must provide a reduce method: {@code public Value
reduce(Key key, IteratorValue iter)}.
*
* This reduce method will be passed the most recent Key and an iterator over
the Values for all non-deleted versions of that Key.
+ *
+ * This class and its implementations do not automatically filter out unwanted
columns from those being combined, thus it is generally recommended to use a
+ * {@link Combiner} implementation with the {@link
ScannerBase#fetchColumnFamily(Text)} or {@link ScannerBase#fetchColumn(Text,
Text)} methods.
*/
public abstract class Combiner extends WrappingIterator implements
OptionDescriber {
static final Logger log = Logger.getLogger(Combiner.class);
protected static final String COLUMNS_OPTION = columns;
protected static final String ALL_OPTION = all;
-
+
/**
* A Java Iterator that iterates over the Values for a given Key from a
source SortedKeyValueIterator.
*/
@@ -54,7 +62,7 @@ public abstract class Combiner extends WrappingIterator
implements OptionDescrib
Key topKey;
SortedKeyValueIteratorKey,Value source;
boolean hasNext;
-
+
/**
* Constructs an iterator over Values whose Keys are versions of the
current topKey of the source SortedKeyValueIterator.
*
@@ -66,11 +74,11 @@ public abstract class Combiner extends WrappingIterator
implements OptionDescrib
topKey = new Key(source.getTopKey());
hasNext = _hasNext();
}
-
+
private boolean _hasNext() {
return source.hasTop() !source.getTopKey().isDeleted()
topKey.equals(source.getTopKey(), PartialKey.ROW_COLFAM_COLQUAL_COLVIS);
}
-
+
[11/14] git commit: Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT
Merge branch '1.4.5-SNAPSHOT' into 1.5.2-SNAPSHOT
Conflicts:
core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/5413823d
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/5413823d
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/5413823d
Branch: refs/heads/1.5.2-SNAPSHOT
Commit: 5413823d6ca91454c81a48730fa75633bd463745
Parents: 22f9559 88761e0
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 13:55:54 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 13:55:54 2014 -0700
--
.../accumulo/core/iterators/Combiner.java | 92 +++-
1 file changed, 51 insertions(+), 41 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/5413823d/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
--
diff --cc core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
index 21cbfe5,000..58071cf
mode 100644,00..100644
--- a/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
+++ b/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
@@@ -1,313 -1,0 +1,323 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.iterators;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.NoSuchElementException;
+
+import org.apache.accumulo.core.client.IteratorSetting;
+import org.apache.accumulo.core.client.IteratorSetting.Column;
++import org.apache.accumulo.core.client.ScannerBase;
+import org.apache.accumulo.core.data.ByteSequence;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.PartialKey;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.iterators.conf.ColumnSet;
++import org.apache.hadoop.io.Text;
+import org.apache.log4j.Logger;
+
+/**
- * A SortedKeyValueIterator that combines the Values for different versions
(timestamps) of a Key into a single Value. Combiner will replace one or more
- * versions of a Key and their Values with the most recent Key and a Value
which is the result of the reduce method.
++ * A SortedKeyValueIterator that combines the Values for different versions
(timestamp) of a Key within a row into a single Value. Combiner will replace
one or
++ * more versions of a Key and their Values with the most recent Key and a
Value which is the result of the reduce method. An {@link
IteratorSetting.Column}
++ * which only specifies a column family will combine all Keys in that column
family individually. Similarly, a {@link IteratorSetting.Column} which
specifies a
++ * column family and column qualifier will combine all Keys in column family
and qualifier individually. Combination is only ever performed on multiple
versions
++ * and not across column qualifiers or column visibilities.
+ *
- * Subclasses must implement a reduce method: {@code public Value reduce(Key
key, IteratorValue iter)}.
++ * Implementations must provide a reduce method: {@code public Value
reduce(Key key, IteratorValue iter)}.
+ *
+ * This reduce method will be passed the most recent Key and an iterator over
the Values for all non-deleted versions of that Key. A combiner will not combine
+ * keys that differ by more than the timestamp.
++ *
++ * This class and its implementations do not automatically filter out
unwanted columns from those being combined, thus it is generally recommended to
use a
++ * {@link Combiner} implementation with the {@link
ScannerBase#fetchColumnFamily(Text)} or {@link ScannerBase#fetchColumn(Text,
Text)} methods.
+ */
+public abstract class Combiner extends WrappingIterator implements
OptionDescriber {
+ static final Logger log =
[08/14] git commit: ACCUMULO-2539 Expand javadoc for Combiner to be more specific about combination implementation
ACCUMULO-2539 Expand javadoc for Combiner to be more specific about combination
implementation
Explicitly note that, most times, fetchColumnFamily or fetchColumns is desired
and that columns
are only combined within a row. Only multiple versions for otherwise equal Keys
are combined.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/88761e05
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/88761e05
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/88761e05
Branch: refs/heads/master
Commit: 88761e05bce7e71c89b1a6033ba74edfcd8db9a6
Parents: 2ab011c
Author: Josh Elser els...@apache.org
Authored: Mon Mar 24 19:10:31 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 13:34:48 2014 -0700
--
.../accumulo/core/iterators/Combiner.java | 94 +++-
1 file changed, 52 insertions(+), 42 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/88761e05/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
--
diff --git
a/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
b/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
index 584eb14..083e93e 100644
--- a/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
+++ b/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
@@ -26,27 +26,35 @@ import java.util.NoSuchElementException;
import org.apache.accumulo.core.client.IteratorSetting;
import org.apache.accumulo.core.client.IteratorSetting.Column;
+import org.apache.accumulo.core.client.ScannerBase;
import org.apache.accumulo.core.data.ByteSequence;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.PartialKey;
import org.apache.accumulo.core.data.Range;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.iterators.conf.ColumnSet;
+import org.apache.hadoop.io.Text;
import org.apache.log4j.Logger;
/**
- * A SortedKeyValueIterator that combines the Values for different versions of
a Key into a single Value. Combiner will replace one or more versions of a Key
- * and their Values with the most recent Key and a Value which is the result
of the reduce method.
+ * A SortedKeyValueIterator that combines the Values for different versions
(timestamp) of a Key within a row into a single Value. Combiner will replace
one or
+ * more versions of a Key and their Values with the most recent Key and a
Value which is the result of the reduce method. An {@link
IteratorSetting.Column}
+ * which only specifies a column family will combine all Keys in that column
family individually. Similarly, a {@link IteratorSetting.Column} which
specifies a
+ * column family and column qualifier will combine all Keys in column family
and qualifier individually. Combination is only ever performed on multiple
versions
+ * and not across column qualifiers or column visibilities.
*
- * Subclasses must implement a reduce method: {@code public Value reduce(Key
key, IteratorValue iter)}.
+ * Implementations must provide a reduce method: {@code public Value
reduce(Key key, IteratorValue iter)}.
*
* This reduce method will be passed the most recent Key and an iterator over
the Values for all non-deleted versions of that Key.
+ *
+ * This class and its implementations do not automatically filter out unwanted
columns from those being combined, thus it is generally recommended to use a
+ * {@link Combiner} implementation with the {@link
ScannerBase#fetchColumnFamily(Text)} or {@link ScannerBase#fetchColumn(Text,
Text)} methods.
*/
public abstract class Combiner extends WrappingIterator implements
OptionDescriber {
static final Logger log = Logger.getLogger(Combiner.class);
protected static final String COLUMNS_OPTION = columns;
protected static final String ALL_OPTION = all;
-
+
/**
* A Java Iterator that iterates over the Values for a given Key from a
source SortedKeyValueIterator.
*/
@@ -54,7 +62,7 @@ public abstract class Combiner extends WrappingIterator
implements OptionDescrib
Key topKey;
SortedKeyValueIteratorKey,Value source;
boolean hasNext;
-
+
/**
* Constructs an iterator over Values whose Keys are versions of the
current topKey of the source SortedKeyValueIterator.
*
@@ -66,11 +74,11 @@ public abstract class Combiner extends WrappingIterator
implements OptionDescrib
topKey = new Key(source.getTopKey());
hasNext = _hasNext();
}
-
+
private boolean _hasNext() {
return source.hasTop() !source.getTopKey().isDeleted()
topKey.equals(source.getTopKey(), PartialKey.ROW_COLFAM_COLQUAL_COLVIS);
}
-
+
/**
[02/14] git commit: ACCUMULO-2169 tighten up rat check in build script
ACCUMULO-2169 tighten up rat check in build script
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2ab011c3
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2ab011c3
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2ab011c3
Branch: refs/heads/master
Commit: 2ab011c3a7dc2eaa52bd8845533af331e28ef7b5
Parents: edfbd2f
Author: Mike Drob md...@cloudera.com
Authored: Tue Mar 25 12:19:34 2014 -0400
Committer: Mike Drob md...@cloudera.com
Committed: Tue Mar 25 12:19:34 2014 -0400
--
src/assemble/build.sh | 31 +++
1 file changed, 23 insertions(+), 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/2ab011c3/src/assemble/build.sh
--
diff --git a/src/assemble/build.sh b/src/assemble/build.sh
index 867595b..6904535 100755
--- a/src/assemble/build.sh
+++ b/src/assemble/build.sh
@@ -15,6 +15,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+set -e
+
loc=`dirname $0`
loc=`cd $loc/../..; pwd`
@@ -38,16 +40,29 @@ runAt() {
( cd $1 ; echo in `pwd`; shift ; run $@ ) || fail
}
+verifyMissingLicenses() {
+ COUNT=$(grep -c '!' $1)
+ EXPECTED=$2
+ if [[ $COUNT != $EXPECTED ]]; then
+fail expected $EXPECTED files missing licenses, but saw $COUNT
+ fi
+}
+
+mvn -version | grep -q Java version: 1.6 || fail Need to use Java 6
+
run mvn -U -P distclean clean
-mvn org.apache.rat:apache-rat-plugin:0.10:check
-COUNT=`grep '!' target/rat.txt | wc -l`
-EXPECTED=51
-if [ $COUNT -ne $EXPECTED ]
-then
- fail expected $EXPECTED files missing licenses, but saw $COUNT
-fi
+
+mvn org.apache.rat:apache-rat-plugin:0.10:check || :
+verifyMissingLicenses target/rat.txt 51
+
+find src -name pom.xml -not -path 'src/server/*' \
+ -exec mvn org.apache.rat:apache-rat-plugin:0.10:check -f {} \;
+
+mvn org.apache.rat:apache-rat-plugin:0.10:check -f src/server/pom.xml || :
+verifyMissingLicenses src/server/target/rat.txt 27
+
#need to run mvn package twice to properly build docs/config.html
run mvn package
-run mvn package javadoc:aggregate javadoc:jar source:jar
+run mvn package javadoc:aggregate javadoc:jar source:jar -DskipTests
runAt ./src/server/src/main/c++ make
run mvn assembly:single -N
[14/14] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/111b56de Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/111b56de Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/111b56de Branch: refs/heads/master Commit: 111b56deee0f4b1dbddc000960983eddd722bd1a Parents: 39e09cb fa49e80 Author: Josh Elser els...@apache.org Authored: Tue Mar 25 14:23:18 2014 -0700 Committer: Josh Elser els...@apache.org Committed: Tue Mar 25 14:23:18 2014 -0700 -- .../accumulo/core/iterators/Combiner.java | 92 +++- 1 file changed, 51 insertions(+), 41 deletions(-) --
[05/14] git commit: ACCUMULO-2539 Expand javadoc for Combiner to be more specific about combination implementation
ACCUMULO-2539 Expand javadoc for Combiner to be more specific about combination
implementation
Explicitly note that, most times, fetchColumnFamily or fetchColumns is desired
and that columns
are only combined within a row. Only multiple versions for otherwise equal Keys
are combined.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/88761e05
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/88761e05
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/88761e05
Branch: refs/heads/1.5.2-SNAPSHOT
Commit: 88761e05bce7e71c89b1a6033ba74edfcd8db9a6
Parents: 2ab011c
Author: Josh Elser els...@apache.org
Authored: Mon Mar 24 19:10:31 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 13:34:48 2014 -0700
--
.../accumulo/core/iterators/Combiner.java | 94 +++-
1 file changed, 52 insertions(+), 42 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/88761e05/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
--
diff --git
a/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
b/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
index 584eb14..083e93e 100644
--- a/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
+++ b/src/core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java
@@ -26,27 +26,35 @@ import java.util.NoSuchElementException;
import org.apache.accumulo.core.client.IteratorSetting;
import org.apache.accumulo.core.client.IteratorSetting.Column;
+import org.apache.accumulo.core.client.ScannerBase;
import org.apache.accumulo.core.data.ByteSequence;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.PartialKey;
import org.apache.accumulo.core.data.Range;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.iterators.conf.ColumnSet;
+import org.apache.hadoop.io.Text;
import org.apache.log4j.Logger;
/**
- * A SortedKeyValueIterator that combines the Values for different versions of
a Key into a single Value. Combiner will replace one or more versions of a Key
- * and their Values with the most recent Key and a Value which is the result
of the reduce method.
+ * A SortedKeyValueIterator that combines the Values for different versions
(timestamp) of a Key within a row into a single Value. Combiner will replace
one or
+ * more versions of a Key and their Values with the most recent Key and a
Value which is the result of the reduce method. An {@link
IteratorSetting.Column}
+ * which only specifies a column family will combine all Keys in that column
family individually. Similarly, a {@link IteratorSetting.Column} which
specifies a
+ * column family and column qualifier will combine all Keys in column family
and qualifier individually. Combination is only ever performed on multiple
versions
+ * and not across column qualifiers or column visibilities.
*
- * Subclasses must implement a reduce method: {@code public Value reduce(Key
key, IteratorValue iter)}.
+ * Implementations must provide a reduce method: {@code public Value
reduce(Key key, IteratorValue iter)}.
*
* This reduce method will be passed the most recent Key and an iterator over
the Values for all non-deleted versions of that Key.
+ *
+ * This class and its implementations do not automatically filter out unwanted
columns from those being combined, thus it is generally recommended to use a
+ * {@link Combiner} implementation with the {@link
ScannerBase#fetchColumnFamily(Text)} or {@link ScannerBase#fetchColumn(Text,
Text)} methods.
*/
public abstract class Combiner extends WrappingIterator implements
OptionDescriber {
static final Logger log = Logger.getLogger(Combiner.class);
protected static final String COLUMNS_OPTION = columns;
protected static final String ALL_OPTION = all;
-
+
/**
* A Java Iterator that iterates over the Values for a given Key from a
source SortedKeyValueIterator.
*/
@@ -54,7 +62,7 @@ public abstract class Combiner extends WrappingIterator
implements OptionDescrib
Key topKey;
SortedKeyValueIteratorKey,Value source;
boolean hasNext;
-
+
/**
* Constructs an iterator over Values whose Keys are versions of the
current topKey of the source SortedKeyValueIterator.
*
@@ -66,11 +74,11 @@ public abstract class Combiner extends WrappingIterator
implements OptionDescrib
topKey = new Key(source.getTopKey());
hasNext = _hasNext();
}
-
+
private boolean _hasNext() {
return source.hasTop() !source.getTopKey().isDeleted()
topKey.equals(source.getTopKey(), PartialKey.ROW_COLFAM_COLQUAL_COLVIS);
}
-
+
[2/3] git commit: ACCUMULO-2503 Fix timezone-related failures in ShellTest
ACCUMULO-2503 Fix timezone-related failures in ShellTest
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/ce38eead
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/ce38eead
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/ce38eead
Branch: refs/heads/master
Commit: ce38eeadc41782fd62a0d46e63cbad93f7c997f6
Parents: fa49e80
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 16:07:22 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:07:22 2014 -0700
--
.../test/java/org/apache/accumulo/core/util/shell/ShellTest.java | 2 ++
1 file changed, 2 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/ce38eead/core/src/test/java/org/apache/accumulo/core/util/shell/ShellTest.java
--
diff --git
a/core/src/test/java/org/apache/accumulo/core/util/shell/ShellTest.java
b/core/src/test/java/org/apache/accumulo/core/util/shell/ShellTest.java
index 82d1f34..3d9b89e 100644
--- a/core/src/test/java/org/apache/accumulo/core/util/shell/ShellTest.java
+++ b/core/src/test/java/org/apache/accumulo/core/util/shell/ShellTest.java
@@ -27,6 +27,7 @@ import java.io.PrintWriter;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
+import java.util.TimeZone;
import jline.console.ConsoleReader;
@@ -103,6 +104,7 @@ public class ShellTest {
@Before
public void setup() throws IOException {
+TimeZone.setDefault(TimeZone.getTimeZone(UTC));
Shell.log.setLevel(Level.OFF);
output = new TestOutputStream();
input = new StringInputStream();
[3/3] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/eb72c5ba Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/eb72c5ba Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/eb72c5ba Branch: refs/heads/master Commit: eb72c5ba0831c5fafe01ac30a7e53f48eefcdba5 Parents: 111b56d ce38eea Author: Josh Elser els...@apache.org Authored: Tue Mar 25 16:07:56 2014 -0700 Committer: Josh Elser els...@apache.org Committed: Tue Mar 25 16:07:56 2014 -0700 -- .../test/java/org/apache/accumulo/core/util/shell/ShellTest.java | 2 ++ 1 file changed, 2 insertions(+) --
[11/11] git commit: Merge branch '1.6.0-SNAPSHOT'
Merge branch '1.6.0-SNAPSHOT' Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/36745b5a Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/36745b5a Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/36745b5a Branch: refs/heads/master Commit: 36745b5a4a351e503ad27a7d1654fb76c2bd86e3 Parents: eb72c5b f3f5480 Author: Josh Elser els...@apache.org Authored: Tue Mar 25 17:34:52 2014 -0700 Committer: Josh Elser els...@apache.org Committed: Tue Mar 25 17:34:52 2014 -0700 -- .../core/file/rfile/bcfile/PrintInfo.java | 1 + .../CachingHDFSSecretKeyEncryptionStrategy.java | 3 +- .../NonCachingSecretKeyEncryptionStrategy.java | 2 + .../apache/accumulo/server/util/FileUtil.java | 25 +- .../accumulo/server/util/FileUtilTest.java | 235 +++ 5 files changed, 253 insertions(+), 13 deletions(-) --
[05/11] git commit: ACCUMULO-2532 Expand on comment as to why we create this (otherwise curious) file
ACCUMULO-2532 Expand on comment as to why we create this (otherwise curious)
file
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/9678a2c5
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/9678a2c5
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/9678a2c5
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: 9678a2c51a33cc8c58969450e81c0acf40714930
Parents: 741
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 16:09:50 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:09:50 2014 -0700
--
.../src/main/java/org/apache/accumulo/server/util/FileUtil.java| 2 ++
1 file changed, 2 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/9678a2c5/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
--
diff --git
a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
index 4fee83d..0f7ac22 100644
--- a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
+++ b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
@@ -96,6 +96,8 @@ public class FileUtil {
fs.mkdirs(result);
// try to reserve the tmp dir
+ // In some versions of hadoop, two clients concurrently trying to create
the same directory might both return true
+ // Creating a file is not subject to this, so create a special file to
make sure we solely will use this directory
if (!fs.createNewFile(new Path(result, __reserve)))
result = null;
}
[04/11] git commit: ACCUMULO-2532 Relax access on cleanupIndexOp and add tests that show failure case.
ACCUMULO-2532 Relax access on cleanupIndexOp and add tests that show failure
case.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/6420dd3d
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/6420dd3d
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/6420dd3d
Branch: refs/heads/master
Commit: 6420dd3d49c97ee5c9a5e173914ca5374a26c8e5
Parents: ce38eea
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 15:27:43 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:08:33 2014 -0700
--
.../apache/accumulo/server/util/FileUtil.java | 2 +-
.../accumulo/server/util/FileUtilTest.java | 162 +++
2 files changed, 163 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/6420dd3d/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
--
diff --git
a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
index b1e39f0..fa186aa 100644
--- a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
+++ b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
@@ -360,7 +360,7 @@ public class FileUtil {
}
}
- private static void cleanupIndexOp(AccumuloConfiguration acuConf, Path
tmpDir, VolumeManager fs, ArrayListFileSKVIterator readers) throws
IOException {
+ protected static void cleanupIndexOp(AccumuloConfiguration acuConf, Path
tmpDir, VolumeManager fs, ArrayListFileSKVIterator readers) throws
IOException {
// close all of the index sequence files
for (FileSKVIterator r : readers) {
try {
http://git-wip-us.apache.org/repos/asf/accumulo/blob/6420dd3d/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
--
diff --git
a/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
b/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
new file mode 100644
index 000..546b4ee
--- /dev/null
+++
b/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
@@ -0,0 +1,162 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.util;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.DefaultConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.file.FileSKVIterator;
+import org.apache.accumulo.server.fs.VolumeManager;
+import org.apache.accumulo.server.fs.VolumeManagerImpl;
+import org.apache.commons.io.FileUtils;
+import org.apache.hadoop.fs.Path;
+import org.junit.Assert;
+import org.junit.Test;
+
+import com.google.common.io.Files;
+
+/**
+ *
+ */
+public class FileUtilTest {
+
+ @SuppressWarnings(deprecation)
+ @Test
+ public void testCleanupIndexOpWithDfsDir() throws IOException {
+File dfsDir = Files.createTempDir();
+
+try {
+ // And a unique tmp directory for each volume
+ File tmp1 = new File(dfsDir, tmp);
+ tmp1.mkdirs();
+ Path tmpPath1 = new Path(tmp1.toURI());
+
+ HashMapProperty,String testProps = new HashMapProperty,String();
+ testProps.put(Property.INSTANCE_DFS_DIR, dfsDir.getAbsolutePath());
+
+ AccumuloConfiguration testConf = new
FileUtilTestConfiguration(testProps);
+ VolumeManager fs = VolumeManagerImpl.getLocal(dfsDir.getAbsolutePath());
+
+ FileUtil.cleanupIndexOp(testConf, tmpPath1, fs, new
ArrayListFileSKVIterator());
+
+ Assert.assertFalse(Expected + tmp1 + to be cleaned up but it
wasn't, tmp1.exists());
+} finally {
+ FileUtils.deleteQuietly(dfsDir);
+}
+ }
+
+ @Test
+ public void
[06/11] git commit: ACCUMULO-2532 Expand on comment as to why we create this (otherwise curious) file
ACCUMULO-2532 Expand on comment as to why we create this (otherwise curious)
file
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/9678a2c5
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/9678a2c5
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/9678a2c5
Branch: refs/heads/master
Commit: 9678a2c51a33cc8c58969450e81c0acf40714930
Parents: 741
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 16:09:50 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:09:50 2014 -0700
--
.../src/main/java/org/apache/accumulo/server/util/FileUtil.java| 2 ++
1 file changed, 2 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/9678a2c5/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
--
diff --git
a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
index 4fee83d..0f7ac22 100644
--- a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
+++ b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
@@ -96,6 +96,8 @@ public class FileUtil {
fs.mkdirs(result);
// try to reserve the tmp dir
+ // In some versions of hadoop, two clients concurrently trying to create
the same directory might both return true
+ // Creating a file is not subject to this, so create a special file to
make sure we solely will use this directory
if (!fs.createNewFile(new Path(result, __reserve)))
result = null;
}
[09/11] git commit: ACCUMULO-2530 add TODO in code to reference ticket
ACCUMULO-2530 add TODO in code to reference ticket
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f3f54806
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f3f54806
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f3f54806
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: f3f54806d8af1bb12da549260338977ffd370e5d
Parents: f857e20
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 16:30:11 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:30:11 2014 -0700
--
.../security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java | 3 ++-
.../security/crypto/NonCachingSecretKeyEncryptionStrategy.java| 2 ++
2 files changed, 4 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/f3f54806/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
b/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
index d25d08c..e5ad13d 100644
---
a/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
+++
b/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
@@ -128,7 +128,8 @@ public class CachingHDFSSecretKeyEncryptionStrategy
implements SecretKeyEncrypti
if (pathToKeyName == null || pathToKeyName.equals()) {
pathToKeyName =
Property.CRYPTO_DEFAULT_KEY_STRATEGY_KEY_LOCATION.getDefaultValue();
}
-
+
+ // TODO ACCUMULO-2530 Ensure volumes a properly supported
Path pathToKey = new Path(pathToKeyName);
FileSystem fs = FileSystem.get(CachedConfiguration.getInstance());
http://git-wip-us.apache.org/repos/asf/accumulo/blob/f3f54806/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
b/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
index efcec01..67278bf 100644
---
a/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
+++
b/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
@@ -153,6 +153,7 @@ public class NonCachingSecretKeyEncryptionStrategy
implements SecretKeyEncryptio
Path pathToKey = new Path(fullPath);
try {
+ // TODO ACCUMULO-2530 Ensure volumes a properly supported
FileSystem fs = FileSystem.get(CachedConfiguration.getInstance());
doKeyEncryptionOperation(Cipher.WRAP_MODE, params, fullPath, pathToKey,
fs);
@@ -176,6 +177,7 @@ public class NonCachingSecretKeyEncryptionStrategy
implements SecretKeyEncryptio
Path pathToKey = new Path(pathToKeyName);
try {
+ // TODO ACCUMULO-2530 Ensure volumes a properly supported
FileSystem fs = FileSystem.get(CachedConfiguration.getInstance());
doKeyEncryptionOperation(Cipher.UNWRAP_MODE, params, pathToKeyName,
pathToKey, fs);
[02/11] git commit: ACCUMULO-2532 Relax access on cleanupIndexOp and add tests that show failure case.
ACCUMULO-2532 Relax access on cleanupIndexOp and add tests that show failure
case.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/6420dd3d
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/6420dd3d
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/6420dd3d
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: 6420dd3d49c97ee5c9a5e173914ca5374a26c8e5
Parents: ce38eea
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 15:27:43 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:08:33 2014 -0700
--
.../apache/accumulo/server/util/FileUtil.java | 2 +-
.../accumulo/server/util/FileUtilTest.java | 162 +++
2 files changed, 163 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/6420dd3d/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
--
diff --git
a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
index b1e39f0..fa186aa 100644
--- a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
+++ b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
@@ -360,7 +360,7 @@ public class FileUtil {
}
}
- private static void cleanupIndexOp(AccumuloConfiguration acuConf, Path
tmpDir, VolumeManager fs, ArrayListFileSKVIterator readers) throws
IOException {
+ protected static void cleanupIndexOp(AccumuloConfiguration acuConf, Path
tmpDir, VolumeManager fs, ArrayListFileSKVIterator readers) throws
IOException {
// close all of the index sequence files
for (FileSKVIterator r : readers) {
try {
http://git-wip-us.apache.org/repos/asf/accumulo/blob/6420dd3d/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
--
diff --git
a/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
b/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
new file mode 100644
index 000..546b4ee
--- /dev/null
+++
b/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
@@ -0,0 +1,162 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.util;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.DefaultConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.file.FileSKVIterator;
+import org.apache.accumulo.server.fs.VolumeManager;
+import org.apache.accumulo.server.fs.VolumeManagerImpl;
+import org.apache.commons.io.FileUtils;
+import org.apache.hadoop.fs.Path;
+import org.junit.Assert;
+import org.junit.Test;
+
+import com.google.common.io.Files;
+
+/**
+ *
+ */
+public class FileUtilTest {
+
+ @SuppressWarnings(deprecation)
+ @Test
+ public void testCleanupIndexOpWithDfsDir() throws IOException {
+File dfsDir = Files.createTempDir();
+
+try {
+ // And a unique tmp directory for each volume
+ File tmp1 = new File(dfsDir, tmp);
+ tmp1.mkdirs();
+ Path tmpPath1 = new Path(tmp1.toURI());
+
+ HashMapProperty,String testProps = new HashMapProperty,String();
+ testProps.put(Property.INSTANCE_DFS_DIR, dfsDir.getAbsolutePath());
+
+ AccumuloConfiguration testConf = new
FileUtilTestConfiguration(testProps);
+ VolumeManager fs = VolumeManagerImpl.getLocal(dfsDir.getAbsolutePath());
+
+ FileUtil.cleanupIndexOp(testConf, tmpPath1, fs, new
ArrayListFileSKVIterator());
+
+ Assert.assertFalse(Expected + tmp1 + to be cleaned up but it
wasn't, tmp1.exists());
+} finally {
+ FileUtils.deleteQuietly(dfsDir);
+}
+ }
+
+ @Test
+ public
[07/11] git commit: ACCUMULO-2462 Add TODO marker for bcfile.PrintInfo
ACCUMULO-2462 Add TODO marker for bcfile.PrintInfo
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f857e20e
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f857e20e
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f857e20e
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: f857e20e8d92eb8c9a7238704a25ac323a3a0336
Parents: 9678a2c
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 16:24:29 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:24:29 2014 -0700
--
.../java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java | 1 +
1 file changed, 1 insertion(+)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/f857e20e/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
b/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
index e2c8a6d..f387cc2 100644
---
a/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
+++
b/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
@@ -57,6 +57,7 @@ public class PrintInfo {
Configuration conf = new Configuration();
@SuppressWarnings(deprecation)
AccumuloConfiguration siteConf =
AccumuloConfiguration.getSiteConfiguration();
+// TODO ACCUMULO-2462 not going to operate as expected with volumes when a
path, not URI, is given
FileSystem hadoopFs = VolumeConfiguration.getDefaultVolume(conf,
siteConf).getFileSystem();
FileSystem localFs = FileSystem.getLocal(conf);
Path path = new Path(args[0]);
[10/11] git commit: ACCUMULO-2530 add TODO in code to reference ticket
ACCUMULO-2530 add TODO in code to reference ticket
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f3f54806
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f3f54806
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f3f54806
Branch: refs/heads/master
Commit: f3f54806d8af1bb12da549260338977ffd370e5d
Parents: f857e20
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 16:30:11 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:30:11 2014 -0700
--
.../security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java | 3 ++-
.../security/crypto/NonCachingSecretKeyEncryptionStrategy.java| 2 ++
2 files changed, 4 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/f3f54806/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
b/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
index d25d08c..e5ad13d 100644
---
a/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
+++
b/core/src/main/java/org/apache/accumulo/core/security/crypto/CachingHDFSSecretKeyEncryptionStrategy.java
@@ -128,7 +128,8 @@ public class CachingHDFSSecretKeyEncryptionStrategy
implements SecretKeyEncrypti
if (pathToKeyName == null || pathToKeyName.equals()) {
pathToKeyName =
Property.CRYPTO_DEFAULT_KEY_STRATEGY_KEY_LOCATION.getDefaultValue();
}
-
+
+ // TODO ACCUMULO-2530 Ensure volumes a properly supported
Path pathToKey = new Path(pathToKeyName);
FileSystem fs = FileSystem.get(CachedConfiguration.getInstance());
http://git-wip-us.apache.org/repos/asf/accumulo/blob/f3f54806/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
b/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
index efcec01..67278bf 100644
---
a/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
+++
b/core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java
@@ -153,6 +153,7 @@ public class NonCachingSecretKeyEncryptionStrategy
implements SecretKeyEncryptio
Path pathToKey = new Path(fullPath);
try {
+ // TODO ACCUMULO-2530 Ensure volumes a properly supported
FileSystem fs = FileSystem.get(CachedConfiguration.getInstance());
doKeyEncryptionOperation(Cipher.WRAP_MODE, params, fullPath, pathToKey,
fs);
@@ -176,6 +177,7 @@ public class NonCachingSecretKeyEncryptionStrategy
implements SecretKeyEncryptio
Path pathToKey = new Path(pathToKeyName);
try {
+ // TODO ACCUMULO-2530 Ensure volumes a properly supported
FileSystem fs = FileSystem.get(CachedConfiguration.getInstance());
doKeyEncryptionOperation(Cipher.UNWRAP_MODE, params, pathToKeyName,
pathToKey, fs);
[03/11] git commit: ACCUMULO-2532 Use VolumeManager to check for tmpDir existence.
ACCUMULO-2532 Use VolumeManager to check for tmpDir existence.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/741a
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/741a
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/741a
Branch: refs/heads/master
Commit: 741a189c807995c33d400aca4fed7b183d38
Parents: 6420dd3
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 15:56:03 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:08:33 2014 -0700
--
.../apache/accumulo/server/util/FileUtil.java | 21 ++--
.../accumulo/server/util/FileUtilTest.java | 117 +++
2 files changed, 105 insertions(+), 33 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/741a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
--
diff --git
a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
index fa186aa..4fee83d 100644
--- a/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
+++ b/server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java
@@ -45,6 +45,7 @@ import
org.apache.accumulo.core.iterators.SortedKeyValueIterator;
import org.apache.accumulo.core.iterators.system.MultiIterator;
import org.apache.accumulo.core.util.CachedConfiguration;
import org.apache.accumulo.core.util.LocalityGroupUtil;
+import org.apache.accumulo.core.volume.Volume;
import org.apache.accumulo.server.ServerConstants;
import org.apache.accumulo.server.fs.FileRef;
import org.apache.accumulo.server.fs.VolumeManager;
@@ -359,7 +360,7 @@ public class FileUtil {
cleanupIndexOp(acuConf, tmpDir, fs, readers);
}
}
-
+
protected static void cleanupIndexOp(AccumuloConfiguration acuConf, Path
tmpDir, VolumeManager fs, ArrayListFileSKVIterator readers) throws
IOException {
// close all of the index sequence files
for (FileSKVIterator r : readers) {
@@ -371,20 +372,18 @@ public class FileUtil {
log.error(e, e);
}
}
-
+
if (tmpDir != null) {
- // TODO ACCUMULO-2532 instance.dfs.dir is not required to be a part of
- // the path for tmpDir. Could result in tmpDir not being removed when it
should
- // Needs unit tests
- @SuppressWarnings(deprecation)
- String tmpPrefix = acuConf.get(Property.INSTANCE_DFS_DIR) + /tmp;
- if (tmpDir.toUri().getPath().startsWith(tmpPrefix))
+ Volume v = fs.getVolumeByPath(tmpDir);
+ if (v.getFileSystem().exists(tmpDir)) {
fs.deleteRecursively(tmpDir);
- else
-log.error(Did not delete tmp dir because it wasn't a tmp dir +
tmpDir);
+return;
+ }
+
+ log.error(Did not delete tmp dir because it wasn't a tmp dir +
tmpDir);
}
}
-
+
private static long countIndexEntries(AccumuloConfiguration acuConf, Text
prevEndRow, Text endRow, CollectionString mapFiles, boolean useIndex,
Configuration conf, VolumeManager fs, ArrayListFileSKVIterator
readers) throws IOException {
http://git-wip-us.apache.org/repos/asf/accumulo/blob/741a/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
--
diff --git
a/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
b/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
index 546b4ee..86678e1 100644
---
a/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
+++
b/server/base/src/test/java/org/apache/accumulo/server/util/FileUtilTest.java
@@ -44,21 +44,21 @@ public class FileUtilTest {
@Test
public void testCleanupIndexOpWithDfsDir() throws IOException {
File dfsDir = Files.createTempDir();
-
+
try {
// And a unique tmp directory for each volume
File tmp1 = new File(dfsDir, tmp);
tmp1.mkdirs();
Path tmpPath1 = new Path(tmp1.toURI());
-
+
HashMapProperty,String testProps = new HashMapProperty,String();
testProps.put(Property.INSTANCE_DFS_DIR, dfsDir.getAbsolutePath());
-
+
AccumuloConfiguration testConf = new
FileUtilTestConfiguration(testProps);
VolumeManager fs = VolumeManagerImpl.getLocal(dfsDir.getAbsolutePath());
-
+
FileUtil.cleanupIndexOp(testConf, tmpPath1, fs, new
ArrayListFileSKVIterator());
-
+
Assert.assertFalse(Expected + tmp1 + to be cleaned up but it
wasn't, tmp1.exists());
} finally {
FileUtils.deleteQuietly(dfsDir);
@@ -68,7 +68,7 @@ public class FileUtilTest {
@Test
public void
[08/11] git commit: ACCUMULO-2462 Add TODO marker for bcfile.PrintInfo
ACCUMULO-2462 Add TODO marker for bcfile.PrintInfo
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/f857e20e
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/f857e20e
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/f857e20e
Branch: refs/heads/master
Commit: f857e20e8d92eb8c9a7238704a25ac323a3a0336
Parents: 9678a2c
Author: Josh Elser els...@apache.org
Authored: Tue Mar 25 16:24:29 2014 -0700
Committer: Josh Elser els...@apache.org
Committed: Tue Mar 25 16:24:29 2014 -0700
--
.../java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java | 1 +
1 file changed, 1 insertion(+)
--
http://git-wip-us.apache.org/repos/asf/accumulo/blob/f857e20e/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
--
diff --git
a/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
b/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
index e2c8a6d..f387cc2 100644
---
a/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
+++
b/core/src/main/java/org/apache/accumulo/core/file/rfile/bcfile/PrintInfo.java
@@ -57,6 +57,7 @@ public class PrintInfo {
Configuration conf = new Configuration();
@SuppressWarnings(deprecation)
AccumuloConfiguration siteConf =
AccumuloConfiguration.getSiteConfiguration();
+// TODO ACCUMULO-2462 not going to operate as expected with volumes when a
path, not URI, is given
FileSystem hadoopFs = VolumeConfiguration.getDefaultVolume(conf,
siteConf).getFileSystem();
FileSystem localFs = FileSystem.getLocal(conf);
Path path = new Path(args[0]);
