[accumulo-testing] branch master updated: Avoid exceeding maximum log length in Travis
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git The following commit(s) were added to refs/heads/master by this push: new af79841 Avoid exceeding maximum log length in Travis af79841 is described below commit af798411045e7f1597df2889ac82cbdc403aa6e2 Author: Christopher Tubbs AuthorDate: Wed Jul 24 20:10:33 2019 -0400 Avoid exceeding maximum log length in Travis Suppress Downloading messages from Maven to avoid exceeding Travis CI's maximum log length of 4MB. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 1faba7e..1161180 100644 --- a/.travis.yml +++ b/.travis.yml @@ -19,4 +19,4 @@ cache: jdk: - openjdk8 install: true -script: mvn clean verify +script: mvn clean verify -B -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
[accumulo] branch 2.0 updated: Improve exception javadoc in TableOperations (#1289)
This is an automated email from the ASF dual-hosted git repository.
ctubbsii pushed a commit to branch 2.0
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/2.0 by this push:
new 58a72f8 Improve exception javadoc in TableOperations (#1289)
58a72f8 is described below
commit 58a72f8e77fbca082f63543f4b0d79d9cea145b4
Author: Mike Miller
AuthorDate: Wed Jul 24 19:59:10 2019 -0400
Improve exception javadoc in TableOperations (#1289)
---
.../java/org/apache/accumulo/core/client/admin/TableOperations.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git
a/core/src/main/java/org/apache/accumulo/core/client/admin/TableOperations.java
b/core/src/main/java/org/apache/accumulo/core/client/admin/TableOperations.java
index 0e645a3..750836c 100644
---
a/core/src/main/java/org/apache/accumulo/core/client/admin/TableOperations.java
+++
b/core/src/main/java/org/apache/accumulo/core/client/admin/TableOperations.java
@@ -507,7 +507,7 @@ public interface TableOperations {
* @param tableName
* the name of the table
* @throws AccumuloException
- * if a general error occurs
+ * if a general error occurs. Wrapped TableNotFoundException if
table does not exist.
* @throws AccumuloSecurityException
* if the user does not have permission
*/
@@ -543,7 +543,7 @@ public interface TableOperations {
* @param value
* the value to set a per-table property to
* @throws AccumuloException
- * if a general error occurs
+ * if a general error occurs. Wrapped TableNotFoundException if
table does not exist.
* @throws AccumuloSecurityException
* if the user does not have permission
*/
@@ -561,7 +561,7 @@ public interface TableOperations {
* @param property
* the name of a per-table property
* @throws AccumuloException
- * if a general error occurs
+ * if a general error occurs. Wrapped TableNotFoundException if
table does not exist.
* @throws AccumuloSecurityException
* if the user does not have permission
*/
[accumulo-testing] branch master updated: Use openjdk8 in Travis (oraclejdk8 gone)
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git The following commit(s) were added to refs/heads/master by this push: new 64a6e18 Use openjdk8 in Travis (oraclejdk8 gone) 64a6e18 is described below commit 64a6e18b91978c67f116ba678f220d482da83b1f Author: Christopher Tubbs AuthorDate: Wed Jul 24 19:52:14 2019 -0400 Use openjdk8 in Travis (oraclejdk8 gone) --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 3688f4a..1faba7e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -17,6 +17,6 @@ cache: directories: - $HOME/.m2 jdk: - - oraclejdk8 + - openjdk8 install: true script: mvn clean verify
[accumulo-testing] branch keith-turner-patch-1 deleted (was c50bd97)
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a change to branch keith-turner-patch-1 in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git. was c50bd97 Improve bulk CI docs The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[accumulo-testing] branch master updated: Improve bulk CI docs (#104)
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git The following commit(s) were added to refs/heads/master by this push: new 3d23c26 Improve bulk CI docs (#104) 3d23c26 is described below commit 3d23c26795f0b1746d38fc2c53c8ebdcc6f50b31 Author: Keith Turner AuthorDate: Wed Jul 24 19:50:06 2019 -0400 Improve bulk CI docs (#104) --- docs/bulk-test.md | 9 + 1 file changed, 9 insertions(+) diff --git a/docs/bulk-test.md b/docs/bulk-test.md index 17076e7..071cce8 100644 --- a/docs/bulk-test.md +++ b/docs/bulk-test.md @@ -30,6 +30,9 @@ for i in $(seq 1 10); do ./bin/cingest bulk /tmp/bt/$i done +# Optionally, copy data before importing. This can be useful in debugging problems. +hadoop distcp hdfs://$NAMENODE/tmp/bt hdfs://$NAMENODE/tmp/bt-copy + for i in $(seq 1 10); do ( echo table ci @@ -53,6 +56,12 @@ scan -t accumulo.metadata -b ~blip -e ~blip~ scan -t accumulo.metadata -c loaded ``` +Additionally check that no rfiles exists in the source dir. + +```bash +hadoop fs -ls -R /tmp/bt | grep rf +``` + The referenced counts output by `cingest verify` should equal : ```
[accumulo] branch 2.0 updated (c2cfa19 -> b9cc144)
This is an automated email from the ASF dual-hosted git repository. mmiller pushed a change to branch 2.0 in repository https://gitbox.apache.org/repos/asf/accumulo.git. from c2cfa19 Purge mortbay logger from example configs (#1284) add fece0bb Fix misspelling add b9cc144 Merge branch '1.9' into 2.0 No new revisions were added by this update. Summary of changes: .../org/apache/accumulo/core/clientImpl/TableOperationsImpl.java| 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[accumulo] 02/02: Merge branch '2.0'
This is an automated email from the ASF dual-hosted git repository. mmiller pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo.git commit 2ca12d9b2bcfe3c199ce355648bf7cd2a6ccf55d Merge: 52d5d8e b9cc144 Author: Mike Miller AuthorDate: Wed Jul 24 18:34:56 2019 -0400 Merge branch '2.0' .../org/apache/accumulo/core/clientImpl/TableOperationsImpl.java| 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[accumulo] 01/02: Merge branch '1.9' into 2.0
This is an automated email from the ASF dual-hosted git repository. mmiller pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/accumulo.git commit b9cc1446aeffe558e93685a7f60d66464845be1e Merge: c2cfa19 fece0bb Author: Mike Miller AuthorDate: Wed Jul 24 18:34:36 2019 -0400 Merge branch '1.9' into 2.0 .../org/apache/accumulo/core/clientImpl/TableOperationsImpl.java| 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --cc core/src/main/java/org/apache/accumulo/core/clientImpl/TableOperationsImpl.java index 61bfb09,000..f9e5af2 mode 100644,00..100644 --- a/core/src/main/java/org/apache/accumulo/core/clientImpl/TableOperationsImpl.java +++ b/core/src/main/java/org/apache/accumulo/core/clientImpl/TableOperationsImpl.java @@@ -1,1924 -1,0 +1,1924 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.accumulo.core.clientImpl; + +import static com.google.common.base.Preconditions.checkArgument; +import static java.nio.charset.StandardCharsets.UTF_8; +import static java.util.Objects.requireNonNull; +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static java.util.concurrent.TimeUnit.SECONDS; +import static java.util.stream.Collectors.toSet; +import static org.apache.accumulo.fate.util.UtilWaitThread.sleepUninterruptibly; + +import java.io.BufferedReader; +import java.io.FileNotFoundException; +import java.io.IOException; +import java.io.InputStreamReader; +import java.nio.ByteBuffer; +import java.security.SecureRandom; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.EnumSet; +import java.util.HashMap; +import java.util.HashSet; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.Map.Entry; +import java.util.Objects; +import java.util.Random; +import java.util.Set; +import java.util.SortedSet; +import java.util.TreeMap; +import java.util.TreeSet; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicReference; +import java.util.function.Predicate; +import java.util.regex.Pattern; +import java.util.stream.Collectors; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; + +import org.apache.accumulo.core.Constants; +import org.apache.accumulo.core.client.AccumuloException; +import org.apache.accumulo.core.client.AccumuloSecurityException; +import org.apache.accumulo.core.client.IteratorSetting; +import org.apache.accumulo.core.client.NamespaceExistsException; +import org.apache.accumulo.core.client.NamespaceNotFoundException; +import org.apache.accumulo.core.client.Scanner; +import org.apache.accumulo.core.client.TableDeletedException; +import org.apache.accumulo.core.client.TableExistsException; +import org.apache.accumulo.core.client.TableNotFoundException; +import org.apache.accumulo.core.client.TableOfflineException; +import org.apache.accumulo.core.client.admin.CompactionConfig; +import org.apache.accumulo.core.client.admin.DiskUsage; +import org.apache.accumulo.core.client.admin.FindMax; +import org.apache.accumulo.core.client.admin.Locations; +import org.apache.accumulo.core.client.admin.NewTableConfiguration; +import org.apache.accumulo.core.client.admin.SummaryRetriever; +import org.apache.accumulo.core.client.admin.TableOperations; +import org.apache.accumulo.core.client.sample.SamplerConfiguration; +import org.apache.accumulo.core.client.summary.SummarizerConfiguration; +import org.apache.accumulo.core.client.summary.Summary; +import org.apache.accumulo.core.clientImpl.TabletLocator.TabletLocation; +import org.apache.accumulo.core.clientImpl.bulk.BulkImport; +import org.apache.accumulo.core.clientImpl.thrift.ClientService.Client; +import org.apache.accumulo.core.clientImpl.thrift.TDiskUsage; +import org.apache.accumulo.core.clientImpl.thrift.ThriftNotActiveServiceException; +import org.apache.accumulo.core.clientImpl.thrift.ThriftSecurityException; +import
[accumulo] branch 1.9 updated: Fix misspelling
This is an automated email from the ASF dual-hosted git repository.
mmiller pushed a commit to branch 1.9
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/1.9 by this push:
new fece0bb Fix misspelling
fece0bb is described below
commit fece0bbf44dcbd986854353927ea50cffaf8cb00
Author: Mike Miller
AuthorDate: Wed Jul 24 18:30:52 2019 -0400
Fix misspelling
---
.../org/apache/accumulo/core/client/impl/TableOperationsImpl.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git
a/core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
b/core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
index 27ce0c8..4a7d22d 100644
---
a/core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
+++
b/core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
@@ -1698,13 +1698,13 @@ public class TableOperationsImpl extends
TableOperationsHelper {
return sci.toSamplerConfiguration();
}
- private static class LoctionsImpl implements Locations {
+ private static class LocationsImpl implements Locations {
private Map> groupedByRanges;
private Map> groupedByTablets;
private Map tabletLocations;
-public LoctionsImpl(Map>> binnedRanges) {
+public LocationsImpl(Map>> binnedRanges) {
groupedByTablets = new HashMap<>();
groupedByRanges = null;
tabletLocations = new HashMap<>();
@@ -1808,6 +1808,6 @@ public class TableOperationsImpl extends
TableOperationsHelper {
locator.invalidateCache();
}
-return new LoctionsImpl(binnedRanges);
+return new LocationsImpl(binnedRanges);
}
}
[accumulo-testing] branch master updated: Replace Thrift types with proper API Exception (#99)
This is an automated email from the ASF dual-hosted git repository.
mmiller pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git
The following commit(s) were added to refs/heads/master by this push:
new a9b6b24 Replace Thrift types with proper API Exception (#99)
a9b6b24 is described below
commit a9b6b24fbf8cb05118aec706d8caf9ed1896eb70
Author: Mike Miller
AuthorDate: Wed Jul 24 13:00:17 2019 -0400
Replace Thrift types with proper API Exception (#99)
---
contrib/import-control.xml | 2 --
.../testing/randomwalk/concurrent/Config.java | 28 --
2 files changed, 10 insertions(+), 20 deletions(-)
diff --git a/contrib/import-control.xml b/contrib/import-control.xml
index 60eae35..f94af1b 100644
--- a/contrib/import-control.xml
+++ b/contrib/import-control.xml
@@ -37,8 +37,6 @@
-
-
diff --git
a/src/main/java/org/apache/accumulo/testing/randomwalk/concurrent/Config.java
b/src/main/java/org/apache/accumulo/testing/randomwalk/concurrent/Config.java
index 3353403..b209d33 100644
---
a/src/main/java/org/apache/accumulo/testing/randomwalk/concurrent/Config.java
+++
b/src/main/java/org/apache/accumulo/testing/randomwalk/concurrent/Config.java
@@ -20,8 +20,8 @@ import java.util.Properties;
import java.util.SortedSet;
import org.apache.accumulo.core.client.AccumuloException;
-import org.apache.accumulo.core.clientImpl.thrift.TableOperationExceptionType;
-import
org.apache.accumulo.core.clientImpl.thrift.ThriftTableOperationException;
+import org.apache.accumulo.core.client.NamespaceNotFoundException;
+import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.conf.Property;
import org.apache.accumulo.testing.randomwalk.RandWalkEnv;
import org.apache.accumulo.testing.randomwalk.State;
@@ -131,10 +131,8 @@ public class Config extends Test {
env.getAccumuloClient().tableOperations().setProperty(table,
property.getKey(),
property.getDefaultValue());
} catch (AccumuloException ex) {
- if (ex.getCause() instanceof ThriftTableOperationException) {
-ThriftTableOperationException ttoe =
(ThriftTableOperationException) ex.getCause();
-if (ttoe.type == TableOperationExceptionType.NOTFOUND)
- return;
+ if (ex.getCause() instanceof TableNotFoundException) {
+return;
}
throw ex;
}
@@ -153,10 +151,8 @@ public class Config extends Test {
env.getAccumuloClient().namespaceOperations().setProperty(namespace,
property.getKey(),
property.getDefaultValue());
} catch (AccumuloException ex) {
- if (ex.getCause() instanceof ThriftTableOperationException) {
-ThriftTableOperationException ttoe =
(ThriftTableOperationException) ex.getCause();
-if (ttoe.type == TableOperationExceptionType.NAMESPACE_NOTFOUND)
- return;
+ if (ex.getCause() instanceof NamespaceNotFoundException) {
+return;
}
throw ex;
}
@@ -197,10 +193,8 @@ public class Config extends Test {
env.getAccumuloClient().tableOperations().setProperty(table,
setting.property.getKey(),
"" + newValue);
} catch (AccumuloException ex) {
- if (ex.getCause() instanceof ThriftTableOperationException) {
-ThriftTableOperationException ttoe = (ThriftTableOperationException)
ex.getCause();
-if (ttoe.type == TableOperationExceptionType.NOTFOUND)
- return;
+ if (ex.getCause() instanceof TableNotFoundException) {
+return;
}
throw ex;
}
@@ -228,10 +222,8 @@ public class Config extends Test {
env.getAccumuloClient().namespaceOperations().setProperty(namespace,
setting.property.getKey(), "" + newValue);
} catch (AccumuloException ex) {
- if (ex.getCause() instanceof ThriftTableOperationException) {
-ThriftTableOperationException ttoe = (ThriftTableOperationException)
ex.getCause();
-if (ttoe.type == TableOperationExceptionType.NAMESPACE_NOTFOUND)
- return;
+ if (ex.getCause() instanceof NamespaceNotFoundException) {
+return;
}
throw ex;
}
[accumulo-testing] branch keith-turner-patch-1 created (now c50bd97)
This is an automated email from the ASF dual-hosted git repository. kturner pushed a change to branch keith-turner-patch-1 in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git. at c50bd97 Improve bulk CI docs This branch includes the following new commits: new c50bd97 Improve bulk CI docs The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[accumulo-testing] 01/01: Improve bulk CI docs
This is an automated email from the ASF dual-hosted git repository. kturner pushed a commit to branch keith-turner-patch-1 in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git commit c50bd97f90f54bf61ed3f42b4bd32d363126e6dc Author: Keith Turner AuthorDate: Wed Jul 24 11:19:38 2019 -0400 Improve bulk CI docs --- docs/bulk-test.md | 9 + 1 file changed, 9 insertions(+) diff --git a/docs/bulk-test.md b/docs/bulk-test.md index 17076e7..071cce8 100644 --- a/docs/bulk-test.md +++ b/docs/bulk-test.md @@ -30,6 +30,9 @@ for i in $(seq 1 10); do ./bin/cingest bulk /tmp/bt/$i done +# Optionally, copy data before importing. This can be useful in debugging problems. +hadoop distcp hdfs://$NAMENODE/tmp/bt hdfs://$NAMENODE/tmp/bt-copy + for i in $(seq 1 10); do ( echo table ci @@ -53,6 +56,12 @@ scan -t accumulo.metadata -b ~blip -e ~blip~ scan -t accumulo.metadata -c loaded ``` +Additionally check that no rfiles exists in the source dir. + +```bash +hadoop fs -ls -R /tmp/bt | grep rf +``` + The referenced counts output by `cingest verify` should equal : ```
