[GitHub] EdColeman commented on issue #394: ACCUMULO-4836 make online table always wait
EdColeman commented on issue #394: ACCUMULO-4836 make online table always wait URL: https://github.com/apache/accumulo/pull/394#issuecomment-369808863 Does this revert the behavior that ACCUMULO-4574 was trying to address - if the table is online, return and do not wait for additional actions? The original behavior would block if there was a fate operation on the table and online was called - the original fix was to not block and return because the table was online and no additional actions were required to be in online. If the table is online - why do we need to wait? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (ACCUMULO-4836) Tables do not always wait for online or offline
[
https://issues.apache.org/jira/browse/ACCUMULO-4836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383086#comment-16383086
]
Ed Coleman commented on ACCUMULO-4836:
--
I am unclear on the proposed fix - actually I'm unclear on the problem - not
saying this is incorrect, I just don't understand the logic being proposed.
If the table is online - what is there to wait for? Should it not be that if
the expected state is not online, then check wait...
if(expectedState == TableStae.ONLINE){
return;
}
if(wait){
waitForTableStateTransisition(tableId, TableState.ONLINE);
return;
}
> Tables do not always wait for online or offline
> ---
>
> Key: ACCUMULO-4836
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4836
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.7.3
>Reporter: Keith Turner
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.7.4, 1.9.0, 2.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While investigating why TabletStateChangeIteratorIT it was discovered that
> online table with wait=true does not always wait. The test relied on this
> API to wait and that is why it was failing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ACCUMULO-4838) Create SPI package
Keith Turner created ACCUMULO-4838:
--
Summary: Create SPI package
Key: ACCUMULO-4838
URL: https://issues.apache.org/jira/browse/ACCUMULO-4838
Project: Accumulo
Issue Type: Improvement
Reporter: Keith Turner
Accumulo has multiple pluggable services. It would be nice if the SPIs
(service provider interface) for these were in one package. For existing
service SPIs, this can not be easily done. However, for new service SPIs it
would be nice to start putting them under a single package. This package could
be {{org.apache.accumulo.core.spi}}. Currently there are at least two new
unreleased SPIs for caching and summarization. These could be moved to the new
package.
For existing SPIs could possibly do the following :
* Create a new SPI in the new package
* Make existing SPI extend new SPI and deprecate it
The contents of this package could be analyzed by APILyzer to ensure only API
and SPI types are used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ACCUMULO-4837) Allow short service names in addition to class names.
Keith Turner created ACCUMULO-4837:
--
Summary: Allow short service names in addition to class names.
Key: ACCUMULO-4837
URL: https://issues.apache.org/jira/browse/ACCUMULO-4837
Project: Accumulo
Issue Type: Improvement
Reporter: Keith Turner
In 2.0.0-SNAPSHOT the cache implementation was made configurable. Currently to
configure it, you set a property like the following.
{noformat}
tserver.cache.manager.class=org.apache.accumulo.core.file.blockfile.cache.tinylfu.TinyLfuBlockCacheManager
{noformat}
I would much rather be able to provide a short service name like the following
when configuring the cache. However I do not want the list to be predefined, I
will want the user to be able to provide implementations.
{noformat}
tserver.cache.implementation=tinylfu
{noformat}
What is a good way to do this? Is there a good reason not do this and just
stick with class names only? I was also thinking it may be nice to have a
shell command for listing services, but this could be done independently.
One way I thought of doing this is having an interface like the the following
that services (balancer, compaction strategy, cache, etc) could implement.
{code:java}
public interface AccumuloService {
/**
* A human readable, short, unique identification that can be specified in
configuration to identify a service implementation.
*/
public String getName();
public static C load(String configId, Class
serviceType, ClassLoader classLoader) {
ServiceLoader services = ServiceLoader.load(serviceType, classLoader);
for (C service : services) {
if(service.getName().equals(configId) ||
service.getClass().getName().equals(configId)) {
return service;
}
}
return null;
}
}
{code}
Then the cache implementation could provide a name
{code:java}
//assume BlockCacheManager implements AccumuloService
public class TinyLfuBlockCacheManager extends BlockCacheManager {
private static final Logger LOG =
LoggerFactory.getLogger(TinyLfuBlockCacheManager.class);
@Override
protected TinyLfuBlockCache createCache(Configuration conf, CacheType type) {
LOG.info("Creating {} cache with configuration {}", type, conf);
return new TinyLfuBlockCache(conf, type);
}
@Override
public String getName() {
return "tinylfu";
}
}
{code}
The code to load a block cache impl would look like the following :
{code:java}
String impl = conf.get(Property.TSERV_CACHE_MANAGER_IMPL);
BlockCacheManager bcm = AccumuloService.load(impl, BlockCacheManager.class,
AccumuloVFSClassLoader.getClassLoader());
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Accumulo-1.8 - Build # 278 - Fixed
The Apache Jenkins build system has built Accumulo-1.8 (build #278) Status: Fixed Check console output at https://builds.apache.org/job/Accumulo-1.8/278/ to view the results.
[GitHub] mikewalch closed pull request #58: Created 1.7.4 release notes
mikewalch closed pull request #58: Created 1.7.4 release notes URL: https://github.com/apache/accumulo-website/pull/58 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/_posts/release/2018-02-21-accumulo-1.7.4.md b/_posts/release/2018-02-21-accumulo-1.7.4.md new file mode 100644 index ..5729cd50 --- /dev/null +++ b/_posts/release/2018-02-21-accumulo-1.7.4.md @@ -0,0 +1,163 @@ +--- +title: Apache Accumulo 1.7.4 +draft: true +--- + +Apache Accumulo 1.7.4 is a maintenance release on the 1.7 version branch. This release contains changes from 46 issues, comprised of bug-fixes, +performance improvements, build quality improvements, and more. See [Jira][JIRA_174] or the bottom of this page for a complete list. + +Below are resources for this release: + +* [User Manual][user_manual] - In-depth developer and administrator documentation. +* [Javadocs][javadoc] - Accumulo 1.7.4 API +* [Examples][examples] - Code with corresponding readme files that give step by step instructions for running example code. + +Accumulo follows [Semantic Versioning][semver] [guidelines][guidelines]. This release is a ?[patch version][patch_version]?, which means that +only backwards compatible bug fixes are introduced in this version. A bug fix is defined as an internal change that fixes incorrect behavior. Users +of any previous 1.7.x release are strongly encouraged to update as soon as possible to benefit from the bug fixes with very little concern in change +of underlying functionality. As always, the Accumulo developers take API compatibility very seriously and have invested much time to ensure that we +meet the promises set forth to our users. Users of 1.6 or earlier that are seeking to upgrade to 1.7 should consider 1.7.4 as a starting point. + +## Major Changes + +### Fixed upgrade process to set version on all volumes + +During upgrades, only one volume in a multiple HDFS volume was updated with the correct version. This would cause all tablet servers +to complain and ultimately fail. [ACCUMULO-4686] fixes this by setting the version on all volumes. + +### Updated Accumulo to work with new releases of Guava + +In [ACCUMULO-4702], dependencies on Beta-annotated Guava classes and methods were removed. While Accumulo still +includes Guava 14 in its tarball, it will work with newer versions of Guava in client code. It has been tested to work +with Guava 23. + +### Updated RFile to prevent very large blocks + +RFiles now use windowed statistics ([ACCUMULO-4669]) to prevent very large blocks when key statistics are not uniform + +## Notable Changes + +* [ACCUMULO-4506] - Add a timeout to a replication RPC call +* [ACCUMULO-4619] - Add splits hung forever +* [ACCUMULO-4633] - Added check to prevent division by zero +* [ACCUMULO-4640] - Accumulo shell is expecting instance.volumens in client.conf instead of accumulo-site.xml +* [ACCUMULO-4657] - BulkImport Performance Bottleneck +* [ACCUMULO-4665] - Must use the "real" user for RPCs when Kerberos is enabled +* [ACCUMULO-4676] - Use HTTPOnly flags in monitor to prevent XSS attacks +* [ACCUMULO-4776] - Fix advertised host in monitor +* [ACCUMULO-4777] - Root tablet got spammed with 1.8 million log entries +* [ACCUMULO-4787] - Close input stream in AccumuloReplicaSystem +* [ACCUMULO-4809] - Avoid blocking during session clean up +* [ACCUMULO-4817] - Update build plugins and parent POM + +## Upgrading + +View the [Upgrading Accumulo documentation][upgrade] for guidance. + +## Testing + +## All Changes + +* [ACCUMULO-1972] - Fixed Range constructor +* [ACCUMULO-3208] - Integration test for the OrIterator and cleanup +* [ACCUMULO-3283] - Create ColumnFQ only once +* [ACCUMULO-3827] - Set default store types for monitor ssl to jks +* [ACCUMULO-4170] - Clarify ClientConfiguration javadocs +* [ACCUMULO-4365] - Fixes to prevent intermittent failures in ShellServerIT and ConditionalWriterIT +* [ACCUMULO-4482] - Mention snappy compression in docs +* [ACCUMULO-4506] - Add a timeout to a replication RPC call +* [ACCUMULO-4546] - Create default log message for table error +* [ACCUMULO-4555] - Removes parsing of version string in Version class +* [ACCUMULO-4576] - Suppress warnings from deprecations +* [ACCUMULO-4587] - Upgrade Monitor jquery to 3.2.1 +* [ACCUMULO-4591] - Add replication latency metrics +* [ACCUMULO-4602] - Deleted AssignmentThreadsIT +* [ACCUMULO-4619] - Add splits hung forever +* [ACCUMULO-4627] - Add corrupt WAL recovery instructions to user manual +* [ACCUMULO-4633] - Added check to prevent division by zero +* [ACCUMULO-4636] - System iterator improvements +* [ACCUMULO-4640] - Accumulo shell is expecting instance.volumens in client.conf instead of accumulo-site.xml +* [ACCUMULO-4648] - Update voting text in build.sh
[GitHub] milleruntime closed pull request #393: Add TableOfflineException to TableOps.addSplits
milleruntime closed pull request #393: Add TableOfflineException to
TableOps.addSplits
URL: https://github.com/apache/accumulo/pull/393
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
b/core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
index 0c3e0e6c60..3015b1dd73 100644
---
a/core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
+++
b/core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
@@ -394,6 +394,9 @@ public void addSplits(String tableName, SortedSet
partitionKeys) throws Ta
if (excep instanceof TableNotFoundException) {
TableNotFoundException tnfe = (TableNotFoundException) excep;
throw new TableNotFoundException(tableId, tableName, "Table not
found by background thread", tnfe);
+ } else if (excep instanceof TableOfflineException) {
+log.debug("TableOfflineException occurred in background thread.
Throwing new exception", excep);
+throw new TableOfflineException(context.getInstance(), tableId);
} else if (excep instanceof AccumuloSecurityException) {
// base == background accumulo security exception
AccumuloSecurityException base = (AccumuloSecurityException) excep;
diff --git
a/test/src/main/java/org/apache/accumulo/test/randomwalk/concurrent/AddSplits.java
b/test/src/main/java/org/apache/accumulo/test/randomwalk/concurrent/AddSplits.java
index e02629918f..dc040a6eb9 100644
---
a/test/src/main/java/org/apache/accumulo/test/randomwalk/concurrent/AddSplits.java
+++
b/test/src/main/java/org/apache/accumulo/test/randomwalk/concurrent/AddSplits.java
@@ -22,7 +22,6 @@
import java.util.Random;
import java.util.TreeSet;
-import org.apache.accumulo.core.client.AccumuloException;
import org.apache.accumulo.core.client.Connector;
import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.client.TableOfflineException;
@@ -58,12 +57,6 @@ public void visit(State state, Environment env, Properties
props) throws Excepti
log.debug("AddSplits " + tableName + " failed, doesnt exist");
} catch (TableOfflineException e) {
log.debug("AddSplits " + tableName + " failed, offline");
-} catch (AccumuloException ae) {
- Throwable cause = ae.getCause();
- if (cause != null && cause instanceof TableOfflineException)
-log.debug("AddSplits " + tableName + " failed, offline");
- else
-throw ae;
}
}
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] keith-turner commented on issue #58: Created 1.7.4 release notes
keith-turner commented on issue #58: Created 1.7.4 release notes URL: https://github.com/apache/accumulo-website/pull/58#issuecomment-369628990 > Can I merge this? It's marked as draft so it won't show up on the site. Since its marked draft, I think its good to merge it because it makes it easier for others to edit. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (ACCUMULO-4836) Tables do not always wait for online or offline
[ https://issues.apache.org/jira/browse/ACCUMULO-4836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ACCUMULO-4836: - Labels: pull-request-available (was: ) > Tables do not always wait for online or offline > --- > > Key: ACCUMULO-4836 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4836 > Project: Accumulo > Issue Type: Bug >Affects Versions: 1.7.3 >Reporter: Keith Turner >Priority: Major > Labels: pull-request-available > Fix For: 1.7.4, 1.9.0, 2.0.0 > > > While investigating why TabletStateChangeIteratorIT it was discovered that > online table with wait=true does not always wait. The test relied on this > API to wait and that is why it was failing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] keith-turner opened a new pull request #394: ACCUMULO-4836 make online table always wait
keith-turner opened a new pull request #394: ACCUMULO-4836 make online table always wait URL: https://github.com/apache/accumulo/pull/394 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (ACCUMULO-4836) Tables do not always wait for online or offline
Keith Turner created ACCUMULO-4836: -- Summary: Tables do not always wait for online or offline Key: ACCUMULO-4836 URL: https://issues.apache.org/jira/browse/ACCUMULO-4836 Project: Accumulo Issue Type: Bug Affects Versions: 1.7.3 Reporter: Keith Turner Fix For: 1.7.4, 1.9.0, 2.0.0 While investigating why TabletStateChangeIteratorIT it was discovered that online table with wait=true does not always wait. The test relied on this API to wait and that is why it was failing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] mikewalch commented on issue #58: Created 1.7.4 release notes
mikewalch commented on issue #58: Created 1.7.4 release notes URL: https://github.com/apache/accumulo-website/pull/58#issuecomment-369614178 Can I merge this? It's marked as draft so it won't show up on the site. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
