[accumulo-website] branch asf-staging updated: Automatic Site Publish by Buildbot
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/asf-staging by this push:
new 7da33815 Automatic Site Publish by Buildbot
7da33815 is described below
commit 7da33815fa36a7a1a71798806548c5e6c019af36
Author: buildbot
AuthorDate: Wed Jun 15 15:01:35 2022 +
Automatic Site Publish by Buildbot
---
output/docs/2.x/administration/caching.html| 3 +++
output/docs/2.x/administration/scan-executors.html | 12
output/docs/2.x/getting-started/features.html | 6 ++
output/feed.xml| 4 ++--
output/search_data.json| 6 +++---
5 files changed, 26 insertions(+), 5 deletions(-)
diff --git a/output/docs/2.x/administration/caching.html
b/output/docs/2.x/administration/caching.html
index 77a4d344..ee3053b1 100644
--- a/output/docs/2.x/administration/caching.html
+++ b/output/docs/2.x/administration/caching.html
@@ -442,6 +442,9 @@ for tables where read performance is critical.
Configuration
+The tserver.cache.manager.class
property controls which block cache implementation is used within the tablet
server. Users
+can supply their own implementation and set custom configuration properties to
control it’s behavior (see
org.apache.accumulo.core.spi.cache.BlockCacheManager$Configuration.java).
+
The index and data block caches are configured for tables by the following
properties:
diff --git a/output/docs/2.x/administration/scan-executors.html
b/output/docs/2.x/administration/scan-executors.html
index 7922615d..35f36a19 100644
--- a/output/docs/2.x/administration/scan-executors.html
+++ b/output/docs/2.x/administration/scan-executors.html
@@ -573,6 +573,18 @@ priority of 1.
scan -t tex --execution-hints scan_type=alpha
+Execution Hints can also be used to influence how the block caches are used
for
+a scan. The following configuration would modify the gamma executor to use blocks
+in the cache if they are already cached, but would never load mising blocks
into the
+cache.
+
+config -t tex -s
table.scan.dispatcher.opts.cacheUsage.gamma=opportunistic
+
+
+Other valid values are disabled which does not use data in the block caches,
+enabled which uses
the block cache as it normally would and table which enables
+the block cache for the scan if it’s enabled on the table.
+
diff --git a/output/docs/2.x/getting-started/features.html
b/output/docs/2.x/getting-started/features.html
index d7126aa1..dad6f37a 100644
--- a/output/docs/2.x/getting-started/features.html
+++ b/output/docs/2.x/getting-started/features.html
@@ -726,6 +726,12 @@ it is very unlikely that more data will be written to it,
and thus paying the pe
to re-write a large file can be avoided. Implementations of this compaction
strategy
can be used to optimize the data that compactions will write.
+Pluggable Block Caches
+
+Accumulo provides two BlockCacheManager implementations
(LruBlockCacheManager and
+TinyLfuBlockCacheManager) that construct on-heap block caches. Users can
provide
+alternate BlockCacheManager implementations using the property tserver.cache.manager.class.
+
General Administration
Monitor page
diff --git a/output/feed.xml b/output/feed.xml
index 8652c881..31cf3087 100644
--- a/output/feed.xml
+++ b/output/feed.xml
@@ -6,8 +6,8 @@
https://accumulo.apache.org/
https://accumulo.apache.org/feed.xml; rel="self"
type="application/rss+xml"/>
-Sat, 28 May 2022 15:05:48 +
-Sat, 28 May 2022 15:05:48 +
+Wed, 15 Jun 2022 15:01:27 +
+Wed, 15 Jun 2022 15:01:27 +
Jekyll v4.2.0
diff --git a/output/search_data.json b/output/search_data.json
index 64e16545..43c1dc9d 100644
--- a/output/search_data.json
+++ b/output/search_data.json
@@ -2,7 +2,7 @@
"docs-2-x-administration-caching": {
"title": "Caching",
- "content" : "Accumulo tablet servers have block caches that
buffer data in memory to limit reads from disk.This caching has the following
benefits: reduces latency when reading data helps alleviate hotspots in
tablesEach tablet server has an index and data block cache that is shared by
all hosted tablets (see the tablet server diagramto learn more). A typical
Accumulo read operation will perform a binary search over several index blocks
followed by a linear scanof one or more da [...]
+ "content" : "Accumulo tablet servers have block caches that
buffer data in memory to limit reads from disk.This caching has the following
benefits: reduces latency when reading data helps alleviate hotspots in
tablesEach tablet server has an index and data block cache that is shared by
all hosted tablets (see the tablet server diagramto learn more). A typical
Accumulo read operation will perform a binary search over
[accumulo-website] branch main updated: Add sections for pluggable block caches and scan execution hints that control caching (#323)
This is an automated email from the ASF dual-hosted git repository.
dlmarion pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/main by this push:
new 2c783d99 Add sections for pluggable block caches and scan execution
hints that control caching (#323)
2c783d99 is described below
commit 2c783d99606cd5780af2659372e904a3c1d65062
Author: Dave Marion
AuthorDate: Wed Jun 15 11:01:13 2022 -0400
Add sections for pluggable block caches and scan execution hints that
control caching (#323)
Closes #306
---
_docs-2/administration/caching.md| 3 +++
_docs-2/administration/scan-executors.md | 13 +
_docs-2/getting-started/features.md | 6 ++
3 files changed, 22 insertions(+)
diff --git a/_docs-2/administration/caching.md
b/_docs-2/administration/caching.md
index 91b673bc..1438cbda 100644
--- a/_docs-2/administration/caching.md
+++ b/_docs-2/administration/caching.md
@@ -18,6 +18,9 @@ for tables where read performance is critical.
## Configuration
+The {% plink tserver.cache.manager.class %} property controls which block
cache implementation is used within the tablet server. Users
+can supply their own implementation and set custom configuration properties to
control it's behavior (see
org.apache.accumulo.core.spi.cache.BlockCacheManager$Configuration.java).
+
The index and data block caches are configured for tables by the following
properties:
* {% plink table.cache.block.enable %} - enables data block cache on the table
(default is `false`)
diff --git a/_docs-2/administration/scan-executors.md
b/_docs-2/administration/scan-executors.md
index 6b22714d..0a73edd7 100644
--- a/_docs-2/administration/scan-executors.md
+++ b/_docs-2/administration/scan-executors.md
@@ -159,6 +159,19 @@ priority of 1.
scan -t tex --execution-hints scan_type=alpha
```
+Execution Hints can also be used to influence how the block caches are used for
+a scan. The following configuration would modify the `gamma` executor to use
blocks
+in the cache if they are already cached, but would never load mising blocks
into the
+cache.
+
+```
+config -t tex -s table.scan.dispatcher.opts.cacheUsage.gamma=opportunistic
+```
+
+Other valid values are `disabled` which does not use data in the block caches,
+`enabled` which uses the block cache as it normally would and `table` which
enables
+the block cache for the scan if it's enabled on the table.
+
[tserver]: {{ page.docs_baseurl }}/getting-started/design#tablet-server-1
[setExecutionHints]: {% jurl
org.apache.accumulo.core.client.ScannerBase#setExecutionHints-java.util.Map- %}
diff --git a/_docs-2/getting-started/features.md
b/_docs-2/getting-started/features.md
index c7257c97..ab83b1ba 100644
--- a/_docs-2/getting-started/features.md
+++ b/_docs-2/getting-started/features.md
@@ -299,6 +299,12 @@ it is very unlikely that more data will be written to it,
and thus paying the pe
to re-write a large file can be avoided. Implementations of this compaction
strategy
can be used to optimize the data that compactions will write.
+### Pluggable Block Caches
+
+Accumulo provides two BlockCacheManager implementations (LruBlockCacheManager
and
+TinyLfuBlockCacheManager) that construct on-heap block caches. Users can
provide
+alternate BlockCacheManager implementations using the property
`tserver.cache.manager.class`.
+
## General Administration
### Monitor page
[accumulo] branch main updated: Make auths classes in Authorizations (#2777)
This is an automated email from the ASF dual-hosted git repository. mmiller pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/accumulo.git The following commit(s) were added to refs/heads/main by this push: new 418eee8fa0 Make auths classes in Authorizations (#2777) 418eee8fa0 is described below commit 418eee8fa0abed50f30cb6c97184964f924d2c67 Author: Mike Miller AuthorDate: Wed Jun 15 13:37:59 2022 + Make auths classes in Authorizations (#2777) * Change private member types from interfaces to classes to get rid of Java 18 warnings * For #2777 --- .../main/java/org/apache/accumulo/core/security/Authorizations.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/core/src/main/java/org/apache/accumulo/core/security/Authorizations.java b/core/src/main/java/org/apache/accumulo/core/security/Authorizations.java index 1df129b9a8..37e42eb1aa 100644 --- a/core/src/main/java/org/apache/accumulo/core/security/Authorizations.java +++ b/core/src/main/java/org/apache/accumulo/core/security/Authorizations.java @@ -44,8 +44,8 @@ public class Authorizations implements Iterable, Serializable, Authoriza private static final long serialVersionUID = 1L; - private Set auths = new HashSet<>(); - private List authsList = new ArrayList<>(); // sorted order + private final HashSet auths = new HashSet<>(); + private final ArrayList authsList = new ArrayList<>(); // sorted order /** * An empty set of authorizations.
[accumulo] branch main updated: Add delete event watcher to ServerConfigurationFactory (#2773)
This is an automated email from the ASF dual-hosted git repository.
edcoleman pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/main by this push:
new 0bc41827a5 Add delete event watcher to ServerConfigurationFactory
(#2773)
0bc41827a5 is described below
commit 0bc41827a59fcff585fa8b349f2dde2746749c5c
Author: EdColeman
AuthorDate: Wed Jun 15 12:48:30 2022 +
Add delete event watcher to ServerConfigurationFactory (#2773)
This partially satisfies PR #2769.
---
.../server/conf/ServerConfigurationFactory.java| 43 +
.../accumulo/test/conf/PropStoreConfigIT.java | 56 ++
2 files changed, 99 insertions(+)
diff --git
a/server/base/src/main/java/org/apache/accumulo/server/conf/ServerConfigurationFactory.java
b/server/base/src/main/java/org/apache/accumulo/server/conf/ServerConfigurationFactory.java
index 6341b45e72..3e4c935878 100644
---
a/server/base/src/main/java/org/apache/accumulo/server/conf/ServerConfigurationFactory.java
+++
b/server/base/src/main/java/org/apache/accumulo/server/conf/ServerConfigurationFactory.java
@@ -30,7 +30,13 @@ import org.apache.accumulo.core.conf.SiteConfiguration;
import org.apache.accumulo.core.data.NamespaceId;
import org.apache.accumulo.core.data.TableId;
import org.apache.accumulo.server.ServerContext;
+import org.apache.accumulo.server.conf.store.NamespacePropKey;
+import org.apache.accumulo.server.conf.store.PropChangeListener;
+import org.apache.accumulo.server.conf.store.PropStoreKey;
import org.apache.accumulo.server.conf.store.SystemPropKey;
+import org.apache.accumulo.server.conf.store.TablePropKey;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import com.google.common.base.Suppliers;
@@ -38,6 +44,7 @@ import com.google.common.base.Suppliers;
* A factor for configurations used by a server process. Instance of this
class are thread-safe.
*/
public class ServerConfigurationFactory extends ServerConfiguration {
+ private final static Logger log =
LoggerFactory.getLogger(ServerConfigurationFactory.class);
private final Map tableParentConfigs = new
ConcurrentHashMap<>();
private final Map tableConfigs = new
ConcurrentHashMap<>();
@@ -48,6 +55,8 @@ public class ServerConfigurationFactory extends
ServerConfiguration {
private final SiteConfiguration siteConfig;
private final Supplier systemConfig;
+ private final DeleteWatcher deleteWatcher = new DeleteWatcher();
+
public ServerConfigurationFactory(ServerContext context, SiteConfiguration
siteConfig) {
this.context = context;
this.siteConfig = siteConfig;
@@ -76,6 +85,7 @@ public class ServerConfigurationFactory extends
ServerConfiguration {
public TableConfiguration getTableConfiguration(TableId tableId) {
return tableConfigs.computeIfAbsent(tableId, key -> {
if (context.tableNodeExists(tableId)) {
+context.getPropStore().registerAsListener(TablePropKey.of(context,
tableId), deleteWatcher);
var conf =
new TableConfiguration(context, tableId,
getNamespaceConfigurationForTable(tableId));
ConfigCheckUtil.validate(conf);
@@ -99,10 +109,43 @@ public class ServerConfigurationFactory extends
ServerConfiguration {
@Override
public NamespaceConfiguration getNamespaceConfiguration(NamespaceId
namespaceId) {
return namespaceConfigs.computeIfAbsent(namespaceId, key -> {
+ context.getPropStore().registerAsListener(NamespacePropKey.of(context,
namespaceId),
+ deleteWatcher);
var conf = new NamespaceConfiguration(context, namespaceId,
getSystemConfiguration());
ConfigCheckUtil.validate(conf);
return conf;
});
}
+ private class DeleteWatcher implements PropChangeListener {
+
+@Override
+public void zkChangeEvent(PropStoreKey propStoreKey) {
+ // no-op. changes handled by prop store impl
+}
+
+@Override
+public void cacheChangeEvent(PropStoreKey propStoreKey) {
+ // no-op. changes handled by prop store impl
+}
+
+@Override
+public void deleteEvent(PropStoreKey propStoreKey) {
+ if (propStoreKey instanceof NamespacePropKey) {
+log.trace("configuration snapshot refresh: Handle namespace delete for
{}", propStoreKey);
+namespaceConfigs.remove(((NamespacePropKey) propStoreKey).getId());
+return;
+ }
+ if (propStoreKey instanceof TablePropKey) {
+log.trace("configuration snapshot refresh: Handle table delete for
{}", propStoreKey);
+tableConfigs.remove(((TablePropKey) propStoreKey).getId());
+tableParentConfigs.remove(((TablePropKey) propStoreKey).getId());
+ }
+}
+
+@Override
+public void connectionEvent() {
+ // no-op. changes handled by prop store impl
+}
+ }
}
diff --git
a/test/src/main/java/org/apache/accumulo/test/conf/PropStoreConfigIT.java
