[GitHub] jomach commented on a change in pull request #325: ACCUMULO-2341?
jomach commented on a change in pull request #325: ACCUMULO-2341?
URL: https://github.com/apache/accumulo/pull/325#discussion_r154865217
##
File path:
core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java
##
@@ -44,14 +44,14 @@
public static MasterClientService.Client
getConnectionWithRetry(ClientContext context) {
while (true) {
- MasterClientService.Client result = getConnection(context);
+ MasterClientService.Client result = getConnection(context, false);
Review comment:
ok, I have done it the other way around. Will check that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] jomach commented on a change in pull request #325: ACCUMULO-2341?
jomach commented on a change in pull request #325: ACCUMULO-2341?
URL: https://github.com/apache/accumulo/pull/325#discussion_r154865158
##
File path:
core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java
##
@@ -183,4 +193,13 @@ public static void executeVoid(ClientContext context,
ClientExec exec) throws AccumuloException,
AccumuloSecurityException {
Review comment:
Please check my comment above.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] jomach commented on a change in pull request #325: ACCUMULO-2341?
jomach commented on a change in pull request #325: ACCUMULO-2341?
URL: https://github.com/apache/accumulo/pull/325#discussion_r154865055
##
File path:
core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java
##
@@ -44,14 +44,14 @@
public static MasterClientService.Client
getConnectionWithRetry(ClientContext context) {
while (true) {
- MasterClientService.Client result = getConnection(context);
+ MasterClientService.Client result = getConnection(context, false);
if (result != null)
return result;
sleepUninterruptibly(250, TimeUnit.MILLISECONDS);
}
}
- public static MasterClientService.Client getConnection(ClientContext
context) {
+ public static MasterClientService.Client getConnection(ClientContext
context, boolean isAdminRequest) {
Review comment:
Sorry but we agreed on a discussion that we really don't want to have that.
The isAdminRequest will mean that it should not try forever as it is a
administration task like shutdown. That's why we need to differ here. My
Initial PR has done exactly that we conclude that what timeout should then we
use ? That's why we discard that option
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on issue #317: Update formatter-maven-plugin to Eclipse Oxygen
ctubbsii commented on issue #317: Update formatter-maven-plugin to Eclipse Oxygen URL: https://github.com/apache/accumulo/pull/317#issuecomment-349137609 LOL, oops. I just pushed it. I didn't realize I still had an open PR. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii closed pull request #317: Update formatter-maven-plugin to Eclipse Oxygen
ctubbsii closed pull request #317: Update formatter-maven-plugin to Eclipse
Oxygen
URL: https://github.com/apache/accumulo/pull/317
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/pom.xml b/pom.xml
index deaccf063b..294426eda6 100644
--- a/pom.xml
+++ b/pom.xml
@@ -944,7 +944,7 @@
net.revelc.code.formatter
formatter-maven-plugin
- 2.6.0
+ 2.7.0
${maven.compiler.source}
${maven.compiler.source}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] milleruntime commented on issue #317: Update formatter-maven-plugin to Eclipse Oxygen
milleruntime commented on issue #317: Update formatter-maven-plugin to Eclipse Oxygen URL: https://github.com/apache/accumulo/pull/317#issuecomment-349136409 @ctubbsii I think this can be closed? Looks like this change was already made in 646b059eb0b2de3bffe329d744c47bd9890103c7 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (ACCUMULO-4754) Fix links to properties in 2.0 documentation
[ https://issues.apache.org/jira/browse/ACCUMULO-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-4754: Component/s: website > Fix links to properties in 2.0 documentation > > > Key: ACCUMULO-4754 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4754 > Project: Accumulo > Issue Type: Bug > Components: website >Affects Versions: 2.0.0 >Reporter: Mike Walch >Assignee: Mike Walch > Fix For: 2.0.0 > > > If you click on a link for a configuration property in 2.0. It will go to > property but property will be hidden below navbar. This can be fixed by > adding hidden whitespace. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ACCUMULO-4611) Commons Configuration either needs bumped or needs to be provided for Hadoop 3
[
https://issues.apache.org/jira/browse/ACCUMULO-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs updated ACCUMULO-4611:
Priority: Critical (was: Major)
> Commons Configuration either needs bumped or needs to be provided for Hadoop 3
> --
>
> Key: ACCUMULO-4611
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4611
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.7.2, 1.8.1
> Environment: CentOS 7
> Accumulo 1.7.x, 1.8.x
> Hadoop 3.0.0-alpha 2
> Zookeeper 3.4.9
>Reporter: Michael Hogue
>Priority: Critical
> Fix For: 2.0.0
>
>
> I was investigating running either Accumulo 1.7.x or 1.8.x on Hadoop
> 3.0.0-alpha 2 and ran into a couple of issues. Since Accumulo assumes some
> dependencies will be provided by Hadoop (per ACCUMULO-1244), if those
> dependencies change then Accumulo might also need to change.
> HADOOP-13660 bumped {{commons-configuration}} from 1.6 -> 2.1 for Hadoop
> 3.0.0 and so NoClassDefFoundErrors are thrown per the below stack trace on
> {{accumulo init}}:
> {noformat}
> [mike@localhost cloud]$ accumulo init
> 2017-03-22 15:38:34,106 [start.Main] ERROR: Uncaught exception
> java.util.ServiceConfigurationError:
> org.apache.accumulo.start.spi.KeywordExecutable: Provider
> org.apache.accumulo.proxy.Proxy could not be instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at org.apache.accumulo.start.Main.checkDuplicates(Main.java:223)
> at org.apache.accumulo.start.Main.getExecutables(Main.java:215)
> at org.apache.accumulo.start.Main.main(Main.java:78)
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/commons/configuration/Configuration
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.newInstance(Class.java:412)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.configuration.Configuration
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
> org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:284)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 10 more
> {noformat}
> I worked around this by dropping the {{commons-configuration}} 1.6 jar on
> Accumulo's classpath, but this should either be provided by Accumulo now or
> Accumulo should be bumped to {{commons-configuration2}} as well. Note that
> the latter change would cause problems for Hadoop 2 installations.
> There was actually one more thing that needed added to the accumulo-site.xml
> in order to get Accumulo to run in Hadoop 3:
> {noformat}
> $HADOOP_PREFIX/share/hadoop/client/[^.].*.jar
> {noformat}
> The reason for this is because {{Hdfs.java}} moved from the hadoop-hdfs jar
> to the hadoop-client-api jar.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ACCUMULO-4611) Commons Configuration either needs bumped or needs to be provided for Hadoop 3
[
https://issues.apache.org/jira/browse/ACCUMULO-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277700#comment-16277700
]
Christopher Tubbs commented on ACCUMULO-4611:
-
Another problem was mentioned on ACCUMULO-4753. This will create a breaking API
change, without a deprecation cycle if we want Accumulo 2.0 to work with Hadoop
3.
> Commons Configuration either needs bumped or needs to be provided for Hadoop 3
> --
>
> Key: ACCUMULO-4611
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4611
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.7.2, 1.8.1
> Environment: CentOS 7
> Accumulo 1.7.x, 1.8.x
> Hadoop 3.0.0-alpha 2
> Zookeeper 3.4.9
>Reporter: Michael Hogue
> Fix For: 2.0.0
>
>
> I was investigating running either Accumulo 1.7.x or 1.8.x on Hadoop
> 3.0.0-alpha 2 and ran into a couple of issues. Since Accumulo assumes some
> dependencies will be provided by Hadoop (per ACCUMULO-1244), if those
> dependencies change then Accumulo might also need to change.
> HADOOP-13660 bumped {{commons-configuration}} from 1.6 -> 2.1 for Hadoop
> 3.0.0 and so NoClassDefFoundErrors are thrown per the below stack trace on
> {{accumulo init}}:
> {noformat}
> [mike@localhost cloud]$ accumulo init
> 2017-03-22 15:38:34,106 [start.Main] ERROR: Uncaught exception
> java.util.ServiceConfigurationError:
> org.apache.accumulo.start.spi.KeywordExecutable: Provider
> org.apache.accumulo.proxy.Proxy could not be instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at org.apache.accumulo.start.Main.checkDuplicates(Main.java:223)
> at org.apache.accumulo.start.Main.getExecutables(Main.java:215)
> at org.apache.accumulo.start.Main.main(Main.java:78)
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/commons/configuration/Configuration
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.newInstance(Class.java:412)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.configuration.Configuration
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
> org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:284)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 10 more
> {noformat}
> I worked around this by dropping the {{commons-configuration}} 1.6 jar on
> Accumulo's classpath, but this should either be provided by Accumulo now or
> Accumulo should be bumped to {{commons-configuration2}} as well. Note that
> the latter change would cause problems for Hadoop 2 installations.
> There was actually one more thing that needed added to the accumulo-site.xml
> in order to get Accumulo to run in Hadoop 3:
> {noformat}
> $HADOOP_PREFIX/share/hadoop/client/[^.].*.jar
> {noformat}
> The reason for this is because {{Hdfs.java}} moved from the hadoop-hdfs jar
> to the hadoop-client-api jar.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (ACCUMULO-4753) Update to commons-configuration2
[ https://issues.apache.org/jira/browse/ACCUMULO-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4753. - Resolution: Duplicate Fix Version/s: (was: 2.0.0) Duplicate of ACCUMULO-4611 > Update to commons-configuration2 > > > Key: ACCUMULO-4753 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4753 > Project: Accumulo > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Critical > > I'm looking at what it would take to get Accumulo working against Hadoop 3. > Thankfully, the only issue seems to be around the commons-configuration2 > update that Hadoop has changed to. With our use of commons-configuration (1) > for ClientConfiguration and others, this brings in multiple versions of > commons-beanutils artifacts which result in exceptions at runtime. > I think the quickest solution is to just upgrade ourselves to use > commons-configuration2. The "long-term" solution is to try to switch over to > the shaded jars that Hadoop is publishing (but I'm reticent to sign up for > that right now :P). > I think we need to have a discussion about where we want to land this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ACCUMULO-4753) Update to commons-configuration2
[ https://issues.apache.org/jira/browse/ACCUMULO-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277694#comment-16277694 ] Christopher Tubbs edited comment on ACCUMULO-4753 at 12/4/17 10:50 PM: --- bq. Best as I could tell (investigated briefly last night), this isn't possible. The difference in beanutil versions between commons-config(1) and commons-config2 would preclude us from making one or the other work. :( bq. Another solution would be to shade commons-configuration (and its dependencies) into an Accumulo jar, but shading is generally met with opposition in Accumulo Shading wouldn't fix the problem here. The reason we can't move away from commons-configuration is that it is used in a public API method. It was short-sighted advice I gave a while back when the Sqrrl folks were contributing the SSL RPC features and needed to add client-side configuration. We can fix this in our 2.0, but only by breaking API without a minor release with the deprecation. I don't think this can be helped, but I think it's probably worth it for 2.0 to work with Hadoop 3, and to move to commons-configuration2 entirely for our own configs. was (Author: ctubbsii): bq. Best as I could tell (investigated briefly last night), this isn't possible. The difference in beanutil versions between commons-config(1) and commons-config2 would preclude us from making one or the other work. :( bq. Another solution would be to shade commons-configuration (and its dependencies) into an Accumulo jar, but shading is generally met with opposition in Accumulo Shading wouldn't fix the problem here. The reason we can't move away from commons-configuration is that it is used in a public API method. It was short-sighted advice I gave a while back when the Sqrl folks were contributing the SSL RPC features and needed to add client-side configuration. We can fix this in our 2.0, but only by breaking API without a minor release with the deprecation. I don't think this can be helped, but I think it's probably worth it for 2.0 to work with Hadoop 3, and to move to commons-configuration2 entirely for our own configs. > Update to commons-configuration2 > > > Key: ACCUMULO-4753 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4753 > Project: Accumulo > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Critical > Fix For: 2.0.0 > > > I'm looking at what it would take to get Accumulo working against Hadoop 3. > Thankfully, the only issue seems to be around the commons-configuration2 > update that Hadoop has changed to. With our use of commons-configuration (1) > for ClientConfiguration and others, this brings in multiple versions of > commons-beanutils artifacts which result in exceptions at runtime. > I think the quickest solution is to just upgrade ourselves to use > commons-configuration2. The "long-term" solution is to try to switch over to > the shaded jars that Hadoop is publishing (but I'm reticent to sign up for > that right now :P). > I think we need to have a discussion about where we want to land this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ACCUMULO-4753) Update to commons-configuration2
[ https://issues.apache.org/jira/browse/ACCUMULO-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277694#comment-16277694 ] Christopher Tubbs commented on ACCUMULO-4753: - bq. Best as I could tell (investigated briefly last night), this isn't possible. The difference in beanutil versions between commons-config(1) and commons-config2 would preclude us from making one or the other work. :( bq. Another solution would be to shade commons-configuration (and its dependencies) into an Accumulo jar, but shading is generally met with opposition in Accumulo Shading wouldn't fix the problem here. The reason we can't move away from commons-configuration is that it is used in a public API method. It was short-sighted advice I gave a while back when the Sqrl folks were contributing the SSL RPC features and needed to add client-side configuration. We can fix this in our 2.0, but only by breaking API without a minor release with the deprecation. I don't think this can be helped, but I think it's probably worth it for 2.0 to work with Hadoop 3, and to move to commons-configuration2 entirely for our own configs. > Update to commons-configuration2 > > > Key: ACCUMULO-4753 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4753 > Project: Accumulo > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Critical > Fix For: 2.0.0 > > > I'm looking at what it would take to get Accumulo working against Hadoop 3. > Thankfully, the only issue seems to be around the commons-configuration2 > update that Hadoop has changed to. With our use of commons-configuration (1) > for ClientConfiguration and others, this brings in multiple versions of > commons-beanutils artifacts which result in exceptions at runtime. > I think the quickest solution is to just upgrade ourselves to use > commons-configuration2. The "long-term" solution is to try to switch over to > the shaded jars that Hadoop is publishing (but I'm reticent to sign up for > that right now :P). > I think we need to have a discussion about where we want to land this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ACCUMULO-4753) Update to commons-configuration2
[ https://issues.apache.org/jira/browse/ACCUMULO-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-4753: Fix Version/s: 2.0.0 > Update to commons-configuration2 > > > Key: ACCUMULO-4753 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4753 > Project: Accumulo > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Critical > Fix For: 2.0.0 > > > I'm looking at what it would take to get Accumulo working against Hadoop 3. > Thankfully, the only issue seems to be around the commons-configuration2 > update that Hadoop has changed to. With our use of commons-configuration (1) > for ClientConfiguration and others, this brings in multiple versions of > commons-beanutils artifacts which result in exceptions at runtime. > I think the quickest solution is to just upgrade ourselves to use > commons-configuration2. The "long-term" solution is to try to switch over to > the shaded jars that Hadoop is publishing (but I'm reticent to sign up for > that right now :P). > I think we need to have a discussion about where we want to land this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ctubbsii commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
ctubbsii commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154795002 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration Review comment: @keith-turner Wait, I'm confused now. I thought that's what we were talking about. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on issue #326: ACCUMULO-4745 Fixed broken links in tables table on monitor
ctubbsii commented on issue #326: ACCUMULO-4745 Fixed broken links in tables table on monitor URL: https://github.com/apache/accumulo/pull/326#issuecomment-349125966 Thanks for the patch, @bfach10 ; @milleruntime was right, I'm ok OK with these changes. I created a separate, follow-on issue at https://issues.apache.org/jira/browse/ACCUMULO-4755 in case somebody's interested in the idea @milleruntime mentioned that I had. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (ACCUMULO-4755) Register a custom serializer for AbstractId types (Table.ID and Namespace.ID)
[ https://issues.apache.org/jira/browse/ACCUMULO-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277640#comment-16277640 ] Christopher Tubbs commented on ACCUMULO-4755: - [~bmfach], if you're interested, feel free to take a look. Your patch on the other issue was perfectly fine, but this might be better. I created a separate issue, so we could merge that one which got the job done, and only look at this if somebody has extra time and interest. > Register a custom serializer for AbstractId types (Table.ID and Namespace.ID) > - > > Key: ACCUMULO-4755 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4755 > Project: Accumulo > Issue Type: Task > Components: monitor >Reporter: Christopher Tubbs >Priority: Minor > Fix For: 2.0.0 > > > As quick follow-on to ACCUMULO-4745, I think it might be worth taking a look > at registering a custom serializer for AbstractId types, so we can keep a > reference to the original Table.ID objects in the REST POJOs, and still have > the serialization do the right thing. > I found this wiki, which may be helpful. > https://github.com/FasterXML/jackson-docs/wiki/JacksonHowToCustomSerializers -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ACCUMULO-4755) Register a custom serializer for AbstractId types (Table.ID and Namespace.ID)
Christopher Tubbs created ACCUMULO-4755: --- Summary: Register a custom serializer for AbstractId types (Table.ID and Namespace.ID) Key: ACCUMULO-4755 URL: https://issues.apache.org/jira/browse/ACCUMULO-4755 Project: Accumulo Issue Type: Task Components: monitor Reporter: Christopher Tubbs Priority: Minor Fix For: 2.0.0 As quick follow-on to ACCUMULO-4745, I think it might be worth taking a look at registering a custom serializer for AbstractId types, so we can keep a reference to the original Table.ID objects in the REST POJOs, and still have the serialization do the right thing. I found this wiki, which may be helpful. https://github.com/FasterXML/jackson-docs/wiki/JacksonHowToCustomSerializers -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
URL: https://github.com/apache/accumulo/pull/325#discussion_r154789645
##
File path:
core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java
##
@@ -44,14 +44,14 @@
public static MasterClientService.Client
getConnectionWithRetry(ClientContext context) {
while (true) {
- MasterClientService.Client result = getConnection(context);
+ MasterClientService.Client result = getConnection(context, false);
Review comment:
In order to limit the number of changes, it's better to keep a method with
the original method signature that wraps the new version with the extra
parameter, so this (and other callers which are expected to keep their original
behavior) is unaffected by this change.
Example:
```java
public static MasterClientService.Client getConnection(ClientContext
context) {
return getConnection(context, false); // or getConnection(context, 0) with
my suggestion to specify the timeout
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
URL: https://github.com/apache/accumulo/pull/325#discussion_r154791885
##
File path:
core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java
##
@@ -183,4 +193,13 @@ public static void executeVoid(ClientContext context,
ClientExec exec) throws AccumuloException,
AccumuloSecurityException {
Review comment:
Maybe call this `executeVoidWithTimeout`? Again, passing timeout here, and
then down through, would be helpful. The timeout used for `Admin.java` should
be a constant in that class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
URL: https://github.com/apache/accumulo/pull/325#discussion_r154788845
##
File path:
core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java
##
@@ -44,14 +44,14 @@
public static MasterClientService.Client
getConnectionWithRetry(ClientContext context) {
while (true) {
- MasterClientService.Client result = getConnection(context);
+ MasterClientService.Client result = getConnection(context, false);
if (result != null)
return result;
sleepUninterruptibly(250, TimeUnit.MILLISECONDS);
}
}
- public static MasterClientService.Client getConnection(ClientContext
context) {
+ public static MasterClientService.Client getConnection(ClientContext
context, boolean isAdminRequest) {
Review comment:
I'm confused about the `isAdminRequest`. The boolean flag should be named
based on what it controls, not what kind of request the caller is going to use
it for. In this case, maybe something like `boolean withTimeout`. Or better
yet, pass the timeout in as an integer: `long timeoutMillis`, and then the `if`
statement below becomes `if (timeoutMillis > 0)`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
URL: https://github.com/apache/accumulo/pull/325#discussion_r154791198
##
File path:
server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java
##
@@ -274,7 +274,7 @@ public void run() {
while (retry) {
MasterClientService.Iface client = null;
try {
- client = MasterClient.getConnection(context);
+ client = MasterClient.getConnection(context, false);
Review comment:
This file doesn't need to be changed at all.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
URL: https://github.com/apache/accumulo/pull/325#discussion_r154790625
##
File path:
core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java
##
@@ -127,6 +131,12 @@ public static void close(MasterClientService.Iface iface)
{
public static void executeGeneric(ClientContext context,
ClientExec exec) throws AccumuloException,
AccumuloSecurityException,
TableNotFoundException {
+executeGeneric(context, exec, false);
+
+ }
+
+ public static void executeGeneric(ClientContext context,
ClientExec exec, boolean isAdminRequest) throws
AccumuloException,
Review comment:
Maybe call this `executeGenericWithTimeout`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
ctubbsii commented on a change in pull request #325: ACCUMULO-2341?
URL: https://github.com/apache/accumulo/pull/325#discussion_r154790998
##
File path:
core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
##
@@ -244,7 +244,9 @@ private long beginFateOperation() throws
ThriftSecurityException, TException {
while (true) {
MasterClientService.Iface client = null;
try {
-client = MasterClient.getConnectionWithRetry(context);
+client = MasterClient.getConnection(context, true);
Review comment:
This file shouldn't have to be changed at all.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] keith-turner commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
keith-turner commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154786207 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration Review comment: We could even suggest checking the release notes in the user manual upgrade docs. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Accumulo-Master - Build # 2194 - Still Unstable
The Apache Jenkins build system has built Accumulo-Master (build #2194) Status: Still Unstable Check console output at https://builds.apache.org/job/Accumulo-Master/2194/ to view the results.
[GitHub] jmark99 commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
jmark99 commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154784187 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration Review comment: I see your point. Given the option, I would lean toward having the overall process described in the upgrade documentation with the stipulation that a user should check the release notes for any specific issues for that particular version. But I'm interested to hear what others would prefer as well. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (ACCUMULO-4745) Monitor Table hyperlinks are broken
[ https://issues.apache.org/jira/browse/ACCUMULO-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4745. - Resolution: Fixed Closing this, since the PR was merged. > Monitor Table hyperlinks are broken > --- > > Key: ACCUMULO-4745 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4745 > Project: Accumulo > Issue Type: Bug > Components: monitor >Affects Versions: 2.0.0 >Reporter: Kyle Van Gilson >Assignee: Benjamin F > Labels: pull-request-available > Fix For: 2.0.0 > > Time Spent: 2h > Remaining Estimate: 0h > > In the accumulo-monitor Table Status page, the hyperlinked table list items > have malformed links of the form: > * http://localhost:9995/tables/[object%20Object] > The should be of the form > * http://localhost:9995/tables/ > * i.e. http://localhost:9995/tables/1 > I'm not 100% sure where the issue is, but it looks like the > server/monitor/resources/js/tables.js ~line 187 might be where the link > itself is being generated. > To reproduce create an assembly off of the master branch (2.0.0-snapshot), > init/start accumulo master/tserver/monitor, go to > http://localhost:9995/tables/ and any of the table links should exhibit the > issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ACCUMULO-4754) Fix links to properties in 2.0 documentation
Mike Walch created ACCUMULO-4754: Summary: Fix links to properties in 2.0 documentation Key: ACCUMULO-4754 URL: https://issues.apache.org/jira/browse/ACCUMULO-4754 Project: Accumulo Issue Type: Bug Affects Versions: 2.0.0 Reporter: Mike Walch Assignee: Mike Walch Fix For: 2.0.0 If you click on a link for a configuration property in 2.0. It will go to property but property will be hidden below navbar. This can be fixed by adding hidden whitespace. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ctubbsii commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
ctubbsii commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154780260 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration Review comment: @jmark99 Understood. I'm thinking ahead, to the situation where the 2.0 upgrade section of the docs doesn't fully describe a situation for 2.0.9, because of some bug which specifically affected the 2.0.8 to 2.0.9 upgrade (as an example). Would this get added to the main 2.0 docs, or just called out in the 2.0.9 release notes? Or both? Are we prepared to put all patch-specific upgrade notes in the major-minor release notes, or would we want to describe the overall process, and mention "check the release notes for your version, for any upgrade issues specifically for that version"? I don't feel strongly about one way or another. I just want to have a plan in place for when we encounter this scenario. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (ACCUMULO-4753) Update to commons-configuration2
[ https://issues.apache.org/jira/browse/ACCUMULO-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277534#comment-16277534 ] Josh Elser commented on ACCUMULO-4753: -- bq. So this is caused by having two different versions of commons beanutils on the classpath? I wonder if this could be avoided by modifying the classpath to only have one version that works for both commons config versions. Best as I could tell (investigated briefly last night), this isn't possible. The difference in beanutil versions between commons-config(1) and commons-config2 would preclude us from making one or the other work. Another solution would be to shade commons-configuration (and its dependencies) into an Accumulo jar, but shading is generally met with opposition in Accumulo :) > Update to commons-configuration2 > > > Key: ACCUMULO-4753 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4753 > Project: Accumulo > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Critical > > I'm looking at what it would take to get Accumulo working against Hadoop 3. > Thankfully, the only issue seems to be around the commons-configuration2 > update that Hadoop has changed to. With our use of commons-configuration (1) > for ClientConfiguration and others, this brings in multiple versions of > commons-beanutils artifacts which result in exceptions at runtime. > I think the quickest solution is to just upgrade ourselves to use > commons-configuration2. The "long-term" solution is to try to switch over to > the shaded jars that Hadoop is publishing (but I'm reticent to sign up for > that right now :P). > I think we need to have a discussion about where we want to land this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ACCUMULO-4746) Create Builder for Mutation
[
https://issues.apache.org/jira/browse/ACCUMULO-4746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277509#comment-16277509
]
Keith Turner commented on ACCUMULO-4746:
Naming stuff is hard. I don't like {{mutate()}} that much. I think
{{newRowChange()}} is too long. I can't think of anything I like ATM
> Create Builder for Mutation
> ---
>
> Key: ACCUMULO-4746
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4746
> Project: Accumulo
> Issue Type: Sub-task
> Components: client
>Reporter: Keith Turner
>Assignee: Benjamin F
> Fix For: 2.0.0
>
>
> Accumulo needs a builder for mutation similar to the one that was added for
> Key.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ACCUMULO-4753) Update to commons-configuration2
[ https://issues.apache.org/jira/browse/ACCUMULO-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277499#comment-16277499 ] Keith Turner commented on ACCUMULO-4753: So this is caused by having two different versions of commons beanutils on the classpath? I wonder if this could be avoided by modifying the classpath to only have one version that works for both commons config versions. > Update to commons-configuration2 > > > Key: ACCUMULO-4753 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4753 > Project: Accumulo > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Critical > > I'm looking at what it would take to get Accumulo working against Hadoop 3. > Thankfully, the only issue seems to be around the commons-configuration2 > update that Hadoop has changed to. With our use of commons-configuration (1) > for ClientConfiguration and others, this brings in multiple versions of > commons-beanutils artifacts which result in exceptions at runtime. > I think the quickest solution is to just upgrade ourselves to use > commons-configuration2. The "long-term" solution is to try to switch over to > the shaded jars that Hadoop is publishing (but I'm reticent to sign up for > that right now :P). > I think we need to have a discussion about where we want to land this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Accumulo-Master - Build # 2193 - Still Unstable
The Apache Jenkins build system has built Accumulo-Master (build #2193) Status: Still Unstable Check console output at https://builds.apache.org/job/Accumulo-Master/2193/ to view the results.
[GitHub] keith-turner commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
keith-turner commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154775162 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration Review comment: > Once the 2.0 docs are 'released' we could update release notes to link to the We may also want to consider making the readme.md link to the upgrade docs (instead of having upgrading.md). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] milleruntime closed pull request #326: ACCUMULO-4745 Fixed broken links in tables table on monitor
milleruntime closed pull request #326: ACCUMULO-4745 Fixed broken links in
tables table on monitor
URL: https://github.com/apache/accumulo/pull/326
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/problems/ProblemDetailInformation.java
b/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/problems/ProblemDetailInformation.java
index d32ccf5e3b..b38862b572 100644
---
a/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/problems/ProblemDetailInformation.java
+++
b/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/problems/ProblemDetailInformation.java
@@ -29,7 +29,7 @@
// Variable names become JSON keys
public String tableName;
- public Table.ID tableID;
+ public String tableID;
public String type;
public String server;
@@ -44,7 +44,7 @@ public ProblemDetailInformation() {}
*
* @param tableName
* Table name of the problem
- * @param tableID
+ * @param tableId
* Table ID of the problem
* @param type
* Type of problem
@@ -57,9 +57,9 @@ public ProblemDetailInformation() {}
* @param exception
* Exception of the problem
*/
- public ProblemDetailInformation(String tableName, Table.ID tableID, String
type, String server, Long time, String resource, String exception) {
+ public ProblemDetailInformation(String tableName, Table.ID tableId, String
type, String server, Long time, String resource, String exception) {
this.tableName = tableName;
-this.tableID = tableID;
+this.tableID = tableId.canonicalID();
this.type = type;
this.server = server;
this.time = time;
diff --git
a/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/problems/ProblemSummaryInformation.java
b/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/problems/ProblemSummaryInformation.java
index 78fdbadd8a..165bf86cfc 100644
---
a/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/problems/ProblemSummaryInformation.java
+++
b/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/problems/ProblemSummaryInformation.java
@@ -29,7 +29,7 @@
// Variable names become JSON keys
public String tableName;
- public Table.ID tableID;
+ public String tableID;
public Integer fileRead;
public Integer fileWrite;
@@ -42,7 +42,7 @@ public ProblemSummaryInformation() {}
*
* @param tableName
* Name of the table with a problem
- * @param tableID
+ * @param tableId
* ID of the table with a problem
* @param fileRead
* Number of files read
@@ -51,9 +51,9 @@ public ProblemSummaryInformation() {}
* @param tableLoad
* Number of table loads
*/
- public ProblemSummaryInformation(String tableName, Table.ID tableID, Integer
fileRead, Integer fileWrite, Integer tableLoad) {
+ public ProblemSummaryInformation(String tableName, Table.ID tableId, Integer
fileRead, Integer fileWrite, Integer tableLoad) {
this.tableName = tableName;
-this.tableID = tableID;
+this.tableID = tableId.canonicalID();
this.fileRead = fileRead;
this.fileWrite = fileWrite;
this.tableLoad = tableLoad;
diff --git
a/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TableInformation.java
b/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TableInformation.java
index d15104611a..cacd3424a5 100644
---
a/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TableInformation.java
+++
b/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TableInformation.java
@@ -30,7 +30,7 @@
// Variable names become JSON keys
public String tablename;
- public Table.ID tableId;
+ public String tableId;
public String tableState;
public int tablets;
@@ -75,7 +75,7 @@ public TableInformation() {}
*/
public TableInformation(String tableName, Table.ID tableId, String
tableState) {
this.tablename = tableName;
-this.tableId = tableId;
+this.tableId = tableId.canonicalID();
this.tableState = tableState;
}
@@ -95,7 +95,7 @@ public TableInformation(String tableName, Table.ID tableId,
String tableState) {
*/
public TableInformation(String tableName, Table.ID tableId, TableInfo info,
Double holdTime, String tableState) {
this.tablename = tableName;
-this.tableId = tableId;
+this.tableId = tableId.canonicalID();
this.tablets = info.tablets;
this.offlineTablets = info.tablets - info.onlineTablets;
diff --git
a/server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/CurrentOperations.java
[GitHub] milleruntime commented on issue #326: ACCUMULO-4745 Fixed broken links in tables table on monitor
milleruntime commented on issue #326: ACCUMULO-4745 Fixed broken links in tables table on monitor URL: https://github.com/apache/accumulo/pull/326#issuecomment-349098428 I spoke with @ctubbsii briefly and I think he is OK with these changes. He mentioned an idea to make the Monitor code more resilient to future changes but that can be follow on work. I will go ahead and merge these changes. @bfach10 thanks for the contribution! If you would like to be added to the list of [contributors](https://accumulo.apache.org/people/#contributors), please create a pull request against the [website](https://github.com/apache/accumulo-website) and we can add you. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jomach commented on issue #325: ACCUMULO-2341?
jomach commented on issue #325: ACCUMULO-2341? URL: https://github.com/apache/accumulo/pull/325#issuecomment-349097248 @keith-turner Done ! plz review This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Accumulo-Master - Build # 2192 - Still Unstable
The Apache Jenkins build system has built Accumulo-Master (build #2192) Status: Still Unstable Check console output at https://builds.apache.org/job/Accumulo-Master/2192/ to view the results.
[GitHub] jmark99 commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
jmark99 commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154762979 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration Review comment: I can understand the preference for release notes, but the conversation started when a user asked how to upgrade from 1.4 to 1.8 and there was some uncertainty as to how easy it was for users to find the upgrade information, especially across multiple version jumps. The thought was that if there was one central location where the information resided then it would be easier to point users to it and it would also serve as a historical record of upgrade processes. Once the 2.0 docs are 'released' we could update release notes to link to the proper information on this page and allow it to be the official location for upgrade guidance. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] milleruntime closed pull request #289: ACCUMULO-4677 Sanitizing PathParam values in REST-based Monitor
milleruntime closed pull request #289: ACCUMULO-4677 Sanitizing PathParam
values in REST-based Monitor
URL: https://github.com/apache/accumulo/pull/289
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/assemble/pom.xml b/assemble/pom.xml
index 2cf4d7aee0..4167de3d83 100644
--- a/assemble/pom.xml
+++ b/assemble/pom.xml
@@ -32,6 +32,10 @@
jcommander
true
+
+ com.fasterxml
+ classmate
+
com.fasterxml.jackson.core
jackson-annotations
@@ -79,6 +83,10 @@
javax.annotation
javax.annotation-api
+
+ javax.el
+ javax.el-api
+
javax.servlet
javax.servlet-api
@@ -283,6 +291,10 @@
org.glassfish.jersey.core
jersey-server
+
+ org.glassfish.jersey.ext
+ jersey-bean-validation
+
org.glassfish.jersey.ext
jersey-entity-filtering
@@ -303,10 +315,26 @@
org.glassfish.jersey.media
jersey-media-json-jackson
+
+ org.glassfish.web
+ el-impl
+
+
+ org.glassfish.web
+ javax.el
+
+
+ org.hibernate
+ hibernate-validator
+
org.javassist
javassist
+
+ org.jboss.logging
+ jboss-logging
+
org.slf4j
slf4j-api
diff --git a/assemble/src/main/assemblies/component.xml
b/assemble/src/main/assemblies/component.xml
index 470b5f7578..405b7b3449 100644
--- a/assemble/src/main/assemblies/component.xml
+++ b/assemble/src/main/assemblies/component.xml
@@ -52,6 +52,7 @@
org.slf4j:slf4j-api
org.slf4j:slf4j-log4j12
+com.fasterxml:classmate
com.fasterxml.jackson.core:jackson-annotations
com.fasterxml.jackson.core:jackson-core
com.fasterxml.jackson.core:jackson-databind
@@ -59,6 +60,7 @@
com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider
com.fasterxml.jackson.module:jackson-module-jaxb-annotations
javax.annotation:javax.annotation-api
+javax.el:javax.el-api
javax.validation:validation-api
javax.ws.rs:javax.ws.rs-api
org.freemarker:freemarker
@@ -75,12 +77,17 @@
org.glassfish.jersey.core:jersey-client
org.glassfish.jersey.core:jersey-common
org.glassfish.jersey.core:jersey-server
+org.glassfish.jersey.ext:jersey-bean-validation
org.glassfish.jersey.ext:jersey-entity-filtering
org.glassfish.jersey.ext:jersey-mvc-freemarker
org.glassfish.jersey.ext:jersey-mvc
org.glassfish.jersey.media:jersey-media-jaxb
org.glassfish.jersey.media:jersey-media-json-jackson
+org.glassfish.web:javax.el
+org.glassfish.web:el-impl
+org.hibernate:hibernate-validator
org.javassist:javassist
+org.jboss.logging:jboss-logging
log4j:log4j
diff --git a/pom.xml b/pom.xml
index 49ce9d8687..4cce01242f 100644
--- a/pom.xml
+++ b/pom.xml
@@ -133,6 +133,7 @@
4.3.1
false
2.9.0
+2.2.4
2.25.1
9.3.21.v20170918
1.8
@@ -162,6 +163,11 @@
jcommander
1.72
+
+com.fasterxml
+classmate
+1.0.0
+
com.fasterxml.jackson.core
jackson-annotations
@@ -259,6 +265,11 @@
javax.annotation-api
1.2
+
+javax.el
+javax.el-api
+${javax.el.version}
+
javax.servlet
javax.servlet-api
@@ -633,6 +644,11 @@
jersey-server
${jersey.version}
+
+org.glassfish.jersey.ext
+jersey-bean-validation
+${jersey.version}
+
org.glassfish.jersey.ext
jersey-entity-filtering
@@ -658,16 +674,46 @@
jersey-media-json-jackson
${jersey.version}
+
+org.glassfish.jersey.test-framework
+jersey-test-framework-core
+${jersey.version}
+
+
+org.glassfish.jersey.test-framework.providers
+jersey-test-framework-provider-grizzly2
+${jersey.version}
+
+
+org.glassfish.web
+el-impl
+2.2
+
+
+org.glassfish.web
+javax.el
+2.2.4
+
org.hamcrest
hamcrest-core
1.3
+
+org.hibernate
+hibernate-validator
+5.1.3.Final
+
org.javassist
javassist
3.18.1-GA
+
+org.jboss.logging
+jboss-logging
+3.1.3.GA
+
org.powermock
powermock-api-easymock
@@ -1040,11 +1086,14 @@
[GitHub] milleruntime commented on issue #289: ACCUMULO-4677 Sanitizing PathParam values in REST-based Monitor
milleruntime commented on issue #289: ACCUMULO-4677 Sanitizing PathParam values in REST-based Monitor URL: https://github.com/apache/accumulo/pull/289#issuecomment-349086232 Thanks @glitch ! I will merge this This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (ACCUMULO-4752) Create troubleshooting page on improving performance
[ https://issues.apache.org/jira/browse/ACCUMULO-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ACCUMULO-4752: - Labels: pull-request-available (was: ) > Create troubleshooting page on improving performance > > > Key: ACCUMULO-4752 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4752 > Project: Accumulo > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Mike Walch >Assignee: Mike Walch > Labels: pull-request-available > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] mikewalch opened a new pull request #46: ACCUMULO-4752 Create documentation on improving peformance
mikewalch opened a new pull request #46: ACCUMULO-4752 Create documentation on improving peformance URL: https://github.com/apache/accumulo-website/pull/46 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
ctubbsii commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154755856 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration Review comment: Should the upgrade documentation exist in this centralized docs, or should we focus more on upgrade steps in the release notes, as they apply to that specific release? Either way, there should be a mention in both places. I'm just curious where the details should go. Personally, I like them in the release notes more. I'm just worried that the details will end up spreading across both locations over time. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (ACCUMULO-4751) Some WALs don't replicate due to lacking a createdTime entry
[
https://issues.apache.org/jira/browse/ACCUMULO-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277195#comment-16277195
]
Josh Elser commented on ACCUMULO-4751:
--
bq. My initial finding is that some entries coming out of the StatusUtil don't
have a createdTime. Whereas something like [1] writes a metadata entry to a WAL
and the replication status is updated with a createdTime. I am guessing that,
in this case, there is no ingest for a particular table during the lifetime of
the WAL, so the work to replicate the WAL to a particular table (via the key
extent) is never created. However, something else is writing some information
that doesn't have a createdTime.
Yeah, I do recall this distinction. I remember having a separation between when
a WAL is "created" in memory (more like the TServer decides it's going to use
that file), but there aren't any records in that file yet. I could also see
replication missing some part of the lifecycle of the WAL itself as this is
pretty obtuse (not explicitly documented). LMK if you need to knock heads
together.
> Some WALs don't replicate due to lacking a createdTime entry
>
>
> Key: ACCUMULO-4751
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4751
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.7.3, 1.8.1
>Reporter: Adam J Shook
>Assignee: Adam J Shook
>
> From what I can tell, the below error is thrown when no data for a particular
> table is written to a WAL, but the file is closed. This would be because the
> {{Status}} entry from the {{StatusUtil}} for {{fileClosed}} is pre-built and
> therefore does not have a {{createdTime}}. This prevents a WAL from being
> replicated until a {{createdTime}} entry is added manually.
> From the Accumulo master:
> {code}
> Status record ([begin: 0 end: 0 infiniteEnd: true closed: true]) for
> hdfs://namenode:9000/accumulo/wal/tserver.example.com+31732/f922df9c-3ffc-49ee-8d0c-261c7a05fea2
> in table 7l was written to metadata table which lacked createdTime
> {code}
> There are two solutions I have in mind:
> 1. Update the {{StatusUtil}} such that every returned {{Status}} object sets
> the {{createdTime}} to {{System.currentTimeMillis}} if not explicitly given.
> 2. Update the Accumulo Master to set the {{createdTime}} to the WAL's
> modification time in HDFS if the WAL is closed but there is no
> {{createdTime}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ACCUMULO-4753) Update to commons-configuration2
[
https://issues.apache.org/jira/browse/ACCUMULO-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277169#comment-16277169
]
Josh Elser commented on ACCUMULO-4753:
--
Example stacktrace:
{noformat}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.124 sec <<<
FAILURE! - in org.apache.accumulo.core.client.mapred.TokenFileTest
testMR(org.apache.accumulo.core.client.mapred.TokenFileTest) Time elapsed:
1.117 sec <<< ERROR!
java.lang.NoClassDefFoundError: org/apache/commons/beanutils/BeanIntrospector
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.sun.proxy.$Proxy13.(Unknown Source)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:739)
at
org.apache.commons.configuration2.builder.fluent.Parameters.createParametersProxy(Parameters.java:294)
at
org.apache.commons.configuration2.builder.fluent.Parameters.fileBased(Parameters.java:185)
at
org.apache.commons.configuration2.builder.fluent.Configurations.fileParams(Configurations.java:602)
at
org.apache.commons.configuration2.builder.fluent.Configurations.fileParams(Configurations.java:638)
at
org.apache.commons.configuration2.builder.fluent.Configurations.fileBasedBuilder(Configurations.java:164)
at
org.apache.commons.configuration2.builder.fluent.Configurations.propertiesBuilder(Configurations.java:264)
at
org.apache.hadoop.metrics2.impl.MetricsConfig.loadFirst(MetricsConfig.java:115)
at
org.apache.hadoop.metrics2.impl.MetricsConfig.create(MetricsConfig.java:98)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:478)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:188)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:163)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:62)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:58)
at
org.apache.hadoop.mapred.LocalJobRunnerMetrics.create(LocalJobRunnerMetrics.java:45)
at
org.apache.hadoop.mapred.LocalJobRunner.(LocalJobRunner.java:771)
at
org.apache.hadoop.mapred.LocalJobRunner.(LocalJobRunner.java:764)
at
org.apache.hadoop.mapred.LocalClientProtocolProvider.create(LocalClientProtocolProvider.java:42)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:130)
at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:109)
at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:102)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:475)
at org.apache.hadoop.mapred.JobClient.(JobClient.java:454)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:872)
at
org.apache.accumulo.core.client.mapred.TokenFileTest$MRTokenFileTester.run(TokenFileTest.java:134)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.accumulo.core.client.mapred.TokenFileTest$MRTokenFileTester.main(TokenFileTest.java:141)
at
org.apache.accumulo.core.client.mapred.TokenFileTest.testMR(TokenFileTest.java:168)
{noformat}
> Update to commons-configuration2
>
>
> Key: ACCUMULO-4753
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4753
> Project: Accumulo
> Issue Type: Task
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
>
> I'm looking at what it would take to get Accumulo working against Hadoop 3.
> Thankfully, the only issue seems to be around the commons-configuration2
> update that Hadoop has changed to. With our use of commons-configuration (1)
> for ClientConfiguration and others, this brings in multiple versions of
> commons-beanutils artifacts which result in exceptions at runtime.
> I think the quickest solution is to just upgrade ourselves to use
> commons-configuration2. The "long-term" solution is to try to switch over to
> the shaded jars that Hadoop is publishing (but I'm reticent to sign up for
> that right now :P).
> I think we need to have a
[jira] [Created] (ACCUMULO-4753) Update to commons-configuration2
Josh Elser created ACCUMULO-4753: Summary: Update to commons-configuration2 Key: ACCUMULO-4753 URL: https://issues.apache.org/jira/browse/ACCUMULO-4753 Project: Accumulo Issue Type: Task Reporter: Josh Elser Assignee: Josh Elser Priority: Critical I'm looking at what it would take to get Accumulo working against Hadoop 3. Thankfully, the only issue seems to be around the commons-configuration2 update that Hadoop has changed to. With our use of commons-configuration (1) for ClientConfiguration and others, this brings in multiple versions of commons-beanutils artifacts which result in exceptions at runtime. I think the quickest solution is to just upgrade ourselves to use commons-configuration2. The "long-term" solution is to try to switch over to the shaded jars that Hadoop is publishing (but I'm reticent to sign up for that right now :P). I think we need to have a discussion about where we want to land this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ACCUMULO-4751) Some WALs don't replicate due to lacking a createdTime entry
[
https://issues.apache.org/jira/browse/ACCUMULO-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277154#comment-16277154
]
Adam J Shook commented on ACCUMULO-4751:
My initial finding is that some entries coming out of the {{StatusUtil}} don't
have a {{createdTime}}. Whereas something like [1] writes a metadata entry to
a WAL and the replication status is updated with a {{createdTime}}. I am
guessing that, in this case, there is no ingest for a particular table during
the lifetime of the WAL, so the work to replicate the WAL to a particular table
(via the key extent) is never created. However, something else is writing some
information that doesn't have a {{createdTime}}.
I'll do some more digging to see if this can be nailed down.
[1]
https://github.com/apache/accumulo/blob/rel/1.8.1/server/tserver/src/main/java/org/apache/accumulo/tserver/log/TabletServerLogger.java#L390
> Some WALs don't replicate due to lacking a createdTime entry
>
>
> Key: ACCUMULO-4751
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4751
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.7.3, 1.8.1
>Reporter: Adam J Shook
>Assignee: Adam J Shook
>
> From what I can tell, the below error is thrown when no data for a particular
> table is written to a WAL, but the file is closed. This would be because the
> {{Status}} entry from the {{StatusUtil}} for {{fileClosed}} is pre-built and
> therefore does not have a {{createdTime}}. This prevents a WAL from being
> replicated until a {{createdTime}} entry is added manually.
> From the Accumulo master:
> {code}
> Status record ([begin: 0 end: 0 infiniteEnd: true closed: true]) for
> hdfs://namenode:9000/accumulo/wal/tserver.example.com+31732/f922df9c-3ffc-49ee-8d0c-261c7a05fea2
> in table 7l was written to metadata table which lacked createdTime
> {code}
> There are two solutions I have in mind:
> 1. Update the {{StatusUtil}} such that every returned {{Status}} object sets
> the {{createdTime}} to {{System.currentTimeMillis}} if not explicitly given.
> 2. Update the Accumulo Master to set the {{createdTime}} to the WAL's
> modification time in HDFS if the WAL is closed but there is no
> {{createdTime}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] mikewalch closed pull request #44: ACCUMULO-4750 Created caching documentation
mikewalch closed pull request #44: ACCUMULO-4750 Created caching documentation
URL: https://github.com/apache/accumulo-website/pull/44
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/_docs-2-0/administration/caching.md
b/_docs-2-0/administration/caching.md
new file mode 100644
index 000..be99cb5
--- /dev/null
+++ b/_docs-2-0/administration/caching.md
@@ -0,0 +1,46 @@
+---
+title: Caching
+category: administration
+order: 11
+---
+
+Accumulo tablet servers have a **block cache** that buffers data in memory to
limit reads from disk.
+This caching has the following benefits:
+
+* reduces latency when reading data
+* helps alleviate hotspots in tables
+
+The block cache stores index and data blocks. A typical Accumulo read
operation will perform a binary search
+over several index blocks followed by a linear scan of one or more data
blocks. Each tablet server
+has its own block cache that is shared by all hosted tablets. Therefore, block
caches are only enabled
+for tables where read performance is critical.
+
+## Configuration
+
+While the block cache is enabled by default for the Accumulo metadata tables,
it must be enabled
+for all other tables by setting the following table properties to `true`:
+
+* [table.cache.block.enable] - enables data block cache on the table
+* [table.cache.index.enable] - enables index block cache on the table
+
+These properties can be set in the Accumulo shell using the following command:
+
+config -t mytable -s table.cache.block.enable=true
+
+Or programatically using [TableOperations.setProperty()][tableops]:
+
+```java
+conn.tableOperations().setProperty("mytable", "table.cache.block.enable",
"true");
+```
+
+The sizes of the index and data block caches can be changed from their
defaults by setting
+the following properties:
+
+* [tserver.cache.data.size]
+* [tserver.cache.index.size]
+
+[table.cache.block.enable]: {{ page.docs_baseurl
}}/administration/configuration-properties#table_cache_block_enable
+[table.cache.index.enable]: {{ page.docs_baseurl
}}/administration/configuration-properties#table_cache_index_enable
+[tserver.cache.data.size]: {{ page.docs_baseurl
}}/administration/configuration-properties#tserver_cache_data_size
+[tserver.cache.index.size]: {{ page.docs_baseurl
}}/administration/configuration-properties#tserver_cache_data_size
+[tableops]: {{ page.javadoc_core
}}/org/apache/accumulo/core/client/admin/TableOperations.html#setProperty(java.lang.String,
java.lang.String, java.lang.String)
diff --git a/_docs-2-0/getting-started/table_configuration.md
b/_docs-2-0/getting-started/table_configuration.md
index 6e11b10..60090df 100644
--- a/_docs-2-0/getting-started/table_configuration.md
+++ b/_docs-2-0/getting-started/table_configuration.md
@@ -341,24 +341,8 @@ See the [combiner example][combiner-example] for example
code.
## Block Cache
-In order to increase throughput of commonly accessed entries, Accumulo employs
a block cache.
-This block cache buffers data in memory so that it doesn't have to be read off
of disk.
-The RFile format that Accumulo prefers is a mix of index blocks and data
blocks, where the index blocks are used to find the appropriate data blocks.
-Typical queries to Accumulo result in a binary search over several index
blocks followed by a linear scan of one or more data blocks.
-
-The block cache can be configured on a per-table basis, and all tablets hosted
on a tablet server share a single resource pool.
-To configure the size of the tablet server's block cache, set the following
properties:
-
-tserver.cache.data.size: Specifies the size of the cache for file data
blocks.
-tserver.cache.index.size: Specifies the size of the cache for file indices.
-
-To enable the block cache for your table, set the following properties:
-
-table.cache.block.enable: Determines whether file (data) block cache is
enabled.
-table.cache.index.enable: Determines whether index cache is enabled.
-
-The block cache can have a significant effect on alleviating hot spots, as
well as reducing query latency.
-It is enabled by default for the metadata tables.
+A Block Cache can be enabled on tables to limit reads from disk which can
result
+in reduced read latency. Read the [Caching] documentation to learn more.
## Compaction
@@ -653,7 +637,6 @@ losing access to the table. See the [export
example](https://github.com/apache/a
for example code.
[bloom-filter-example]:
https://github.com/apache/accumulo-examples/blob/master/docs/bloom.md
-[config]: /docs/{{ page.version }}/config/
[constraint]: {{ page.javadoc_core
}}/org/apache/accumulo/core/constraints/Constraint.html
[constraints-example]:
https://github.com/apache/accumulo-examples/blob/master/docs/contraints.md
[GitHub] keith-turner commented on issue #45: ACCUMULO-4747 Create a unified upgrade reference
keith-turner commented on issue #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#issuecomment-349037736 Upgrading.md is already gone from master, so there is not duplication for 2.0.0-SNAPSHOT. So please disregard my comments about duplication. Need to look at the issue that removed the files in master branch for insights. https://issues.apache.org/jira/browse/ACCUMULO-4479 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (ACCUMULO-4751) Some WALs don't replicate due to lacking a createdTime entry
[
https://issues.apache.org/jira/browse/ACCUMULO-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277096#comment-16277096
]
Josh Elser commented on ACCUMULO-4751:
--
I would say #2 at first glance, but I am more worried about how we missed the
createdTime situation.
Ideally, even in the face of TServer crashes, the TServer would set the correct
"metadata" on each Status record. Do you have any hunch as to how this record
exists without the createdTime attribute set?
It would be nice to confirm that we don't have some other kind of bug lingering
in which we're just not writing the record correctly. I wouldn't be surprised
if we would actually need some kind of solution (like #2) to guard against some
kind of unlikely situation (e.g. tserver failure) in addition another bug. In
other words, the catch-all to prevent the system from "wedging" on these WALs
would be appreciated.
> Some WALs don't replicate due to lacking a createdTime entry
>
>
> Key: ACCUMULO-4751
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4751
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.7.3, 1.8.1
>Reporter: Adam J Shook
>Assignee: Adam J Shook
>
> From what I can tell, the below error is thrown when no data for a particular
> table is written to a WAL, but the file is closed. This would be because the
> {{Status}} entry from the {{StatusUtil}} for {{fileClosed}} is pre-built and
> therefore does not have a {{createdTime}}. This prevents a WAL from being
> replicated until a {{createdTime}} entry is added manually.
> From the Accumulo master:
> {code}
> Status record ([begin: 0 end: 0 infiniteEnd: true closed: true]) for
> hdfs://namenode:9000/accumulo/wal/tserver.example.com+31732/f922df9c-3ffc-49ee-8d0c-261c7a05fea2
> in table 7l was written to metadata table which lacked createdTime
> {code}
> There are two solutions I have in mind:
> 1. Update the {{StatusUtil}} such that every returned {{Status}} object sets
> the {{createdTime}} to {{System.currentTimeMillis}} if not explicitly given.
> 2. Update the Accumulo Master to set the {{createdTime}} to the WAL's
> modification time in HDFS if the WAL is closed but there is no
> {{createdTime}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] keith-turner commented on issue #45: ACCUMULO-4747 Create a unified upgrade reference
keith-turner commented on issue #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#issuecomment-349030626 Were the sections copied verbatim from the upgrading.md docs? If so, I am wondering about duplication of documentation. It would be nice to avoid duplication. One possible strategy would be to link to the pre 2.0 docs and only have 2.0 upgrade instructions in the manual. Also remove upgrading.md from 2.0 and modify the 2.0 readme to link to the upgrade instructions on the website. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (ACCUMULO-4752) Create troubleshooting page on improving performance
Mike Walch created ACCUMULO-4752: Summary: Create troubleshooting page on improving performance Key: ACCUMULO-4752 URL: https://issues.apache.org/jira/browse/ACCUMULO-4752 Project: Accumulo Issue Type: Task Affects Versions: 2.0.0 Reporter: Mike Walch Assignee: Mike Walch Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ACCUMULO-4751) Some WALs don't replicate due to lacking a createdTime entry
Adam J Shook created ACCUMULO-4751:
--
Summary: Some WALs don't replicate due to lacking a createdTime
entry
Key: ACCUMULO-4751
URL: https://issues.apache.org/jira/browse/ACCUMULO-4751
Project: Accumulo
Issue Type: Bug
Affects Versions: 1.8.1, 1.7.3
Reporter: Adam J Shook
Assignee: Adam J Shook
>From what I can tell, the below error is thrown when no data for a particular
>table is written to a WAL, but the file is closed. This would be because the
>{{Status}} entry from the {{StatusUtil}} for {{fileClosed}} is pre-built and
>therefore does not have a {{createdTime}}. This prevents a WAL from being
>replicated until a {{createdTime}} entry is added manually.
>From the Accumulo master:
{code}
Status record ([begin: 0 end: 0 infiniteEnd: true closed: true]) for
hdfs://namenode:9000/accumulo/wal/tserver.example.com+31732/f922df9c-3ffc-49ee-8d0c-261c7a05fea2
in table 7l was written to metadata table which lacked createdTime
{code}
There are two solutions I have in mind:
1. Update the {{StatusUtil}} such that every returned {{Status}} object sets
the {{createdTime}} to {{System.currentTimeMillis}} if not explicitly given.
2. Update the Accumulo Master to set the {{createdTime}} to the WAL's
modification time in HDFS if the WAL is closed but there is no {{createdTime}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ACCUMULO-4751) Some WALs don't replicate due to lacking a createdTime entry
[
https://issues.apache.org/jira/browse/ACCUMULO-4751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277044#comment-16277044
]
Adam J Shook commented on ACCUMULO-4751:
[~elserj] when you have the time, could you let me know your thoughts on 1 and
2 above? Happy to contribute a fix.
> Some WALs don't replicate due to lacking a createdTime entry
>
>
> Key: ACCUMULO-4751
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4751
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.7.3, 1.8.1
>Reporter: Adam J Shook
>Assignee: Adam J Shook
>
> From what I can tell, the below error is thrown when no data for a particular
> table is written to a WAL, but the file is closed. This would be because the
> {{Status}} entry from the {{StatusUtil}} for {{fileClosed}} is pre-built and
> therefore does not have a {{createdTime}}. This prevents a WAL from being
> replicated until a {{createdTime}} entry is added manually.
> From the Accumulo master:
> {code}
> Status record ([begin: 0 end: 0 infiniteEnd: true closed: true]) for
> hdfs://namenode:9000/accumulo/wal/tserver.example.com+31732/f922df9c-3ffc-49ee-8d0c-261c7a05fea2
> in table 7l was written to metadata table which lacked createdTime
> {code}
> There are two solutions I have in mind:
> 1. Update the {{StatusUtil}} such that every returned {{Status}} object sets
> the {{createdTime}} to {{System.currentTimeMillis}} if not explicitly given.
> 2. Update the Accumulo Master to set the {{createdTime}} to the WAL's
> modification time in HDFS if the WAL is closed but there is no
> {{createdTime}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] mikewalch commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
mikewalch commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154683584 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration Review comment: Nice. I think it's a good idea to put this in the documentation. It can be linked to from release notes but it's nice that this will carry forward in the docs with each release for historical purposes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #44: ACCUMULO-4750 Created caching documentation
mikewalch commented on a change in pull request #44: ACCUMULO-4750 Created caching documentation URL: https://github.com/apache/accumulo-website/pull/44#discussion_r154688964 ## File path: _docs-2-0/administration/caching.md ## @@ -0,0 +1,46 @@ +--- +title: Caching +category: administration +order: 11 +--- + +Accumulo tablet servers have a **block cache** that buffers data in memory to limit reads from disk. +This caching has the following benefits: + +* reduces latency when reading data +* helps alleviate hotspots in tables + +The block cache stores index and data blocks. A typical Accumulo read will perfrom a binary search Review comment: Fixed in 4c22f3edc2b6 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
mikewalch commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154683735 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration +order: 11 +--- + +## Upgrading from 1.7 to 1.8 + +Upgrades from 1.7 to 1.8 are possible with little effort as no changes were made at the data layer and RPC changes were made in a backwards-compatible way. The recommended way is to stop Accumulo 1.7, perform the Accumulo upgrade to 1.8, and then start 1.8. Like previous versions, after 1.8 is started on a 1.7 instance, a one-time upgrade will happen by the Master which will prevent a downgrade back to 1.7. Upgrades are still one way. Upgrades from versions prior to 1.7 to 1.8 should follow the below path to 1.7 and then perform the upgrade to 1.8 ? direct upgrades to 1.8 for versions other than 1.7 are untested. + +Existing configuration files from 1.7 should be compared against the examples provided in 1.8. The 1.7 configuration files should all function with 1.8 code, but you will likely want to include changes found in the 1.8.0 release notes and these release notes for 1.8.1. + +For upgrades from prior to 1.7, follow the upgrade instructions to 1.7 first. + +## Upgrading from 1.7.x to 1.7.y + +The recommended way to upgrade from a prior 1.7.x release is to stop Accumulo, upgrade to 1.7.y and then start 1.7.y. + +When upgrading, there is a known issue if the upgrade fails due to outstanding [FATE] operations, see [ACCUMULO-4496] The work around if this situation is encountered: + +- Start tservers +- Start shell +- Run ```fate print``` to list all +- If completed, just delete with ```fate delete``` +- Start masters once there are no more fate operations + +If any of the FATE operations are not complete, you should rollback the upgrade and troubleshoot completing them with your prior version. When performing an upgrade between major versions, the upgrade is one-way, therefore it is important that you do not have any outstanding FATE operations before starting the upgrade. + +## Upgrading from 1.6 to 1.7 + +Upgrades from 1.6 to 1.7 are possible with little effort as no changes were made at the data layer and RPC changes were made in a backwards-compatible way. The recommended way is to stop Accumulo 1.6, perform the Accumulo upgrade to 1.7, and then start 1.7. Like previous versions, after 1.7.0 is started on a 1.6 instance, a one-time upgrade will happen by the Master which will prevent a downgrade back to 1.6. Upgrades are still one way. Upgrades from versions prior to 1.6 to 1.7 should follow the below path to 1.6 and then perform the upgrade to 1.7 ? direct upgrades to 1.7 for versions other than 1.6 are untested. + +After upgrading to 1.7.0, users will notice the addition of a replication table in the accumulo namespace. This table is created and put offline to avoid any additional maintenance if the data-center replication feature is not in use. + +Existing configuration files from 1.6 should be compared against the examples provided in 1.7. The 1.6 configuration files should all function with 1.7 code, but you will likely want to include a new file (hadoop-metrics2-accumulo.properties) to enable the new metrics subsystem. Read the section on Hadoop Metrics2 in the Administration chapter of the Accumulo User Manual. + +For each of the other new features, new configuration properties exist to support the feature. Refer to the added sections in the User Manual for the feature for information on how to properly configure and use the new functionality. +Testing Review comment: `Testing` can be removed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference
mikewalch commented on a change in pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45#discussion_r154683584 ## File path: _docs-2-0/administration/upgrading.md ## @@ -0,0 +1,110 @@ +--- +title: Upgrading Accumulo +category: administration Review comment: Nice. I think it's a good idea this in the documentation. It can be linked to from release notes but it's nice that this will carry forward in the docs with each release for historical purposes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #44: ACCUMULO-4750 Created caching documentation
keith-turner commented on a change in pull request #44: ACCUMULO-4750 Created caching documentation URL: https://github.com/apache/accumulo-website/pull/44#discussion_r154681136 ## File path: _docs-2-0/administration/caching.md ## @@ -0,0 +1,46 @@ +--- +title: Caching +category: administration +order: 11 +--- + +Accumulo tablet servers have a **block cache** that buffers data in memory to limit reads from disk. +This caching has the following benefits: + +* reduces latency when reading data +* helps alleviate hotspots in tables + +The block cache stores index and data blocks. A typical Accumulo read will perfrom a binary search +over several index blocks followed by a linear scan of one or more data blocks. Each tablet server +has its own block cache that is shared by all hosted tablets. Therefore, block caches are only enabled +for tables where read performance is critical. + +## Configuration + +While the block cache is enabled by default for the Accumulo metadata tables, it must be enabled +for all other tables by setting the following table properties to `true`: + +* [table.cache.block.enable] - enables data block cache on the table Review comment: This is nice, linking to the prop docs. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (ACCUMULO-4747) Create a unified upgrade reference
[ https://issues.apache.org/jira/browse/ACCUMULO-4747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ACCUMULO-4747: - Labels: documentation pull-request-available (was: documentation) > Create a unified upgrade reference > -- > > Key: ACCUMULO-4747 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4747 > Project: Accumulo > Issue Type: Improvement > Components: docs >Reporter: Mark Owens >Assignee: Mark Owens >Priority: Minor > Labels: documentation, pull-request-available > > It could be helpful to have a page which provides upgrade instructions for > the various releases collected into one location. This page could then be > updated as needed as new versions are released. Prompted by someone needing > to upgrade for 1.4 to 1.8 and looking for a good location to find what steps > would be needed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] jmark99 opened a new pull request #45: ACCUMULO-4747 Create a unified upgrade reference
jmark99 opened a new pull request #45: ACCUMULO-4747 Create a unified upgrade reference URL: https://github.com/apache/accumulo-website/pull/45 ACCUMULO-4747 Create a unified upgrade reference Based on discussion with several other developers, we concluded it would be a good idea to gather the various upgrading instructions from around the website into a single unified location that users could use when upgrading Accumulo. This PR is an attempt to collect those instructions from various documents into one location. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
