[
https://issues.apache.org/jira/browse/ACCUMULO-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411743#comment-13411743
]
Keith Turner commented on ACCUMULO-683:
---------------------------------------
One concern I have about issues like this is for indirect users of Accumulo.
An indirect user of Accumulo is someone who is using something like OpenTSDB[1]
or Jaccson[2]. These users want to know as little about Accumuo as possible,
they just want OpenTSDB or Jaccson to work. For these users, the assumption
that setting a config option on the metadata table is a trivial operation does
not hold. It could take them a while to figure out what they need to do to fix
this issue.
This issue is somewhat similar to the java policy file that could keep Accumulo
from running and make users spend a lot of time figuring it out. The issue
differs in that its much less likely and increases the probability of loss of
critical data. It changes probabilities, but it does not make data loss
certain.
Also I think if a user sets max replication, they mean it. I think Accumulo
should just work (automatically take the min setting) and log a warning.
Basically I am in favor of Accumulo just working in as many situations as
possible.
Also, setting the metadata table replication to 5 instead of 3 was something I
did w/o much thought when adding per table replication setting. The basic
premise what that 5 is better than 3. I certainly did not analyze the
probabilities of how it affects your odds under different datanode loss
situations. For example instead of being 5, should it be some function of the
cluster size? I dunno know.
We will probably also run into a similar issue if someone sets the min hdfs
replication > 5. We should handle that issue in this ticket also.
[1] : https://github.com/ericnewton/opentsdb
[2] : https://github.com/acordova/jaccson
John, I like your suggestion about checking in when the user runs init and
prompting. The prompt should suggest a default that will work.
> Accumulo ignores HDFS max replication configuration
> ---------------------------------------------------
>
> Key: ACCUMULO-683
> URL: https://issues.apache.org/jira/browse/ACCUMULO-683
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Affects Versions: 1.4.1
> Reporter: Jim Klucar
> Assignee: Keith Turner
> Priority: Minor
>
> I setup a new 1.4.1 instance that was running on top of a Hadoop installation
> that had the maximum block replications set to 3 and the following error
> showed up on the monitor page.
> java.io.IOException: failed to create file
> /accumulo/tables/!0/table_info/F0000001.rf_tmp on client 127.0.0.1. Requested
> replication 5 exceeds maximum 3 at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1220)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1123)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:551)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> org.apache.hadoop.ipc.RPC$Server.call(RPC.java:523) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1383) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1379) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:396) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1377)
> Tablet server error is:
> 10 10:56:25,408 [tabletserver.MinorCompactor] WARN : MinC failed
> (java.io.IOException: failed to create file /accumulo/tables/!0/
> table_info/F0000001.rf_tmp on client 127.0.0.1.
> Requested replication 5 exceeds maximum 3
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1220)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1123)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:551)
> at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:523)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1383)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1379)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1377)
> ) to create /accumulo/tables/!0/table_info/F0000001.rf_tmp retrying ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
