[
https://issues.apache.org/jira/browse/HBASE-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632919#comment-13632919
]
Jonathan Hsieh commented on HBASE-8347:
---------------------------------------
I gave it some thought -- since this is per HStore and per HRegion (and there
are a lot of them) memory usage is probably a good reason for having this
exist. Next question is why not add something so that the mutability api acts
the same as the base classes?
bq. hadoop 1.0 (1.1) doesn't use ProtobufRpcEngine. So the issue doesn't exist
in hadoop 1.0
Ok, so in summary, the hadoop2-only NameNodeProxies part of the hdfs client
attempts to modify the configuration we pass in but fails because it is
unchangeable. I guess the hadoop1 the hdfs client doesn't try to modify so
there is no attempt to modify the immutable config. Is this right?
Is there a good reason for having the CompoundConfiguration immutable (other
than laziness?).
bq. Changes in LoadIncrementalHFiles.java add waiting for the
ThreadPoolExecutor to shutdown.
bq. Changes in TestLoadIncrementalHFilesSplitRecovery close HTable instance.
Yes, I can see that. Are they in scope for this jira (e.g. are the required to
fix the hang due to UnsupportedOperationException in TestSecureLoadInc*) or
should they be a separate issue?
> TestSecureLoadInc* tests hang repeatedly getting
> UnsupportedOperationException in hadoop2 profile
> --------------------------------------------------------------------------------------------------
>
> Key: HBASE-8347
> URL: https://issues.apache.org/jira/browse/HBASE-8347
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop2, regionserver, security
> Affects Versions: 0.98.0, 0.95.0
> Reporter: Jonathan Hsieh
> Assignee: Ted Yu
> Fix For: 0.95.1
>
> Attachments: 8347-v1.txt
>
>
> These two unit tests fail by hanging under the hadoop2 profile.
> Running
> org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
> Running org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
> {code}
> 2013-04-15 14:03:50,311 ERROR [IPC Server handler 0 on 53755]
> access.SecureBulkLoadEndpoint$1(240): Failed to complete bulk load
> java.lang.UnsupportedOperationException: Immutable Configuration
> at
> org.apache.hadoop.hbase.CompoundConfiguration.setClass(CompoundConfiguration.java:511)
> at org.apache.hadoop.ipc.RPC.setProtocolEngine(RPC.java:193)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createNNProxyWithClientProtocol(NameNodeProxies.java:249)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:168)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:409)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:376)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:123)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2296)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:85)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2330)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2312)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:316)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:162)
> at
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:224)
> at
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:218)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:337)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1347)
> at
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.secureBulkLoadHFiles(SecureBulkLoadEndpoint.java:218)
> at
> org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos$SecureBulkLoadService.callMethod(SecureBulkLoadProtos.java:3654)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:5024)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3076)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.ProtobufRpcServerEngine$Server.call(ProtobufRpcServerEngine.java:174)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1871)
> {code}
> There has been some discussion from Ted and Andrew about an acceptable
> approaches to fix the problem here :
> https://issues.apache.org/jira/browse/HBASE-8258?focusedCommentId=13623091&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13623091
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
