The following JIRA has been integrated to branch 2.2 : HADOOP-10132 RPC#stopProxy() should log the class of proxy when IllegalArgumentException is encountered
FYI On Mon, Nov 25, 2013 at 9:56 PM, Ted Yu <[email protected]> wrote: > Update: > Henry tried my patch attached to HBASE-10029 > > From master log, it seems my patch worked. > > I will get back to this thread after further testing / code review. > > Cheers > > On Nov 25, 2013, at 6:05 PM, Henry Hung <[email protected]> wrote: > > > @Ted: > > > > I create the JIRA, is the information sufficient? > > https://issues.apache.org/jira/browse/HBASE-10029 > > > > Best regards, > > Henry > > > > -----Original Message----- > > From: Ted Yu [mailto:[email protected]] > > Sent: Tuesday, November 26, 2013 9:30 AM > > To: [email protected] > > Subject: Re: hbase 0.96 stop master receive ERROR ipc.RPC: RPC.stopProxy > called on non proxy. > > > > Henry: > > Thanks for the additional information. > > > > Looks like HA namenode with QJM is not covered by current code. > > > > Mind filing a JIRA with summary of this thread ? > > > > Cheers > > > > > > On Tue, Nov 26, 2013 at 9:12 AM, Henry Hung <[email protected]> wrote: > > > >> @Ted > >> Yes, I use the hadoop-hdfs-2.2.0.jar. > >> > >> BTW, how do you certain that the namenode class is > >> ClientNamenodeProtocolTranslatorPB? > >> > >> From the NameNodeProxies, I can only assume the > >> ClientNamenodeProtocolTranslatorPB is used only when connecting to > >> single hadoop namenode. > >> > >> public static <T> ProxyAndInfo<T> createNonHAProxy( > >> Configuration conf, InetSocketAddress nnAddr, Class<T> xface, > >> UserGroupInformation ugi, boolean withRetries) throws IOException { > >> Text dtService = SecurityUtil.buildTokenService(nnAddr); > >> > >> T proxy; > >> if (xface == ClientProtocol.class) { > >> proxy = (T) createNNProxyWithClientProtocol(nnAddr, conf, ugi, > >> withRetries); > >> > >> > >> But I'm using HA configuration using QJM, so the my guess is the > >> createProxy will go to the HA case because I provide > >> failoverProxyProviderClass with > >> > "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider". > >> > >> public static <T> ProxyAndInfo<T> createProxy(Configuration conf, > >> URI nameNodeUri, Class<T> xface) throws IOException { > >> Class<FailoverProxyProvider<T>> failoverProxyProviderClass = > >> getFailoverProxyProviderClass(conf, nameNodeUri, xface); > >> > >> if (failoverProxyProviderClass == null) { > >> // Non-HA case > >> return createNonHAProxy(conf, NameNode.getAddress(nameNodeUri), > >> xface, > >> UserGroupInformation.getCurrentUser(), true); > >> } else { > >> // HA case > >> FailoverProxyProvider<T> failoverProxyProvider = NameNodeProxies > >> .createFailoverProxyProvider(conf, > >> failoverProxyProviderClass, xface, > >> nameNodeUri); > >> Conf config = new Conf(conf); > >> T proxy = (T) RetryProxy.create(xface, failoverProxyProvider, > >> RetryPolicies > >> .failoverOnNetworkException(RetryPolicies.TRY_ONCE_THEN_FAIL, > >> config.maxFailoverAttempts, config.failoverSleepBaseMillis, > >> config.failoverSleepMaxMillis)); > >> > >> Text dtService = > HAUtil.buildTokenServiceForLogicalUri(nameNodeUri); > >> return new ProxyAndInfo<T>(proxy, dtService); > >> } > >> } > >> > >> Here is the snippet of my hdfs-site.xml: > >> > >> <property> > >> <name>dfs.nameservices</name> > >> <value>hadoopdev</value> > >> </property> > >> <property> > >> <name>dfs.ha.namenodes.hadoopdev</name> > >> <value>nn1,nn2</value> > >> </property> > >> <property> > >> <name>dfs.namenode.rpc-address.hadoopdev.nn1</name> > >> <value>fphd9.ctpilot1.com:9000</value> > >> </property> > >> <property> > >> <name>dfs.namenode.http-address.hadoopdev.nn1</name> > >> <value>fphd9.ctpilot1.com:50070</value> > >> </property> > >> <property> > >> <name>dfs.namenode.rpc-address.hadoopdev.nn2</name> > >> <value>fphd10.ctpilot1.com:9000</value> > >> </property> > >> <property> > >> <name>dfs.namenode.http-address.hadoopdev.nn2</name> > >> <value>fphd10.ctpilot1.com:50070</value> > >> </property> > >> <property> > >> <name>dfs.namenode.shared.edits.dir</name> > >> <value>qjournal://fphd8.ctpilot1.com:8485;fphd9.ctpilot1.com:8485; > >> fphd10.ctpilot1.com:8485/hadoopdev</value> > >> </property> > >> <property> > >> <name>dfs.client.failover.proxy.provider.hadoopdev</name> > >> > >> > <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> > >> </property> > >> <property> > >> <name>dfs.ha.fencing.methods</name> > >> <value>shell(/bin/true)</value> > >> </property> > >> <property> > >> <name>dfs.journalnode.edits.dir</name> > >> <value>/data/hadoop/hadoop-data-2/journal</value> > >> </property> > >> <property> > >> <name>dfs.ha.automatic-failover.enabled</name> > >> <value>true</value> > >> </property> > >> <property> > >> <name>ha.zookeeper.quorum</name> > >> <value>fphd1.ctpilot1.com:2222</value> > >> </property> > >> > >> -----Original Message----- > >> From: Ted Yu [mailto:[email protected]] > >> Sent: Tuesday, November 26, 2013 1:56 AM > >> To: [email protected] > >> Subject: Re: hbase 0.96 stop master receive ERROR ipc.RPC: > >> RPC.stopProxy called on non proxy. > >> > >> Here is the caller to createReorderingProxy(): > >> > >> ClientProtocol cp1 = createReorderingProxy(namenode, lrb, conf); > >> > >> where namenode > >> is org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB > : > >> > >> public class ClientNamenodeProtocolTranslatorPB implements > >> > >> ProtocolMetaInterface, ClientProtocol, Closeable, > >> ProtocolTranslator { > >> > >> In createReorderingProxy() : > >> > >> new Class[]{ClientProtocol.class, Closeable.class}, > >> > >> We ask for Closeable interface. > >> > >> > >> Did the error persist after you replaced with the hadoop-hdfs-2.2.0.jar > ? > >> Meaning, did you start HBase using the new hadoop jars ? > >> > >> Cheers > >> > >> > >> On Mon, Nov 25, 2013 at 1:04 PM, Henry Hung <[email protected]> > wrote: > >> > >>> I looked into the source code of > >>> org/apache/hadoop/hbase/fs/HFileSystem.java > >>> and whenever I execute hbase-daemon.sh stop master (or > >>> regionserver), the > >>> method.getName() is "close", > >>> but the org/apache/hadoop/hdfs/protocol/ClientProtocol.java do not > >>> have method named "close", thus it result in error "object is not an > >>> instance of declaring class" > >>> > >>> Could someone that familiar with hbase-0.96.0 hadoop2 can tell me if > >>> this problem need to be fixed? And how to fix it? > >>> > >>> private static ClientProtocol createReorderingProxy(final > >>> ClientProtocol cp, final ReorderBlocks lrb, final Configuration conf) { > >>> return (ClientProtocol) Proxy.newProxyInstance > >>> (cp.getClass().getClassLoader(), > >>> new Class[]{ClientProtocol.class, Closeable.class}, > >>> new InvocationHandler() { > >>> public Object invoke(Object proxy, Method method, > >>> Object[] args) throws Throwable { > >>> try { > >>> // method.invoke will failed if > >>> method.getName().equals("close") > >>> // because ClientProtocol do not have method "close" > >>> Object res = method.invoke(cp, args); > >>> if (res != null && args != null && args.length == 3 > >>> && "getBlockLocations".equals(method.getName()) > >>> && res instanceof LocatedBlocks > >>> && args[0] instanceof String > >>> && args[0] != null) { > >>> lrb.reorderBlocks(conf, (LocatedBlocks) res, > >>> (String) args[0]); > >>> } > >>> return res; > >>> } catch (InvocationTargetException ite) { > >>> // We will have this for all the exception, > >>> checked on not, sent > >>> // by any layer, including the functional exception > >>> Throwable cause = ite.getCause(); > >>> if (cause == null){ > >>> throw new RuntimeException( > >>> "Proxy invocation failed and getCause is > >>> null", > >> ite); > >>> } > >>> if (cause instanceof UndeclaredThrowableException) { > >>> Throwable causeCause = cause.getCause(); > >>> if (causeCause == null) { > >>> throw new > >>> RuntimeException("UndeclaredThrowableException had null cause!"); > >>> } > >>> cause = cause.getCause(); > >>> } > >>> throw cause; > >>> } > >>> } > >>> }); > >>> } > >>> > >>> > >>> > >>> -----Original Message----- > >>> From: MA11 YTHung1 > >>> Sent: Thursday, November 21, 2013 9:57 AM > >>> To: [email protected] > >>> Subject: RE: hbase 0.96 stop master receive ERROR ipc.RPC: > >>> RPC.stopProxy called on non proxy. > >>> > >>> Additional information: > >>> > >>> I replace all files with prefix hadoop in hbase-0.96.0-hadoop2/lib > >>> with > >>> hadoop-2.2.0 libraries. > >>> > >>> the ls -l of hbase-0.96.0-hadoop2/lib as below: > >>> > >>> -rw-r--r-- 1 hadoop users 62983 Sep 17 16:13 activation-1.1.jar > >>> -rw-r--r-- 1 hadoop users 4467 Sep 17 23:29 aopalliance-1.0.jar > >>> -rw-r--r-- 1 hadoop users 43033 Sep 17 16:13 asm-3.1.jar > >>> -rw-r--r-- 1 hadoop users 263268 Sep 17 16:27 avro-1.5.3.jar > >>> -rw-r--r-- 1 hadoop users 188671 Sep 17 16:12 > >> commons-beanutils-1.7.0.jar > >>> -rw-r--r-- 1 hadoop users 206035 Sep 17 16:13 > >>> commons-beanutils-core-1.8.0.jar > >>> -rw-r--r-- 1 hadoop users 41123 Sep 17 16:12 commons-cli-1.2.jar > >>> -rw-r--r-- 1 hadoop users 259600 Sep 17 16:13 commons-codec-1.7.jar > >>> -rw-r--r-- 1 hadoop users 575389 Sep 17 16:12 > >>> commons-collections-3.2.1.jar > >>> -rw-r--r-- 1 hadoop users 238681 Sep 17 16:27 > commons-compress-1.4.jar > >>> -rw-r--r-- 1 hadoop users 298829 Sep 17 16:13 > >>> commons-configuration-1.6.jar > >>> -rw-r--r-- 1 hadoop users 24239 Sep 17 23:28 > commons-daemon-1.0.13.jar > >>> -rw-r--r-- 1 hadoop users 143602 Sep 17 16:12 > commons-digester-1.8.jar > >>> -rw-r--r-- 1 hadoop users 112341 Sep 17 16:13 commons-el-1.0.jar > >>> -rw-r--r-- 1 hadoop users 305001 Sep 17 16:12 > >> commons-httpclient-3.1.jar > >>> -rw-r--r-- 1 hadoop users 185140 Sep 17 16:13 commons-io-2.4.jar > >>> -rw-r--r-- 1 hadoop users 284220 Sep 17 16:12 commons-lang-2.6.jar > >>> -rw-r--r-- 1 hadoop users 60686 Sep 17 16:12 > commons-logging-1.1.1.jar > >>> -rw-r--r-- 1 hadoop users 988514 Sep 17 16:13 commons-math-2.2.jar > >>> -rw-r--r-- 1 hadoop users 273370 Sep 17 16:27 commons-net-3.1.jar > >>> -rw-r--r-- 1 hadoop users 3566844 Sep 17 16:15 core-3.1.1.jar > >>> -rw-r--r-- 1 hadoop users 15322 Sep 17 16:12 > >>> findbugs-annotations-1.3.9-1.jar > >>> -rw-r--r-- 1 hadoop users 21817 Sep 17 23:29 > >>> gmbal-api-only-3.0.0-b023.jar > >>> -rw-r--r-- 1 hadoop users 684337 Sep 17 23:29 > >> grizzly-framework-2.1.1.jar > >>> -rw-r--r-- 1 hadoop users 210846 Sep 17 23:29 > >>> grizzly-framework-2.1.1-tests.jar > >>> -rw-r--r-- 1 hadoop users 248346 Sep 17 23:29 grizzly-http-2.1.1.jar > >>> -rw-r--r-- 1 hadoop users 193583 Sep 17 23:29 > >>> grizzly-http-server-2.1.1.jar > >>> -rw-r--r-- 1 hadoop users 336878 Sep 17 23:29 > >>> grizzly-http-servlet-2.1.1.jar > >>> -rw-r--r-- 1 hadoop users 8072 Sep 17 23:29 grizzly-rcm-2.1.1.jar > >>> -rw-r--r-- 1 hadoop users 1795932 Sep 17 16:13 guava-12.0.1.jar > >>> -rw-r--r-- 1 hadoop users 710492 Sep 17 23:29 guice-3.0.jar > >>> -rw-r--r-- 1 hadoop users 65012 Sep 17 23:29 guice-servlet-3.0.jar > >>> -rw-r--r-- 1 hadoop users 16778 Nov 20 17:39 > >>> hadoop-annotations-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 49750 Nov 20 17:40 hadoop-auth-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 2576 Oct 12 06:20 > >>> hadoop-client-2.1.0-beta.jar > >>> -rw-r--r-- 1 hadoop users 2735584 Nov 20 17:50 > >>> hadoop-common-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 5242252 Nov 21 08:48 > >>> hadoop-hdfs-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 1988460 Nov 21 08:48 > >> hadoop-hdfs-2.2.0-tests.jar > >>> -rw-r--r-- 1 hadoop users 482042 Nov 21 08:49 > >>> hadoop-mapreduce-client-app-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 656365 Nov 21 08:49 > >>> hadoop-mapreduce-client-common-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 1455001 Nov 21 08:50 > >>> hadoop-mapreduce-client-core-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 35216 Nov 21 08:50 > >>> hadoop-mapreduce-client-jobclient-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 1434852 Nov 21 08:50 > >>> hadoop-mapreduce-client-jobclient-2.2.0-tests.jar > >>> -rw-r--r-- 1 hadoop users 21537 Nov 21 08:51 > >>> hadoop-mapreduce-client-shuffle-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 1158936 Nov 21 08:51 > hadoop-yarn-api-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 94728 Nov 21 08:51 > >>> hadoop-yarn-client-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 1301627 Nov 21 08:51 > >>> hadoop-yarn-common-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 175554 Nov 21 08:52 > >>> hadoop-yarn-server-common-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 467638 Nov 21 08:52 > >>> hadoop-yarn-server-nodemanager-2.2.0.jar > >>> -rw-r--r-- 1 hadoop users 825853 Oct 12 06:28 > >>> hbase-client-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 354845 Oct 12 06:28 > >>> hbase-common-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 132690 Oct 12 06:28 > >>> hbase-common-0.96.0-hadoop2-tests.jar > >>> -rw-r--r-- 1 hadoop users 97428 Oct 12 06:28 > >>> hbase-examples-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 72765 Oct 12 06:28 > >>> hbase-hadoop2-compat-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 32096 Oct 12 06:28 > >>> hbase-hadoop-compat-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 12174 Oct 12 06:28 > >> hbase-it-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 288784 Oct 12 06:28 > >>> hbase-it-0.96.0-hadoop2-tests.jar > >>> -rw-r--r-- 1 hadoop users 94784 Oct 12 06:28 > >>> hbase-prefix-tree-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 3134214 Oct 12 06:28 > >>> hbase-protocol-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 3058804 Oct 12 06:28 > >>> hbase-server-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 3150292 Oct 12 06:28 > >>> hbase-server-0.96.0-hadoop2-tests.jar > >>> -rw-r--r-- 1 hadoop users 12554 Oct 12 06:28 > >>> hbase-shell-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 10941 Oct 12 06:28 > >>> hbase-testing-util-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 2276333 Oct 12 06:28 > >>> hbase-thrift-0.96.0-hadoop2.jar > >>> -rw-r--r-- 1 hadoop users 95975 Sep 17 16:15 > high-scale-lib-1.1.1.jar > >>> -rw-r--r-- 1 hadoop users 31020 Sep 17 16:14 htrace-core-2.01.jar > >>> -rw-r--r-- 1 hadoop users 352585 Sep 17 16:15 httpclient-4.1.3.jar > >>> -rw-r--r-- 1 hadoop users 181201 Sep 17 16:15 httpcore-4.1.3.jar > >>> -rw-r--r-- 1 hadoop users 227517 Sep 17 16:13 > >> jackson-core-asl-1.8.8.jar > >>> -rw-r--r-- 1 hadoop users 17884 Sep 17 16:13 jackson-jaxrs-1.8.8.jar > >>> -rw-r--r-- 1 hadoop users 669065 Sep 17 16:13 > >>> jackson-mapper-asl-1.8.8.jar > >>> -rw-r--r-- 1 hadoop users 32353 Sep 17 16:13 jackson-xc-1.8.8.jar > >>> -rw-r--r-- 1 hadoop users 20642 Sep 17 16:15 jamon-runtime-2.3.1.jar > >>> -rw-r--r-- 1 hadoop users 408133 Sep 17 16:13 > >> jasper-compiler-5.5.23.jar > >>> -rw-r--r-- 1 hadoop users 76844 Sep 17 16:13 > jasper-runtime-5.5.23.jar > >>> -rw-r--r-- 1 hadoop users 2497 Sep 17 23:29 javax.inject-1.jar > >>> -rw-r--r-- 1 hadoop users 83586 Sep 17 23:29 javax.servlet-3.0.jar > >>> -rw-r--r-- 1 hadoop users 105134 Sep 17 16:27 jaxb-api-2.2.2.jar > >>> -rw-r--r-- 1 hadoop users 890168 Sep 17 16:13 jaxb-impl-2.2.3-1.jar > >>> -rw-r--r-- 1 hadoop users 129217 Sep 17 23:29 jersey-client-1.8.jar > >>> -rw-r--r-- 1 hadoop users 458233 Sep 17 16:13 jersey-core-1.8.jar > >>> -rw-r--r-- 1 hadoop users 17585 Sep 17 23:29 jersey-grizzly2-1.8.jar > >>> -rw-r--r-- 1 hadoop users 14712 Sep 17 23:29 jersey-guice-1.8.jar > >>> -rw-r--r-- 1 hadoop users 147933 Sep 17 16:13 jersey-json-1.8.jar > >>> -rw-r--r-- 1 hadoop users 694352 Sep 17 16:13 jersey-server-1.8.jar > >>> -rw-r--r-- 1 hadoop users 28034 Sep 17 23:29 > >>> jersey-test-framework-core-1.8.jar > >>> -rw-r--r-- 1 hadoop users 12907 Sep 17 23:29 > >>> jersey-test-framework-grizzly2-1.8.jar > >>> -rw-r--r-- 1 hadoop users 321806 Sep 17 16:27 jets3t-0.6.1.jar > >>> -rw-r--r-- 1 hadoop users 75963 Sep 17 16:13 jettison-1.3.1.jar > >>> -rw-r--r-- 1 hadoop users 539912 Sep 17 16:13 jetty-6.1.26.jar > >>> -rw-r--r-- 1 hadoop users 18891 Sep 17 16:15 > >> jetty-sslengine-6.1.26.jar > >>> -rw-r--r-- 1 hadoop users 177131 Sep 17 16:13 jetty-util-6.1.26.jar > >>> -rw-r--r-- 1 hadoop users 13832273 Sep 17 16:15 > jruby-complete-1.6.8.jar > >>> -rw-r--r-- 1 hadoop users 185746 Sep 17 16:27 jsch-0.1.42.jar > >>> -rw-r--r-- 1 hadoop users 1024680 Sep 17 16:13 jsp-2.1-6.1.14.jar > >>> -rw-r--r-- 1 hadoop users 134910 Sep 17 16:13 jsp-api-2.1-6.1.14.jar > >>> -rw-r--r-- 1 hadoop users 100636 Sep 17 16:27 jsp-api-2.1.jar > >>> -rw-r--r-- 1 hadoop users 33015 Sep 17 16:13 jsr305-1.3.9.jar > >>> -rw-r--r-- 1 hadoop users 245039 Sep 17 16:12 junit-4.11.jar > >>> -rw-r--r-- 1 hadoop users 347531 Sep 17 16:15 libthrift-0.9.0.jar > >>> -rw-r--r-- 1 hadoop users 489884 Sep 17 16:12 log4j-1.2.17.jar > >>> -rw-r--r-- 1 hadoop users 42212 Sep 17 23:29 > >>> management-api-3.0.0-b012.jar > >>> -rw-r--r-- 1 hadoop users 82445 Sep 17 16:14 metrics-core-2.1.2.jar > >>> drwxr-xr-x 3 hadoop users 4096 Nov 21 09:10 native > >>> -rw-r--r-- 1 hadoop users 1206119 Sep 18 04:00 netty-3.6.6.Final.jar > >>> -rw-r--r-- 1 hadoop users 29555 Sep 17 16:27 paranamer-2.3.jar > >>> -rw-r--r-- 1 hadoop users 533455 Sep 17 16:13 protobuf-java-2.5.0.jar > >>> drwxr-xr-x 5 hadoop users 4096 Sep 28 10:37 ruby > >>> -rw-r--r-- 1 hadoop users 132368 Sep 17 16:13 > >> servlet-api-2.5-6.1.14.jar > >>> -rw-r--r-- 1 hadoop users 105112 Sep 17 16:12 servlet-api-2.5.jar > >>> -rw-r--r-- 1 hadoop users 25962 Sep 17 16:14 slf4j-api-1.6.4.jar > >>> -rw-r--r-- 1 hadoop users 9748 Oct 3 07:15 slf4j-log4j12-1.6.4.jar > >>> -rw-r--r-- 1 hadoop users 995720 Sep 17 16:27 snappy-java-1.0.3.2.jar > >>> -rw-r--r-- 1 hadoop users 26514 Sep 17 16:13 stax-api-1.0.1.jar > >>> -rw-r--r-- 1 hadoop users 15010 Sep 17 16:13 xmlenc-0.52.jar > >>> -rw-r--r-- 1 hadoop users 94672 Sep 17 16:27 xz-1.0.jar > >>> -rw-r--r-- 1 hadoop users 779974 Sep 17 16:14 zookeeper-3.4.5.jar > >>> > >>> Best regards, > >>> Henry > >>> > >>> -----Original Message----- > >>> From: MA11 YTHung1 > >>> Sent: Thursday, November 21, 2013 9:51 AM > >>> To: [email protected] > >>> Subject: RE: hbase 0.96 stop master receive ERROR ipc.RPC: > >>> RPC.stopProxy called on non proxy. > >>> > >>> I'm using hadoop-2.2.0 stable > >>> > >>> -----Original Message----- > >>> From: Jimmy Xiang [mailto:[email protected]] > >>> Sent: Thursday, November 21, 2013 9:49 AM > >>> To: user > >>> Subject: Re: hbase 0.96 stop master receive ERROR ipc.RPC: > >>> RPC.stopProxy called on non proxy. > >>> > >>> Which version of Hadoop do you use? > >>> > >>> > >>> On Wed, Nov 20, 2013 at 5:43 PM, Henry Hung <[email protected]> > wrote: > >>> > >>>> Hi All, > >>>> > >>>> When stopping master or regionserver, I found some ERROR and WARN > >>>> in the log files, are these errors can cause problem in hbase: > >>>> > >>>> 13/11/21 09:31:16 INFO zookeeper.ClientCnxn: EventThread shut down > >>>> 13/11/21 09:35:36 ERROR ipc.RPC: RPC.stopProxy called on non proxy. > >>>> java.lang.IllegalArgumentException: object is not an instance of > >>>> declaring class > >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>>> at > >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl > >>> .j > >>> ava:39) > >>>> at > >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce > >>> ss > >>> orImpl.java:25) > >>>> at java.lang.reflect.Method.invoke(Method.java:597) > >>>> at > >>>> org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:266) > >>>> at $Proxy18.close(Unknown Source) > >>>> at org.apache.hadoop.ipc.RPC.stopProxy(RPC.java:621) > >>>> at > >>> org.apache.hadoop.hdfs.DFSClient.closeConnectionToNamenode(DFSClient > >>> .j > >>> ava:738) > >>>> at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:794) > >>>> at > >>> org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSy > >>> st > >>> em.java:847) > >>>> at > >>>> org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2524) > >>>> at > >>> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem > >>> .j > >>> ava:2541) > >>>> at > >>>> org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManag > >>>> er > >>>> .j > >>>> ava:54) > >>>> 13/11/21 09:35:36 WARN util.ShutdownHookManager: ShutdownHook > >>>> 'ClientFinalizer' failed, > >>> org.apache.hadoop.HadoopIllegalArgumentException: > >>>> Cannot close proxy - is not Closeable or does not provide > >>>> closeable invocation handler class $Proxy18 > >>>> org.apache.hadoop.HadoopIllegalArgumentException: Cannot close > >>>> proxy > >>>> - is not Closeable or does not provide closeable invocation > >>>> handler class > >>>> $Proxy18 > >>>> at org.apache.hadoop.ipc.RPC.stopProxy(RPC.java:639) > >>>> at > >>> org.apache.hadoop.hdfs.DFSClient.closeConnectionToNamenode(DFSClient > >>> .j > >>> ava:738) > >>>> at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:794) > >>>> at > >>> org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSy > >>> st > >>> em.java:847) > >>>> at > >>>> org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2524) > >>>> at > >>> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem > >>> .j > >>> ava:2541) > >>>> at > >>>> org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManag > >>>> er > >>>> .j > >>>> ava:54) > >>>> > >>>> Best regards, > >>>> Henry > >>>> > >>>> ________________________________ > >>>> The privileged confidential information contained in this email is > >>>> intended for use only by the addressees as indicated by the > >>>> original sender of this email. If you are not the addressee > >>>> indicated in this email or are not responsible for delivery of the > >>>> email to such a person, please kindly reply to the sender > >>>> indicating this fact and delete all copies of it from your > >>>> computer and network server immediately. Your cooperation is > >>>> highly appreciated. It is advised that any unauthorized use of > >>>> confidential information of Winbond is strictly prohibited; and > >>>> any information in this email irrelevant to the official business > >>>> of Winbond shall be deemed as neither given nor > >>> endorsed by Winbond. > >>> > >>> The privileged confidential information contained in this email is > >>> intended for use only by the addressees as indicated by the original > >>> sender of this email. If you are not the addressee indicated in this > >>> email or are not responsible for delivery of the email to such a > >>> person, please kindly reply to the sender indicating this fact and > >>> delete all copies of it from your computer and network server > >>> immediately. Your cooperation is highly appreciated. It is advised > >>> that any unauthorized use of confidential information of Winbond is > >>> strictly prohibited; and any information in this email irrelevant to > >>> the official business of Winbond shall be deemed as neither given > >>> nor > >> endorsed by Winbond. > >>> > >>> The privileged confidential information contained in this email is > >>> intended for use only by the addressees as indicated by the original > >>> sender of this email. If you are not the addressee indicated in this > >>> email or are not responsible for delivery of the email to such a > >>> person, please kindly reply to the sender indicating this fact and > >>> delete all copies of it from your computer and network server > >>> immediately. Your cooperation is highly appreciated. It is advised > >>> that any unauthorized use of confidential information of Winbond is > >>> strictly prohibited; and any information in this email irrelevant to > >>> the official business of Winbond shall be deemed as neither given > >>> nor > >> endorsed by Winbond. > >>> > >>> The privileged confidential information contained in this email is > >>> intended for use only by the addressees as indicated by the original > >>> sender of this email. If you are not the addressee indicated in this > >>> email or are not responsible for delivery of the email to such a > >>> person, please kindly reply to the sender indicating this fact and > >>> delete all copies of it from your computer and network server > >>> immediately. Your cooperation is highly appreciated. It is advised > >>> that any unauthorized use of confidential information of Winbond is > >>> strictly prohibited; and any information in this email irrelevant to > >>> the official business of Winbond shall be deemed as neither given > >>> nor > >> endorsed by Winbond. > >> > >> The privileged confidential information contained in this email is > >> intended for use only by the addressees as indicated by the original > >> sender of this email. If you are not the addressee indicated in this > >> email or are not responsible for delivery of the email to such a > >> person, please kindly reply to the sender indicating this fact and > >> delete all copies of it from your computer and network server > >> immediately. Your cooperation is highly appreciated. It is advised > >> that any unauthorized use of confidential information of Winbond is > >> strictly prohibited; and any information in this email irrelevant to > >> the official business of Winbond shall be deemed as neither given nor > endorsed by Winbond. > > > > The privileged confidential information contained in this email is > intended for use only by the addressees as indicated by the original sender > of this email. If you are not the addressee indicated in this email or are > not responsible for delivery of the email to such a person, please kindly > reply to the sender indicating this fact and delete all copies of it from > your computer and network server immediately. Your cooperation is highly > appreciated. It is advised that any unauthorized use of confidential > information of Winbond is strictly prohibited; and any information in this > email irrelevant to the official business of Winbond shall be deemed as > neither given nor endorsed by Winbond. >
