Sure no problem. I understand that for some people this type of issue can be frustrating. HBase being distributed by nature there's a lot more going on than just setting up one mysql instance and it requires skills/knowledge more attuned to devops rather than just devs. Hopefully HBase will get better at identifying those issues for the user. Right now it's just being stubborn and looks the other way :)
J-D On Fri, Dec 2, 2011 at 11:01 PM, Vamshi Krishna <[email protected]> wrote: > Ohh Thank you very much J-D, its working fine. I am very glad for getting > your continuous help for hbase successfull setup. Soon i want to start a > blog in which i would post clear setup of hbase and problems along with > corresponding solutions i faced during setting up all these days. > > > > > On Fri, Dec 2, 2011 at 11:38 PM, Jean-Daniel Cryans > <[email protected]>wrote: > >> > i could not find 10.0.1.54:60000 any where on the console, rather i >> found> 127.0.1.1:60000 in some places. >> >> Remove the 127.0.1.1 line in /etc/hosts, do it for all your ubuntu >> machines. >> >> J-D >> On Fri, Dec 2, 2011 at 6:34 AM, Vamshi Krishna <[email protected]> >> wrote: >> > I disabled the firewall in all the machines.Then i started >> > hbase(bin/start-hbase.sh), after 2-3 minutesi stopped hbase on master >> node >> > (bin/stop-hbase.sh). Even then HRegionserver daemon is running on region >> > server machine. So i checked regionserver machines log, the full log is >> > here >> > >> > http://pastebin.com/FXsNLMD9 >> > >> > I found that regionserver machine is not able to communicate with master >> > though it is attempting to connect to master (hbase-master:60000)for >> every >> > one minute or so. >> > >> > so i tried to do the following as you have told. >> > >> > hduser@vamshi-laptop:~$ telnet hbase-master 60000 >> > Trying 10.0.1.54... >> > telnet: Unable to connect to remote host: Connection refused >> > >> > hduser@vamshi-laptop:~$ telnet hbase-master:60000 >> > telnet: could not resolve hbase-master:60000/telnet: Name or service not >> > known >> > >> > i made hbase-master to bind to 10.0.1.54 in /etc/hosts file, so i >> tried >> > even following also. >> > >> > hduser@vamshi-laptop:~$ telnet 10.0.1.54 60000 >> > Trying 10.0.1.54... >> > telnet: Unable to connect to remote host: Connection refused >> > hduser@vamshi-laptop:~$ >> > >> > So, i thought whether hbase-master is bound to port 60000 on the master >> > node or not, so while hbase master is running, i checked by typing >> > netstat -anp | grep 60000 >> > >> > i could not find 10.0.1.54:60000 any where on the console, rather i >> found >> > 127.0.1.1:60000 in some places. >> > >> > Is this the correct way of representing /etc/hosts file on master node? >> > >> > 127.0.0.1 localhost >> > 127.0.1.1 vamshikrishna-laptop >> > 10.0.1.54 hbase-master hadoop-namenode vamshikrishna-desktop >> > 10.0.1.55 hbase-regionserver1 hadoop-datanode1 >> > 10.0.1.97 hbase-regionserver2 hadoop-datanode2 arun-desktop >> > # The following lines are desirable for IPv6 capable hosts >> > ::1 localhost ip6-localhost ip6-loopback >> > fe00::0 ip6-localnet >> > ff00::0 ip6-mcastprefix >> > ff02::1 ip6-allnodes >> > ff02::2 ip6-allrouters >> > ff02::3 ip6-allhosts >> > >> > Even i checked by typing the command on both master node and region >> server >> > machine >> > $ nmap -p 60000 hbase-master >> > then i got some thing on console like >> > 60000/tcp closed >> > >> > >> > Please help me, i dont understand what went wrong in my set up. >> > >> > >> > On Thu, Dec 1, 2011 at 11:28 PM, Jean-Daniel Cryans <[email protected] >> >wrote: >> > >> >> So since I don't see the rest of the log I'll have to assume that the >> >> region server was never able to connect to the master. Connection >> >> refused could be a firewall, start the master and then try to telnet >> >> from the other machines to master:60000. >> >> >> >> J-D >> >> >> >> On Thu, Dec 1, 2011 at 6:45 AM, Vamshi Krishna <[email protected]> >> >> wrote: >> >> > I found in the logs of region server machines, i found this error (on >> >> both >> >> > regionserver machines) >> >> > >> >> > 2011-11-30 14:43:42,447 INFO org.apache.hadoop.ipc.HbaseRPC: Server at >> >> > hbase-master/10.0.1.54:60020 could not be reached after 1 tries, >> giving >> >> up. >> >> > *2011-11-30 14:44:37,762* WARN >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to connect >> to >> >> > master. Retrying. Error was: >> >> > java.net.ConnectException: Connection refused >> >> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> >> > at >> >> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) >> >> > at >> >> > >> >> >> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) >> >> > at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883) >> >> > at >> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750) >> >> > at >> >> > org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257) >> >> > at $Proxy5.getProtocolVersion(Unknown Source) >> >> > at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419) >> >> > at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393) >> >> > at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444) >> >> > at >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1462) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1515) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.tryReportForDuty(HRegionServer.java:1499) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:572) >> >> > at java.lang.Thread.run(Thread.java:662) >> >> > 2011-11-30 14:44:40,768 INFO >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Attempting >> connect to >> >> > Master server at hbase-master:60000 >> >> > *2011-11-30 14:45:40,847* WARN >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to connect >> to >> >> > master. Retrying. Error was: >> >> > java.net.ConnectException: Connection refused >> >> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> >> > at >> >> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) >> >> > at >> >> > >> >> >> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) >> >> > at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883) >> >> > at >> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750) >> >> > at >> >> > org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257) >> >> > at $Proxy5.getProtocolVersion(Unknown Source) >> >> > at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419) >> >> > at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393) >> >> > at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444) >> >> > at >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1462) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1515) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.tryReportForDuty(HRegionServer.java:1499) >> >> > at >> >> > >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:572) >> >> > at java.lang.Thread.run(Thread.java:662) >> >> > >> >> > >> >> > and the same error is observed in the whole log repeatedly. After >> seeing >> >> it >> >> > what i understand is that some how master started HRegionServer >> daemons >> >> on >> >> > the machines but from then onwards the RegionServer machines are not >> able >> >> > to communicate with master. If we observe it is trying to communicate >> >> with >> >> > master for evry one minute. >> >> > >> >> > But i am not understanding where to check and modify the things.. >> please >> >> > help. i feel all connections are OK. >> >> > >> >> > On Thu, Dec 1, 2011 at 12:28 AM, Jean-Daniel Cryans < >> [email protected] >> >> >wrote: >> >> > >> >> >> stop-hbase.sh only tells the master to stop, which in turn will tell >> >> >> the region servers to stop. If they are still running, it might be >> >> >> because of an error. Look at their logs to figure what's going on. >> >> >> >> >> >> J-D >> >> >> >> >> >> On Tue, Nov 29, 2011 at 10:46 PM, Vamshi Krishna < >> [email protected]> >> >> >> wrote: >> >> >> > hey soryy for posting multiple times. >> >> >> > J-D, As you said, i refered to my regionserver log, there i found >> >> >> > Could not resolve the DNS name of >> vamshikrishna-desktop >> >> >> > so i added an alias ' vamshikrishna-desktop ' to its corresponding >> IP >> >> >> > address in /etc/hosts. So, from then master is able to run >> >> HRegionServer >> >> >> > daemon in the regionserver machines also. >> >> >> > >> >> >> > But the ONLY problem now is when i stop hbase on my master node by >> >> >> running >> >> >> > bin/stop-hbase.sh, all hbase daemons are stopping on matser node >> but >> >> NOT >> >> >> on >> >> >> > regionserver nodes.The HRegionServer daemon is still running on the >> >> other >> >> >> > regionserver machines. >> >> >> > I think the HRegionServer daemons on all the regionserver machines >> >> should >> >> >> > also get stopped, when we execute bin/sto-hbase.sh on master node, >> >> right? >> >> >> > please help.. >> >> >> > >> >> >> > On Wed, Nov 30, 2011 at 12:38 AM, Jean-Daniel Cryans < >> >> >> [email protected]>wrote: >> >> >> > >> >> >> >> You posted this in two threads, please refrain from doing this in >> the >> >> >> >> future. >> >> >> >> >> >> >> >> From what I can read in there, it tried to get the master address >> >> >> >> that's supposed to be in zookeeper but it failed because it was >> >> >> >> missing and then died. The way it's handled is a bit ugly but the >> >> >> >> effect is the same, take a look at the master log to see what's >> going >> >> >> >> on. >> >> >> >> >> >> >> >> J-D >> >> >> >> >> >> >> >> On Mon, Nov 28, 2011 at 10:33 PM, Vamshi Krishna < >> >> [email protected]> >> >> >> >> wrote: >> >> >> >> > Hi Lars, >> >> >> >> > i am not using cygwin, i am using 3 ubuntu-10.04 machines. >> >> >> >> > Finally that problem i mentioned got resolved i.e now i can see >> the >> >> >> >> > following after i run bin/start-hbase.sh on my master machine, >> >> >> >> > >> >> >> >> > hbase-master: starting zookeeper, logging to >> >> >> >> > >> >> >> >> >> >> >> >> >> >> /home/hduser/Documents/HBASE_SOFTWARE/hbase-0.90.4/bin/../logs/hbase-hduser-zookeeper-vamshikrishna-desktop.out >> >> >> >> > starting master, logging to >> >> >> >> > >> >> >> >> >> >> >> >> >> >> /home/hduser/Documents/HBASE_SOFTWARE/hbase-0.90.4/logs/hbase-hduser-master-vamshikrishna-desktop.out >> >> >> >> > hbase-regionserver2: starting regionserver, logging to >> >> >> >> > >> >> >> >> >> >> >> >> >> >> /home/hduser/Documents/HBASE_SOFTWARE/hbase-0.90.4/bin/../logs/hbase-hduser-regionserver-arun-desktop.out >> >> >> >> > hbase-regionserver1: starting regionserver, logging to >> >> >> >> > >> >> >> >> >> >> >> >> >> >> /home/hduser/Documents/HBASE_SOFTWARE/hbase-0.90.4/bin/../logs/hbase-hduser-regionserver-vamshi-laptop.out >> >> >> >> > >> >> >> >> > hbase-master: starting regionserver, logging to >> >> >> >> > >> >> >> >> >> >> >> >> >> >> /home/hduser/Documents/HBASE_SOFTWARE/hbase-0.90.4/bin/../logs/hbase-hduser-regionserver-vamshikrishna-desktop.out >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > But still , if i do jps on both region server machines, >> >> HRegionServer >> >> >> >> > daemon is not seen.And i checked the logs on both the >> regionservere >> >> >> >> > machines, i.e >> >> >> >> > >> >> >> >> > >> >> >> >> >> >> >> >> >> >> /home/hduser/Documents/HBASE_SOFTWARE/hbase-0.90.4/bin/../logs/hbase-hduser-regionserver-vamshi-laptop.log >> >> >> >> > ---(hbase-regionserver1) >> >> >> >> > and >> >> >> >> > >> >> >> >> >> >> >> >> >> >> /home/hduser/Documents/HBASE_SOFTWARE/hbase-0.90.4/bin/../logs/hbase-hduser-regionserver-arun-desktop.log >> >> >> >> > ---(hbase-regionserver2) >> >> >> >> > >> >> >> >> > i found the same in both .it is like this, (here >> >> >> vamshikrishna-desktop >> >> >> >> is >> >> >> >> > the master and namenode machine) >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > 2011-11-29 11:24:36,714 ERROR >> >> org.apache.hadoop.hbase.HServerAddress: >> >> >> >> Could >> >> >> >> > not resolve the DNS name of vamshikrishna-desktop >> >> >> >> > 2011-11-29 11:24:36,716 FATAL >> >> >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING >> region >> >> >> >> server >> >> >> >> > serverName=vamshi-laptop,60020,1322546076276, load=(requests=0, >> >> >> >> regions=0, >> >> >> >> > usedHeap=0, maxHeap=0): Unhandled exception: hostname can't be >> null >> >> >> >> > java.lang.IllegalArgumentException: hostname can't be null >> >> >> >> > at >> java.net.InetSocketAddress.<init>(InetSocketAddress.java:121) >> >> >> >> > at >> >> >> >> > >> >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.HServerAddress.getResolvedAddress(HServerAddress.java:108) >> >> >> >> > at >> >> >> >> >> org.apache.hadoop.hbase.HServerAddress.<init>(HServerAddress.java:64) >> >> >> >> > at >> >> >> >> > >> >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.MasterAddressTracker.getMasterAddress(MasterAddressTracker.java:63) >> >> >> >> > at >> >> >> >> > >> >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.getMasterAddress(HRegionServer.java:1484) >> >> >> >> > at >> >> >> >> > >> >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1457) >> >> >> >> > at >> >> >> >> > >> >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1515) >> >> >> >> > at >> >> >> >> > >> >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.tryReportForDuty(HRegionServer.java:1499) >> >> >> >> > at >> >> >> >> > >> >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:572) >> >> >> >> > at java.lang.Thread.run(Thread.java:662) >> >> >> >> > 2011-11-29 11:24:36,719 INFO >> >> >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: >> >> Unhandled >> >> >> >> > exception: hostname can't be null >> >> >> >> > 2011-11-29 11:24:36,719 INFO org.apache.hadoop.ipc.HBaseServer: >> >> >> Stopping >> >> >> >> > server on 60020 >> >> >> >> > 2011-11-29 11:24:36,749 INFO >> >> >> >> > org.apache.hadoop.hbase.regionserver.StoreFile: Allocating >> >> >> LruBlockCache >> >> >> >> > with maximum size 199.4m >> >> >> >> > 2011-11-29 11:24:36,775 INFO >> >> >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: aborting >> server >> >> >> at: >> >> >> >> > vamshi-laptop,60020,1322546076276 >> >> >> >> > 2011-11-29 11:24:36,775 DEBUG >> >> >> >> > org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog >> >> >> tracker >> >> >> >> > org.apache.hadoop.hbase.catalog.CatalogTracker@da2cef >> >> >> >> > 2011-11-29 11:24:36,776 INFO >> >> >> org.apache.hadoop.hbase.regionserver.Leases: >> >> >> >> > regionserver60020 closing leases >> >> >> >> > 2011-11-29 11:24:36,776 INFO >> >> >> org.apache.hadoop.hbase.regionserver.Leases: >> >> >> >> > regionserver60020 closed leases >> >> >> >> > 2011-11-29 11:24:36,776 INFO >> >> >> >> > >> >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: >> >> >> >> > Closed zookeeper sessionid=0x133ede31f3c0004 >> >> >> >> > 2011-11-29 11:24:36,781 INFO org.apache.zookeeper.ClientCnxn: >> >> >> EventThread >> >> >> >> > shut down >> >> >> >> > 2011-11-29 11:24:36,781 INFO org.apache.zookeeper.ZooKeeper: >> >> Session: >> >> >> >> > 0x133ede31f3c0004 closed >> >> >> >> > 2011-11-29 11:24:36,789 INFO org.apache.zookeeper.ClientCnxn: >> >> >> EventThread >> >> >> >> > shut down >> >> >> >> > 2011-11-29 11:24:36,789 INFO org.apache.zookeeper.ZooKeeper: >> >> Session: >> >> >> >> > 0x133ede31f3c0005 closed >> >> >> >> > 2011-11-29 11:24:36,792 INFO >> >> >> >> > org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook >> >> >> >> starting; >> >> >> >> > hbase.shutdown.hook=true; >> fsShutdownHook=Thread[Thread-14,5,main] >> >> >> >> > 2011-11-29 11:24:36,792 INFO >> >> >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: >> >> Shutdown >> >> >> >> hook >> >> >> >> > 2011-11-29 11:24:36,792 INFO >> >> >> >> > org.apache.hadoop.hbase.regionserver.ShutdownHook: Starting fs >> >> >> shutdown >> >> >> >> > hook thread. >> >> >> >> > 2011-11-29 11:24:36,792 INFO >> >> >> >> > org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook >> >> >> >> finished. >> >> >> >> > >> >> >> >> > what is the cause for problem here, please help. >> >> >> >> > >> >> >> >> > On Mon, Nov 28, 2011 at 7:28 PM, Lars George < >> >> [email protected]> >> >> >> >> wrote: >> >> >> >> > >> >> >> >> >> Hi, >> >> >> >> >> >> >> >> >> >> Did you add the list of servers to the regionservers file in >> the >> >> >> >> >> $HBASE_HOME/conf/ dir? Are you using Cygwin? Or what else is >> your >> >> >> >> >> environment? >> >> >> >> >> >> >> >> >> >> Lars >> >> >> >> >> >> >> >> >> >> On Nov 26, 2011, at 7:37 AM, Vamshi Krishna wrote: >> >> >> >> >> >> >> >> >> >> > Hi i am running hbase on 3 machines, on one node master and >> >> >> >> regionserver, >> >> >> >> >> > on other two nodes regionservers. i ran bin/start-hbase.sh on >> >> >> master, >> >> >> >> all >> >> >> >> >> > the hbase daemons running on master node(HMaster, >> HQuorumpeer, >> >> >> >> >> > HRegionServer), But on the other regionserver machines i >> could >> >> not >> >> >> >> find >> >> >> >> >> any >> >> >> >> >> > hbase daemons running. According to my knowledge the >> >> HRegionserver >> >> >> >> >> daemons >> >> >> >> >> > should run on other machines automatically when i run >> >> >> >> bin/start-hbase.sh >> >> >> >> >> on >> >> >> >> >> > master node, right? But its not happening so. >> >> >> >> >> > More over its printing errors on my master node when i run >> >> >> >> >> > bin/start-hbase.sh >> >> >> >> >> > >> >> >> >> >> > hbase-regionserver1: bash: line 0: cd: {HBASE_HOME}/bin/..: >> No >> >> such >> >> >> >> file >> >> >> >> >> or >> >> >> >> >> > directory >> >> >> >> >> > hbase-regionserver1: bash: {HBASE_HOME}/bin/hbase-daemon.sh: >> No >> >> >> such >> >> >> >> file >> >> >> >> >> > or directory >> >> >> >> >> > hbase-regionserver2: bash: line 0: cd: {HBASE_HOME}/bin/..: >> No >> >> such >> >> >> >> file >> >> >> >> >> or >> >> >> >> >> > directory >> >> >> >> >> > hbase-regionserver2: bash: {HBASE_HOME}/bin/hbase-daemon.sh: >> No >> >> >> such >> >> >> >> file >> >> >> >> >> > or directory , >> >> >> >> >> > >> >> >> >> >> > but i could find out {HBASE_HOME}/bin/hbase-daemon.sh on both >> >> >> >> machines. >> >> >> >> >> > Infact the path of the {HBASE_HOME} folder on each of the >> >> >> respective >> >> >> >> >> > machines machines is same. i.e >> >> >> >> >> > /home/hduser/Documents/HBASE_SOFTWRAE/hbase-0.90.4 >> >> >> >> >> > >> >> >> >> >> > please can anybody help? >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > -- >> >> >> >> >> > *Regards* >> >> >> >> >> > * >> >> >> >> >> > Vamshi Krishna >> >> >> >> >> > * >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> >> >> >> > >> >> >> >> > -- >> >> >> >> > *Regards* >> >> >> >> > * >> >> >> >> > Vamshi Krishna >> >> >> >> > * >> >> >> >> > >> >> >> >> >> >> >> > >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > *Regards* >> >> >> > * >> >> >> > Vamshi Krishna >> >> >> > * >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > *Regards* >> >> > * >> >> > Vamshi Krishna >> >> > * >> >> >> > >> > >> > >> > -- >> > *Regards* >> > * >> > Vamshi Krishna >> > * >> > > > > -- > *Regards* > * > Vamshi Krishna > *
