> > telnet 10.41.55.18 7199 # doesnt connect > Trying 10.41.55.18... > telnet: connect to address 10.41.55.18: Connection refused > > Could this be the reason? >
I don't think so. But I have not a clue on what's going on... Did you tried this ? - Try increasing the RF / Strategy in the stress tool > -schema option, pass a file if it is easier, with a custom keyspace, defining NTS and a RF of 2 or 3. You don't want to use RF = 1 nor SimpleStrategy in Multi DC under real conditions anyway, so your test will be more relevant. You can paste us the command + schema + output / error (if any) > - Try with an other consistency level (not LOCAL_*, as mentioned here : > http://stackoverflow.com/questions/32055251/not-enough-replica-available-for-query-at-consistency-local-one-1-required-but > ) > Shouldn't affect you, yet, it is easy enough to test, so it might be worth it. Maybe the first thing I would try, just out of curiosity. C*heers, ----------------- Alain The Last Pickle http://www.thelastpickle.com 2016-01-23 11:13 GMT+01:00 Bhuvan Rawal <bhu1ra...@gmail.com>: > Alright, I uninstalled the DSE from all the nodes in the cluster and > reinstalled them again from scratch. Ran nodetool status and output is U/N > for all nodes again. > > Tried running cassandra-stress and it doesnt work. > > I can see in the system.peer table that token ranges has been distributed, > further, other columns seem normal too. > cqlsh> select peer, data_center, host_id, rack, release_version, > rpc_address, workload from system.peers; > > peer | data_center | host_id | rack | release_version | rpc_address | > workload > -------------+-------------+--------------------------------------+-------+-----------------+-------------+----------- > 10.41.55.23 > | Cassandra | b48f7683-3f42-4c49-9ce8-694112a21d5d | rack1 | 2.1.12.1046 | > 10.41.55.23 | Cassandra 10.41.55.19 | Analytics | > f271657f-52e2-4f43-8bbe-03a979852679 | rack1 | 2.1.12.1046 | 10.41.55.19 | > Analytics 10.41.55.20 | Analytics | e4f484c9-8e7f-48b7-bce8-2a91d70be4d1 > | rack1 | 2.1.12.1046 | 10.41.55.20 | Analytics 10.41.55.18 | Analytics | > 813c2610-0656-4714-9ec6-99d51c72da92 | rack1 | 2.1.12.1046 | 10.41.55.18 | > Analytics 10.41.55.15 | Cassandra | 2b09a481-3e08-45e5-a2d0-4c3d5d611ef6 > | rack1 | 2.1.12.1046 | 10.41.55.15 | Cassandra 10.41.55.22 | Cassandra | > d16cc704-4453-4dc0-880e-4af6fbd0dc48 | rack1 | 2.1.12.1046 | 10.41.55.22 | > Cassandra 10.41.55.17 | Analytics | dfbb4f0e-2ffa-4694-811c-1167662ad537 > | rack1 | 2.1.12.1046 | 10.41.55.17 | Analytics > > This seems to be fine too: > $ nodetool describecluster Cluster Information: Name: POC Cluster Snitch: > org.apache.cassandra.locator.DynamicEndpointSnitch Partitioner: > org.apache.cassandra.dht.Murmur3Partitioner Schema versions: > 07d395d2-e5b8-3e67-addd-3f8d1e8d7803: [10.41.55.15, 10.41.55.17, > 10.41.55.19, 10.41.55.18, 10.41.55.21, 10.41.55.20, 10.41.55.23, > 10.41.55.22] > > I tried to telnet to a node of other cluster which is not a seed, here are > the observations : > Telnet to ports 9042, 7000,9160 seem to work fine > telnet 10.41.55.18 9042 # connects > telnet 10.41.55.18 7000 # connects > telnet 10.41.55.18 9160 # connects > But telnet to JMX port is not working > telnet 10.41.55.18 7199 # doesnt connect > Trying 10.41.55.18... > telnet: connect to address 10.41.55.18: Connection refused > > Could this be the reason? > > On Sat, Jan 23, 2016 at 7:33 AM, Alain RODRIGUEZ <arodr...@gmail.com> > wrote: > >> Should I use nodetool repair utility >>> >> >> That wouldn't help, this an anti-entropy mechanism (see >> https://docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsRepair.html#toolsRepair__description_unique_27 >> ). >> >> It is something really important too often left aside. >> >> Yet, your issue here is not about consistency. The client can't find any >> node in charge of the read / written tokens in the ring. This depends on >> the topology, the replication factor and your network mainly. I think there >> is something wrong in your setup. I would try this: >> >> - Make sure connection / port are ok >> - Try increasing the RF / Strategy in the stress tool >> - Try with an other consistency level (not LOCAL_*, as mentioned here : >> http://stackoverflow.com/questions/32055251/not-enough-replica-available-for-query-at-consistency-local-one-1-required-but >> ) >> >> Good luck, >> >> ----------------- >> Alain >> >> The Last Pickle >> http://www.thelastpickle.com >> >> 2016-01-22 23:02 GMT+01:00 Bhuvan Rawal <bhu1ra...@gmail.com>: >> >>> Getting same exception again. Should I use nodetool repair utility? >>> >>> On Sat, Jan 23, 2016 at 3:10 AM, Sebastian Estevez < >>> sebastian.este...@datastax.com> wrote: >>> >>>> https://github.com/brianmhess/cassandra-loader >>>> >>>> All the best, >>>> >>>> >>>> [image: datastax_logo.png] <http://www.datastax.com/> >>>> >>>> Sebastián Estévez >>>> >>>> Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com >>>> >>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax> [image: >>>> facebook.png] <https://www.facebook.com/datastax> [image: twitter.png] >>>> <https://twitter.com/datastax> [image: g+.png] >>>> <https://plus.google.com/+Datastax/about> >>>> <http://feeds.feedburner.com/datastax> >>>> <http://goog_410786983> >>>> >>>> >>>> <http://www.datastax.com/gartner-magic-quadrant-odbms> >>>> >>>> DataStax is the fastest, most scalable distributed database >>>> technology, delivering Apache Cassandra to the world’s most innovative >>>> enterprises. Datastax is built to be agile, always-on, and predictably >>>> scalable to any size. With more than 500 customers in 45 countries, >>>> DataStax >>>> is the database technology and transactional backbone of choice for the >>>> worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay. >>>> >>>> On Fri, Jan 22, 2016 at 4:37 PM, Bhuvan Rawal <bhu1ra...@gmail.com> >>>> wrote: >>>> >>>>> Yes im specifying -node parameter to stress, otherwise it throws >>>>> network connection failed. >>>>> >>>>> Can you point me to a sample java application to test pushing data >>>>> from external server? Let's see if that works >>>>> >>>>> On Sat, Jan 23, 2016 at 2:55 AM, Sebastian Estevez < >>>>> sebastian.este...@datastax.com> wrote: >>>>> >>>>>> when i opened my cassandra-rackdc.properties i saw that DC names were >>>>>>> DC1 & DC2, rack name was RAC1 . Please note that this is the default >>>>>>> configuration, I have not modified any file. >>>>>> >>>>>> >>>>>> cassandra-rackdc.properties is only respected based on your snitch >>>>>> <https://docs.datastax.com/en/cassandra/2.1/cassandra/architecture/architectureSnitchesAbout_c.html> >>>>>> . >>>>>> >>>>>> $ cqlsh >>>>>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1': >>>>>>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: >>>>>>> Connection refused")}) >>>>>>> whereas >>>>>>> $ cqlsh <ip> >>>>>>> works fine >>>>>>> is that the reason why the cassandra-stress is not able to >>>>>>> communicate with other replicas? >>>>>> >>>>>> >>>>>> Are you providing the -node parameter to stress >>>>>> <http://docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsCStress_t.html> >>>>>> ? >>>>>> >>>>>> >>>>>> >>>>>> All the best, >>>>>> >>>>>> >>>>>> [image: datastax_logo.png] <http://www.datastax.com/> >>>>>> >>>>>> Sebastián Estévez >>>>>> >>>>>> Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com >>>>>> >>>>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax> [image: >>>>>> facebook.png] <https://www.facebook.com/datastax> [image: >>>>>> twitter.png] <https://twitter.com/datastax> [image: g+.png] >>>>>> <https://plus.google.com/+Datastax/about> >>>>>> <http://feeds.feedburner.com/datastax> >>>>>> <http://goog_410786983> >>>>>> >>>>>> >>>>>> <http://www.datastax.com/gartner-magic-quadrant-odbms> >>>>>> >>>>>> DataStax is the fastest, most scalable distributed database >>>>>> technology, delivering Apache Cassandra to the world’s most innovative >>>>>> enterprises. Datastax is built to be agile, always-on, and predictably >>>>>> scalable to any size. With more than 500 customers in 45 countries, >>>>>> DataStax >>>>>> is the database technology and transactional backbone of choice for the >>>>>> worlds most innovative companies such as Netflix, Adobe, Intuit, and >>>>>> eBay. >>>>>> >>>>>> On Fri, Jan 22, 2016 at 4:07 PM, Bhuvan Rawal <bhu1ra...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> I had a look at the jira below: >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-7905 >>>>>>> >>>>>>> when i opened my cassandra-rackdc.properties i saw that DC names >>>>>>> were DC1 & DC2, rack name was RAC1 . Please note that this is the >>>>>>> default >>>>>>> configuration, I have not modified any file. >>>>>>> >>>>>>> There is another point of concern here which might be relevant to >>>>>>> previous one as well, im not able to login to cqlsh directly, i.e. I >>>>>>> have >>>>>>> to specify ip as well even when im logged in to that machine. >>>>>>> >>>>>>> $ cqlsh >>>>>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1': >>>>>>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: >>>>>>> Connection refused")}) >>>>>>> >>>>>>> whereas >>>>>>> $ cqlsh <ip> >>>>>>> works fine >>>>>>> >>>>>>> is that the reason why the cassandra-stress is not able to >>>>>>> communicate with other replicas? >>>>>>> >>>>>>> On Sat, Jan 23, 2016 at 1:37 AM, Sebastian Estevez < >>>>>>> sebastian.este...@datastax.com> wrote: >>>>>>> >>>>>>>> Sorry I missed that. >>>>>>>> >>>>>>>> Both your nodetool status and keyspace replication settings say >>>>>>>> Cassandra and Analytics for the DC names. I'm not sure where you're >>>>>>>> seeing >>>>>>>> DC1, DC2, etc. and why you suspect that is the problem. >>>>>>>> >>>>>>>> All the best, >>>>>>>> >>>>>>>> >>>>>>>> [image: datastax_logo.png] <http://www.datastax.com/> >>>>>>>> >>>>>>>> Sebastián Estévez >>>>>>>> >>>>>>>> Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com >>>>>>>> >>>>>>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax> >>>>>>>> [image: >>>>>>>> facebook.png] <https://www.facebook.com/datastax> [image: >>>>>>>> twitter.png] <https://twitter.com/datastax> [image: g+.png] >>>>>>>> <https://plus.google.com/+Datastax/about> >>>>>>>> <http://feeds.feedburner.com/datastax> >>>>>>>> <http://goog_410786983> >>>>>>>> >>>>>>>> >>>>>>>> <http://www.datastax.com/gartner-magic-quadrant-odbms> >>>>>>>> >>>>>>>> DataStax is the fastest, most scalable distributed database >>>>>>>> technology, delivering Apache Cassandra to the world’s most innovative >>>>>>>> enterprises. Datastax is built to be agile, always-on, and predictably >>>>>>>> scalable to any size. With more than 500 customers in 45 countries, >>>>>>>> DataStax >>>>>>>> is the database technology and transactional backbone of choice for the >>>>>>>> worlds most innovative companies such as Netflix, Adobe, Intuit, and >>>>>>>> eBay. >>>>>>>> >>>>>>>> On Fri, Jan 22, 2016 at 1:45 PM, Bhuvan Rawal <bhu1ra...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Sebastian, >>>>>>>>> >>>>>>>>> I had attached nodetool status output in previous mail, pasting it >>>>>>>>> again : >>>>>>>>> >>>>>>>>> $ nodetool status Datacenter: Analytics ===================== >>>>>>>>> Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address >>>>>>>>> Load Tokens Owns Host ID Rack UN 10.41.55.17 428.5 KB 256 ? >>>>>>>>> 39d6d585-e641-4046-9d0b-797356597b5e rack1 UN 10.41.55.19 404.44 KB >>>>>>>>> 256 ? >>>>>>>>> 69edf930-efd9-4d74-a798-f3d4ac02e516 rack1 UN 10.41.55.18 423.21 KB >>>>>>>>> 256 ? >>>>>>>>> b74bab13-09b2-4760-bce9-c8ef05e50f6d rack1 UN 10.41.55.20 683.23 KB >>>>>>>>> 256 ? >>>>>>>>> fb5c4fed-6e1e-4ea8-838d-358106906830 rack1 Datacenter: Cassandra >>>>>>>>> ===================== Status=Up/Down |/ >>>>>>>>> State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns >>>>>>>>> Host ID Rack UN 10.41.55.15 209.4 KB 256 ? >>>>>>>>> ffc3b9a0-5d5c-4a3d-a99e-49d255731278 rack1 UN 10.41.55.21 227.44 KB >>>>>>>>> 256 ? >>>>>>>>> c68deba4-b9a2-43fc-bb13-6af74c88c210 rack1 UN 10.41.55.23 222.71 KB >>>>>>>>> 256 ? >>>>>>>>> 8229aa87-af00-48fa-ad6b-3066d3dc0e58 rack1 UN 10.41.55.22 218.72 KB >>>>>>>>> 256 ? >>>>>>>>> c7ba84fd-7992-41de-8c88-11574a72db99 rack1 >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Bhuvan Rawal >>>>>>>>> >>>>>>>>> On Sat, Jan 23, 2016 at 12:11 AM, Sebastian Estevez < >>>>>>>>> sebastian.este...@datastax.com> wrote: >>>>>>>>> >>>>>>>>>> The output of `nodetool status` would help us diagnose. >>>>>>>>>> >>>>>>>>>> All the best, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [image: datastax_logo.png] <http://www.datastax.com/> >>>>>>>>>> >>>>>>>>>> Sebastián Estévez >>>>>>>>>> >>>>>>>>>> Solutions Architect | 954 905 8615 | >>>>>>>>>> sebastian.este...@datastax.com >>>>>>>>>> >>>>>>>>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax> >>>>>>>>>> [image: >>>>>>>>>> facebook.png] <https://www.facebook.com/datastax> [image: >>>>>>>>>> twitter.png] <https://twitter.com/datastax> [image: g+.png] >>>>>>>>>> <https://plus.google.com/+Datastax/about> >>>>>>>>>> <http://feeds.feedburner.com/datastax> >>>>>>>>>> <http://goog_410786983> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> <http://www.datastax.com/gartner-magic-quadrant-odbms> >>>>>>>>>> >>>>>>>>>> DataStax is the fastest, most scalable distributed database >>>>>>>>>> technology, delivering Apache Cassandra to the world’s most >>>>>>>>>> innovative >>>>>>>>>> enterprises. Datastax is built to be agile, always-on, and >>>>>>>>>> predictably >>>>>>>>>> scalable to any size. With more than 500 customers in 45 countries, >>>>>>>>>> DataStax >>>>>>>>>> is the database technology and transactional backbone of choice for >>>>>>>>>> the >>>>>>>>>> worlds most innovative companies such as Netflix, Adobe, Intuit, and >>>>>>>>>> eBay. >>>>>>>>>> >>>>>>>>>> On Fri, Jan 22, 2016 at 1:39 PM, Bhuvan Rawal < >>>>>>>>>> bhu1ra...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks for the response Alain, >>>>>>>>>>> >>>>>>>>>>> cqlsh> create keyspace mykeyspace WITH replication = >>>>>>>>>>> {'class':'NetworkTopologyStrategy', 'Analytics':2, 'Cassandra':3} >>>>>>>>>>> cqlsh> use mykeyspace; >>>>>>>>>>> cqlsh:mykeyspace>create table mytable (id int primary key, name >>>>>>>>>>> text, address text, phone text); >>>>>>>>>>> cqlsh:mykeyspace> insert into mytable (id, name, address, phone) >>>>>>>>>>> values (1, 'Kiyu','Texas', '555-1212'); # and other similar >>>>>>>>>>> statement >>>>>>>>>>> I then issued the below command from every node and found >>>>>>>>>>> consistent results. >>>>>>>>>>> cqlsh:mykeyspace> select * from mytable; >>>>>>>>>>> >>>>>>>>>>> // Then i repeated the above steps for NetworkTopologyStrategy >>>>>>>>>>> and found same results >>>>>>>>>>> >>>>>>>>>>> I ran basic cassandra stress >>>>>>>>>>> seed1 - seed of datacenter 1 >>>>>>>>>>> $ cassandra-stress write n=50000 -rate threads=4 -node >>>>>>>>>>> any_random_ip >>>>>>>>>>> $ cassandra-stress write n=50000 -rate threads=4 -node seed1 >>>>>>>>>>> $ cassandra-stress write n=50000 -rate threads=4 -node >>>>>>>>>>> seed1,seed2 >>>>>>>>>>> $ cassandra-stress write n=50000 -rate threads=4 -node >>>>>>>>>>> all_8_ip_comma_seperated >>>>>>>>>>> $ cassandra-stress write n=1000000 cl=one -mode native cql3 >>>>>>>>>>> -schema keyspace="keyspace1" -pop seq=1..1000000 -node >>>>>>>>>>> ip1,ip2,ip3,ip4 >>>>>>>>>>> >>>>>>>>>>> All of them threw the exception >>>>>>>>>>> *com.datastax.driver.core.exceptions.UnavailableException: Not >>>>>>>>>>> enough replica available for query at consistency LOCAL_ONE (1 >>>>>>>>>>> required but >>>>>>>>>>> only 0 alive)* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I have a feeling that the issue is with datacenter name for some >>>>>>>>>>> reason, because in some config files I found DC name to be like >>>>>>>>>>> DC1/DC2/DC3 >>>>>>>>>>> in some it is like Cassandra/Analytics (The ones I had specified >>>>>>>>>>> while >>>>>>>>>>> installation). Im unsure which yaml/property file to look for >>>>>>>>>>> correct >>>>>>>>>>> inconsistency. >>>>>>>>>>> >>>>>>>>>>> (C*heers :) - im so tempted to copy that) >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Bhuvan >>>>>>>>>>> >>>>>>>>>>> On Fri, Jan 22, 2016 at 8:47 PM, Alain RODRIGUEZ < >>>>>>>>>>> arodr...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> The the exact command you ran (stress-tool with options) could >>>>>>>>>>>> be useful to help you on that. >>>>>>>>>>>> >>>>>>>>>>>> However, Im able to create keyspace, tables and insert data >>>>>>>>>>>>> using cqlsh and it is replicating fine to all the nodes. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Having the schema might be useful too. >>>>>>>>>>>> >>>>>>>>>>>> Did you ran the cqlsh and the stress-tool from the same server >>>>>>>>>>>> ? If not, you might want to check the port you use (9042/9160/...) >>>>>>>>>>>> are open. >>>>>>>>>>>> Also, cqlsh uses local_one by default too. If both commands >>>>>>>>>>>> were run against the same DC, from the same machine they should >>>>>>>>>>>> behave the >>>>>>>>>>>> same way. Are they ? >>>>>>>>>>>> >>>>>>>>>>>> C*heers, >>>>>>>>>>>> >>>>>>>>>>>> ----------------- >>>>>>>>>>>> Alain >>>>>>>>>>>> >>>>>>>>>>>> The Last Pickle >>>>>>>>>>>> http://www.thelastpickle.com >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 2016-01-22 9:57 GMT+01:00 Bhuvan Rawal <bhu1ra...@gmail.com>: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> i have created a POC cluster with 2 DC , each having 4 nodes >>>>>>>>>>>>> with DSE 4.8.1 installed. >>>>>>>>>>>>> >>>>>>>>>>>>> On issuing cassandra stress im getting an error and data is >>>>>>>>>>>>> not being inserted: >>>>>>>>>>>>> *com.datastax.driver.core.exceptions.UnavailableException: Not >>>>>>>>>>>>> enough replica available for query at consistency LOCAL_ONE (1 >>>>>>>>>>>>> required but >>>>>>>>>>>>> only 0 alive)* >>>>>>>>>>>>> >>>>>>>>>>>>> However, Im able to create keyspace, tables and insert data >>>>>>>>>>>>> using cqlsh and it is replicating fine to all the nodes. >>>>>>>>>>>>> >>>>>>>>>>>>> Details of the cluster can be found below (all the nodes seem >>>>>>>>>>>>> to be alive and kicking): >>>>>>>>>>>>> >>>>>>>>>>>>> $ nodetool status Datacenter: Analytics ===================== >>>>>>>>>>>>> Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address >>>>>>>>>>>>> Load Tokens Owns Host ID Rack UN 10.41.55.17 428.5 KB 256 ? >>>>>>>>>>>>> 39d6d585-e641-4046-9d0b-797356597b5e rack1 UN 10.41.55.19 404.44 >>>>>>>>>>>>> KB 256 ? >>>>>>>>>>>>> 69edf930-efd9-4d74-a798-f3d4ac02e516 rack1 UN 10.41.55.18 423.21 >>>>>>>>>>>>> KB 256 ? >>>>>>>>>>>>> b74bab13-09b2-4760-bce9-c8ef05e50f6d rack1 UN 10.41.55.20 683.23 >>>>>>>>>>>>> KB 256 ? >>>>>>>>>>>>> fb5c4fed-6e1e-4ea8-838d-358106906830 rack1 Datacenter: >>>>>>>>>>>>> Cassandra ===================== Status=Up/Down |/ >>>>>>>>>>>>> State=Normal/Leaving/Joining/Moving -- Address Load Tokens >>>>>>>>>>>>> Owns Host ID Rack UN 10.41.55.15 209.4 KB 256 ? >>>>>>>>>>>>> ffc3b9a0-5d5c-4a3d-a99e-49d255731278 rack1 UN 10.41.55.21 227.44 >>>>>>>>>>>>> KB 256 ? >>>>>>>>>>>>> c68deba4-b9a2-43fc-bb13-6af74c88c210 rack1 UN 10.41.55.23 222.71 >>>>>>>>>>>>> KB 256 ? >>>>>>>>>>>>> 8229aa87-af00-48fa-ad6b-3066d3dc0e58 rack1 UN 10.41.55.22 218.72 >>>>>>>>>>>>> KB 256 ? >>>>>>>>>>>>> c7ba84fd-7992-41de-8c88-11574a72db99 rack1 >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Bhuvan Rawal >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
