Re: CLOSE_WAIT after connecting to multiple shards from a primary shard

[Mukunda Madhava]

Hi Otis,
Sending to solr-user mailing list.

We see this CLOSE_WAIT connections even when i do a simple http request via
curl, that is, even when i do a simple curl using a primary and secondary
shard query, like for e.g.

curl 
http://primaryshardhost:8180/solr/core0/select?q=*%3A*shards=secondaryshardhost1:8090/solr/appgroup1_11053000_11053100


While fetching data it is in ESTABLISHED state

-sh-3.2$ netstat | grep ESTABLISHED | grep 8090
tcp0  0 primaryshardhost:36805 secondaryshardhost1:8090
ESTABLISHED

After the request has come back, it is in CLOSE_WAIT state

-sh-3.2$ netstat | grep CLOSE_WAIT | grep 8090
tcp1  0 primaryshardhost:36805 secondaryshardhost1:8090
CLOSE_WAIT

why does Solr keep the connection to the shards in CLOSE_WAIT?

Is this a feature of Solr? If we modify an OS property (I dont know how) to
cleanup the CLOSE_WAITs will it cause an issue with subsequent searches?

Can someone help me please?

thanks,
Mukunda

On Mon, May 30, 2011 at 5:59 PM, Otis Gospodnetic 
otis_gospodne...@yahoo.com wrote:

 Hi,

 A few things:
 1) why not send this to the Solr list?
 2) you talk about searching, but the code sample is about optimizing the
 index.

 3) I don't have SolrJ API in front of me, but isn't there is
 CommonsSolrServe
 ctor that takes in a URL instead of HttpClient instance?  Try that one.

 Otis
 -
 Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
 Lucene ecosystem search :: http://search-lucene.com/



 - Original Message 
  From: Mukunda Madhava mukunda...@gmail.com
  To: gene...@lucene.apache.org
  Sent: Mon, May 30, 2011 1:54:07 PM
  Subject: CLOSE_WAIT after connecting to multiple shards from a primary
 shard
 
  Hi,
  We are having a primary Solr shard, and multiple secondary shards.
  We
  query data from the secondary shards by specifying the shards param in
  the
  query params.
 
  But we found that after recieving the data, there  are large number of
  CLOSE_WAIT on the secondary shards from the primary  shards.
 
  Like for e.g.
 
  tcp1   0 primaryshardhost:56109  secondaryshardhost1:8090
  CLOSE_WAIT
  tcp1   0 primaryshardhost:51049  secondaryshardhost1:8090
  CLOSE_WAIT
  tcp1   0 primaryshardhost:49537  secondaryshardhost1:8089
  CLOSE_WAIT
  tcp1   0 primaryshardhost:44109  secondaryshardhost2:8090
  CLOSE_WAIT
  tcp1   0 primaryshardhost:32041  secondaryshardhost2:8090
  CLOSE_WAIT
  tcp1   0 primaryshardhost:48533  secondaryshardhost2:8089
  CLOSE_WAIT
 
 
  We open the Solr connections  as below..
 
  SimpleHttpConnectionManager cm =  new
  SimpleHttpConnectionManager(true);
   cm.closeIdleConnections(0L);
  HttpClient  httpClient = new HttpClient(cm);
  solrServer = new  CommonsHttpSolrServer(url,httpClient);
   solrServer.optimize();
 
  But still we see these issues. Any ideas?
  --
  Thanks,
  Mukunda
 




-- 
Thanks,
Mukunda

CLOSE_WAIT after connecting to multiple shards from a primary shard

[Mukunda Madhava]

Hi,
We are having a primary Solr shard, and multiple secondary shards. We
query data from the secondary shards by specifying the shards param in the

query params.

But we found that after recieving the data, there are large number of
CLOSE_WAIT on the secondary shards from the primary shards.

Like for e.g.

tcp 1 0 primaryshardhost:56109 secondaryshardhost1:8090 CLOSE_WAIT
tcp 1 0 primaryshardhost:51049 secondaryshardhost1:8090 CLOSE_WAIT
tcp 1 0 primaryshardhost:49537 secondaryshardhost1:8089 CLOSE_WAIT
tcp 1 0 primaryshardhost:44109 secondaryshardhost2:8090 CLOSE_WAIT
tcp 1 0 primaryshardhost:32041 secondaryshardhost2:8090 CLOSE_WAIT
tcp 1 0 primaryshardhost:48533 secondaryshardhost2:8089 CLOSE_WAIT


We even changed the code to open the Solr connections as below..

SimpleHttpConnectionManager cm = new
SimpleHttpConnectionManager(true);
cm.closeIdleConnections(0L);
HttpClient httpClient = new HttpClient(cm);
solrServer = new CommonsHttpSolrServer(url,httpClient);
solrServer.optimize();

But still we see these issues. Any ideas?

Does Solr persist the connections to the secondary shards?
-- 
Thanks,
Mukunda