We have synonym files bigger than 5MB so even with compression that would be probably failing (not using solr cloud yet) Roman On 6 May 2013 23:09, "David Parks" <davidpark...@yahoo.com> wrote:
> Wouldn't it make more sense to only store a pointer to a synonyms file in > zookeeper? Maybe just make the synonyms file accessible via http so other > boxes can copy it if needed? Zookeeper was never meant for storing > significant amounts of data. > > > -----Original Message----- > From: Jan Høydahl [mailto:jan....@cominvent.com] > Sent: Tuesday, May 07, 2013 4:35 AM > To: solr-user@lucene.apache.org > Subject: Re: Solr Cloud with large synonyms.txt > > See discussion here > http://lucene.472066.n3.nabble.com/gt-1MB-file-to-Zookeeper-td3958614.html > > One idea was compression. Perhaps if we add gzip support to SynonymFilter > it > can read synonyms.txt.gz which would then fit larger raw dicts? > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > 6. mai 2013 kl. 18:32 skrev Son Nguyen <s...@trancorp.com>: > > > Hello, > > > > I'm building a Solr Cloud (version 4.1.0) with 2 shards and a Zookeeper > (the Zookeeer is on different machine, version 3.4.5). > > I've tried to start with a 1.7MB synonyms.txt, but got a > "ConnectionLossException": > > Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for /configs/solr1/synonyms.txt > > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > > at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1266) > > at > org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:270) > > at > org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:267) > > at > > org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java > :65) > > at > org.apache.solr.common.cloud.SolrZkClient.setData(SolrZkClient.java:267) > > at > org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:436) > > at > org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:315) > > at > org.apache.solr.cloud.ZkController.uploadToZK(ZkController.java:1135) > > at > org.apache.solr.cloud.ZkController.uploadConfigDir(ZkController.java:955) > > at > org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:285) > > ... 43 more > > > > I did some researches on internet and found out that because Zookeeper > znode size limit is 1MB. I tried to increase the system property > "jute.maxbuffer" but it won't work. > > Does anyone have experience of dealing with it? > > > > Thanks, > > Son > >
