Re: zookeeper data and collection properties were lost
On 7/20/2020 10:30 AM, yaswanth kumar wrote: 1# I did make sure that zoo.cfg got the proper data dir and its not pointing to temp folder; do I need to set the variables in ZK_ENV.sh. as well on top of the zoo.cfg ?? Those are questions about the ZK server, which we are not completely qualified to answer. ZK and Solr are separate Apache projects, with separate mailing lists. We have some familiarity with ZK because it is required to run Solr in cloud mode, but are not experts. We can only provide minimal help with standalone ZK servers ... you would need to talk to the ZK project for the best information. Here are my confusions, as I said we are in two node architecture in DEV but maintaining only one instance of zookeeper, is that true that I need to maintain the same folder structure that we specify on the dataDir of zoo.cfg on both the nodes ?? Each ZK server is independent of the others and should have its own data directory. ZK will handle creating the contents of that directory, it is likely not something you would do. Each server could have a different setting for the data directory, or the same setting. Note that if the setting is the same on multiple servers, that each of those directories should point to separate storage. If you try to use a shared directory (perhaps with NFS) then I would imagine that ZK will not function correctly. A fault tolerant install of ZK cannot be created with only two servers. It requires a minimum of three. For the Solr part, only two servers are required for minimal fault tolerance. Each Solr server must be configured with the addresses and ports of all 3 (or more) zookeeper servers. See the Note in the following sections of the ZK documentation: https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#sc_zkMulitServerSetup https://zookeeper.apache.org/doc/r3.5.8/zookeeperStarted.html#sc_RunningReplicatedZooKeeper Thanks, Shawn
Re: zookeeper data and collection properties were lost
Thanks Erick for a quick response. Here are my responses for your questions 1# I did make sure that zoo.cfg got the proper data dir and its not pointing to temp folder; do I need to set the variables in ZK_ENV.sh. as well on top of the zoo.cfg ?? 2# I can confirm that we are not using the embedded one but we are using a standalone zookeeper 3.4.14 and also the admin UI is showing what we configured (port 2181) Here are my confusions, as I said we are in two node architecture in DEV but maintaining only one instance of zookeeper, is that true that I need to maintain the same folder structure that we specify on the dataDir of zoo.cfg on both the nodes ?? Thanks, On Mon, Jul 20, 2020 at 12:22 PM Erick Erickson wrote: > Some possibilities: > > 1> you haven’t changed your data dir for Zookeeper from the default > "/tmp/zookeeper” > > 2> you aren’t pointing to the Zookeepers you think you are. In particular > are you running embedded zookeeper? This should be apparent if you look on > the admin page ant the zookeeper URLs you’re pointing at are on port 9983 > > this is almost certainly some kind of misconfiguration, zookeeper data > doesn’t just disappear on its own that I know of. The admin UI will also > show you the exact parameters that Solr starts up with, check that they’re > all pointing to the ZK ensemble you expect and that the data directory is > preserved across restarts/reboots etc. > > Best, > Erick > > > On Jul 20, 2020, at 12:02 PM, yaswanth kumar > wrote: > > > > HI Team, > > > > Can someone help me understand on what could be the reason to lose both > > zookeeper data and also the collection information that will be stored > for > > each collection in the path ../solr/server/solr/ > > > > Here are the details of what versions that we use > > > > Solr - 8.2 > > Zookeeper 3.4.14 > > > > Two node solr cloud with zookeeper on single node, and when ever we see > an > > issue with networking between these two nodes, and once the connectivity > is > > restored, but when we restart the zookeeper service , everything was lost > > under /zookeeper_data/version-2/ and also the collection folders that > used > > to exists under ../solr/server/solr/ > > > > *Note*: We are testing this in DEV environment, but with this behavior we > > are afraid of moving this to production without knowing if that's an > issue > > with some configuration or zookeeper behavior and we need to adjust > > something else to not to wipe out the configs. > > > > -- > > Thanks & Regards, > > Yaswanth Kumar Konathala. > > yaswanth...@gmail.com > > -- Thanks & Regards, Yaswanth Kumar Konathala. yaswanth...@gmail.com
Re: zookeeper data and collection properties were lost
Some possibilities: 1> you haven’t changed your data dir for Zookeeper from the default "/tmp/zookeeper” 2> you aren’t pointing to the Zookeepers you think you are. In particular are you running embedded zookeeper? This should be apparent if you look on the admin page ant the zookeeper URLs you’re pointing at are on port 9983 this is almost certainly some kind of misconfiguration, zookeeper data doesn’t just disappear on its own that I know of. The admin UI will also show you the exact parameters that Solr starts up with, check that they’re all pointing to the ZK ensemble you expect and that the data directory is preserved across restarts/reboots etc. Best, Erick > On Jul 20, 2020, at 12:02 PM, yaswanth kumar wrote: > > HI Team, > > Can someone help me understand on what could be the reason to lose both > zookeeper data and also the collection information that will be stored for > each collection in the path ../solr/server/solr/ > > Here are the details of what versions that we use > > Solr - 8.2 > Zookeeper 3.4.14 > > Two node solr cloud with zookeeper on single node, and when ever we see an > issue with networking between these two nodes, and once the connectivity is > restored, but when we restart the zookeeper service , everything was lost > under /zookeeper_data/version-2/ and also the collection folders that used > to exists under ../solr/server/solr/ > > *Note*: We are testing this in DEV environment, but with this behavior we > are afraid of moving this to production without knowing if that's an issue > with some configuration or zookeeper behavior and we need to adjust > something else to not to wipe out the configs. > > -- > Thanks & Regards, > Yaswanth Kumar Konathala. > yaswanth...@gmail.com
zookeeper data and collection properties were lost
HI Team, Can someone help me understand on what could be the reason to lose both zookeeper data and also the collection information that will be stored for each collection in the path ../solr/server/solr/ Here are the details of what versions that we use Solr - 8.2 Zookeeper 3.4.14 Two node solr cloud with zookeeper on single node, and when ever we see an issue with networking between these two nodes, and once the connectivity is restored, but when we restart the zookeeper service , everything was lost under /zookeeper_data/version-2/ and also the collection folders that used to exists under ../solr/server/solr/ *Note*: We are testing this in DEV environment, but with this behavior we are afraid of moving this to production without knowing if that's an issue with some configuration or zookeeper behavior and we need to adjust something else to not to wipe out the configs. -- Thanks & Regards, Yaswanth Kumar Konathala. yaswanth...@gmail.com
