Hi Steve, Regarding the 2nd issue, a JIRA is already created and patch is uploaded (SOLR-9527). Can someone review and commit the patch?
Regarding 1st issue, does backup command succeeds? Also do you see any warning/error log messages? How about the restore command? Thanks Hrishikesh On Sat, Sep 24, 2016 at 12:14 PM, Stephen Weiss <steve.we...@wgsn.com> wrote: > Hi everyone, > > We're very excited about SolrCloud's new backup / restore collection APIs, > which should introduce some major new efficiencies into our indexing > workflow. Unfortunately, we've run into some snags with it that are > preventing us from moving into production. I was hoping someone on the > list could help. > > 1) Data inconsistencies > > There seems to be a problem getting all the data consistently. Sometimes, > the backup will contain all of the data in the collection, and sometimes, > large portions of the collection (as much as 40%) will be missing. We > haven't quite figured out what might cause this yet, although one thing > I've noticed is the chances of success are greater when we are only backing > up one collection at a time. Unfortunately, for our workflow, it will be > difficult to make that work, and there still doesn't seem to be a guarantee > of success either way. > > 2) Shards are not distributed > > To make matters worse, for some reason, any type of restore operation > always seems to put all shards of the collection on the same node. We've > tried setting maxShardsPerNode to 1 in the restore command, but this has no > effect. We are seeing the same behavior on both 6.1 and 6.2.1. No matter > what we do, all the shards always go to the same node - and it's not even > the node that we execute the restore request on, but oddly enough, a > totally different node, and always the same one (the 4th one). It should > be noted that all nodes of our 8 node cloud are up and totally functional > when this happens. > > To work around this, we wrote up a quick script to create an empty > collection, which always distributes itself across the cloud quite well > (another indication that there's nothing wrong with the nodes themselves), > and then we rsync the individual shards' data into the empty shards and > reload the collection. This works fine, however, because of the data > inconsistencies mentioned above, we can't really move forward anyway. > > > Problem #2, we have a reasonable workaround for, but problem #1 we do > not. If anyone has any thoughts about either of these problems, I would be > very grateful. Thanks! > > -- > Steve > > ________________________________ > > WGSN is a global foresight business. Our experts provide deep insight and > analysis of consumer, fashion and design trends. We inspire our clients to > plan and trade their range with unparalleled confidence and accuracy. > Together, we Create Tomorrow. > > WGSN<http://www.wgsn.com/> is part of WGSN Limited, comprising of > market-leading products including WGSN.com<http://www.wgsn.com>, WGSN > Lifestyle & Interiors<http://www.wgsn.com/en/lifestyle-interiors>, WGSN > INstock<http://www.wgsninstock.com/>, WGSN StyleTrial<http://www.wgsn. > com/en/styletrial/> and WGSN Mindset<http://www.wgsn.com/ > en/services/consultancy/>, our bespoke consultancy services. > > The information in or attached to this email is confidential and may be > legally privileged. If you are not the intended recipient of this message, > any use, disclosure, copying, distribution or any action taken in reliance > on it is prohibited and may be unlawful. If you have received this message > in error, please notify the sender immediately by return email and delete > this message and any copies from your computer and network. WGSN does not > warrant that this email and any attachments are free from viruses and > accepts no liability for any loss resulting from infected email > transmissions. > > WGSN reserves the right to monitor all email through its networks. Any > views expressed may be those of the originator and not necessarily of WGSN. > WGSN is powered by Ascential plc<http://www.ascential.com>, which > transforms knowledge businesses to deliver exceptional performance. > > Please be advised all phone calls may be recorded for training and quality > purposes and by accepting and/or making calls from and/or to us you > acknowledge and agree to calls being recorded. > > WGSN Limited, Company number 4858491 > > registered address: > > Ascential plc, The Prow, 1 Wilder Walk, London W1B 5AP > > WGSN Inc., tax ID 04-3851246, registered office c/o National Registered > Agents, Inc., 160 Greentree Drive, Suite 101, Dover DE 19904, United States > > 4C Serviços de Informação Ltda., CNPJ/MF (Taxpayer's Register): > 15.536.968/0001-04, Address: Avenida Cidade Jardim, 377, 7˚ andar CEP > 01453-000, Itaim Bibi, São Paulo > > 4C Business Information Consulting (Shanghai) Co., Ltd, 富新商务信息咨询（上海）有限公司, > registered address Unit 4810/4811, 48/F Tower 1, Grand Gateway, 1 Hong Qiao > Road, Xuhui District, Shanghai >
