Sorry guys I spoke too fast. I looked at the code again. No it doesn't correlate with commits at all. I was mistaken.
On Wed, Mar 19, 2014 at 10:06 AM, Chris W <chris1980....@gmail.com> wrote: > Thanks, Shawn and Shalin > > How does the frequency of commit affect zookeeper? > > > Thanks > > > On Tue, Mar 18, 2014 at 9:12 PM, Shalin Shekhar Mangar < > shalinman...@gmail.com> wrote: > >> SolrCloud will update Zookeeper on state changes (node goes to >> recovery, comes back up etc) or for leader election and during >> collection API commands. It doesn't correlate directly with indexing >> but is correlated with how frequently you call commit. >> >> On Wed, Mar 19, 2014 at 5:46 AM, Shawn Heisey <s...@elyograg.org> wrote: >> > On 3/18/2014 5:46 PM, Chris W wrote: >> >> >> >> I am running a 3 node zookeeper 3.4.5 Quorum. I am running into issues >> >> with Zookeeper transaction logs >> >> >> >> [myid:2] - ERROR [main:QuorumPeer@453] - Unable to load database on >> disk >> >> java.io.IOException: Unreasonable length = 1048587 >> >> at >> >> >> org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:100) >> >> at >> >> org.apache.zookeeper.server.persistence.Util.readTxnBytes(Util.java:233) >> >> at >> >> >> >> >> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:602) >> >> at >> >> >> >> >> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:157) >> >> at >> >> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223) >> >> >> >> To unblock temporarily, i deleted the most recent txn log. How do i tell >> >> zookeeper to not grow the transaction log beyond x MegaBytes? >> >> >> >> How often does the transaction log get updated? Does zk transactions log >> >> grow everytime we index data into a new collection? >> > >> > >> > Zookeeper is a separate project at Apache. ZK file management is >> discussed >> > in the ZK documentation. >> > >> > >> http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance >> > >> > There is a bug filed on Zookeeper for the issue you are seeing, with a >> > fairly simple workaround. It is fixed in the 3.4.6 version, which was >> > released last week. I will see whether we can get ZK upgraded to 3.4.6 >> in >> > the Solr 4.8.0 release. I don't think we want to risk doing that >> upgrade in >> > 4.7.1, but I could be wrong. >> > >> > https://issues.apache.org/jira/browse/ZOOKEEPER-1513 >> > http://zookeeper.apache.org/releases.html >> > >> > I am actually not sure how often SolrCloud updates Zookeeper. It happens >> > whenever the collections API is called for sure, and it may happen >> anytime >> > you index data as well. >> > >> > Thanks, >> > Shawn >> > >> >> >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> > > > > -- > Best > -- > C -- Regards, Shalin Shekhar Mangar.
