On Sunday 18 May 2008 19:44, Ian Clarke wrote: > I've got to say, I really hope Perst employ better software engineers > than their web designers, because their website is awful. It somewhat > shakes my confidence in them. I know this seems like a very > superficial judgement, but if they put little care into the public > face of their software, it is reasonable to suspect that they may not > put too much care into the software itself.
You said pretty much the same about ours! > > Looking at the manual, it looks like Perst operates at a lower level > than db4o - you need to manually create and maintain indexes. This is > closer to the Java collections API, which could be good because its > more familiar, but it leaves more opportunities for the programmer to > screw up, which is bad. IMHO we will get better performance and fewer screwups from a more familiar API. > > My preference for db4o is based mainly on my familiarity with it, the > fact that it is more widely used, and the fact that it superficially > appears to have a more credible company behind it. > > None of these is a solid justification for choosing one option over > the other, but they all suggest that db4o is the way to go. We should, as bback suggests, test Perst, see what happens when it runs out of disk space. I will look into it soon. > > Ian. > > On Sun, May 18, 2008 at 11:08 AM, <bbackde at googlemail.com> wrote: > > I know I repeat myself, but if you consider db4o you should also > > consider perst as an option. > > Toad, can you do your 'disk full test' with perst to compare it against db4o? > > > > Perst and db4o seem to provide the same things. But according to > > http://www.garret.ru/~knizhnik/perstbench.html > > perst is much faster. > > > > No, I don't get money for advertising perst, but I made good > > experiences with it. > > > > On Sun, May 18, 2008 at 12:46 PM, Daniel Cheng > > <j16sdiz+freenet at gmail.com> wrote: > >> On Sun, May 18, 2008 at 12:27 PM, Florent Daigni?re > >> <nextgens at freenetproject.org> wrote: > >>> * Matthew Toseland <toad at amphibian.dyndns.org> [2008-05-17 19:00:13]: > >>> > >>>> On Saturday 17 May 2008 00:29, Matthew Toseland wrote: > >>>> > Ian and I have eventually come to the conclusion that we should include > >>>> db4o, > >>>> > and use it for our various persistence needs. I eventually reached the > >>>> > conclusion that while we can do most of what we need to do with simple > >>>> > flatfile databases, there are big chunks that will require a real database > >>>> of > >>>> > some kind (even if it's only a persistent hash table). db4o has various > >>>> > advantages: > >>>> > - Robust in real-world use. See for example this testimonial from a company > >>>> > who used it on cell phones: > >>>> > http://www.db4o.com/about/customers/success/mandalait.aspx > >>>> > BDBJE has not met our expectations in this regard. It seems very sensitive > >>>> to > >>>> > unusual situations - in particular, it will spontaneously corrupt and lose > >>>> > all data on running out of disk space. > >>>> > - True object database: no SQL, simple and powerful queries, etc. > >>>> > - Transparent or manual activation of objects from storage. > >>>> > - 800K jar, so not big enough to be a problem. > >>>> > - Mature and actively maintained. > >>>> > - Allows for future expansion (e.g. passive requests will need to store a > >>>> fair > >>>> > amount of persistent data). > >>>> > - Much more flexible than the hand-coded solution I was thinking of. We can > >>>> > persistent the entire queue (not just the splitfiles), if it's useful to do > >>>> > that. > >>>> > - Transactions (although this requires some juggling of in-memory objects on > >>>> > rollback). > >>>> > > >>>> > Tasks: > >>>> > - Add db4o to freenet-ext.jar. > >>>> > - Think about using it for the datastore. We don't want to have two > >>>> databases! > >>>> > Sdiz's new datastore may be the One True Store, or it may not be. If it's > >>>> > not, we don't want to keep BDBJE: we could build a db4o-based store, with or > >>>> > without LRU replacement. It would have the advantage of filling up more > >>>> > quickly than sdiz's store. It should require reconstructing less frequently > >>>> > than BDBJE! > >>>> > - Migrate the client layer, including splitfiles, pendingKeys, and so on, to > >>>> > be persisted via db4o. Of course there will be latency here when objects are > >>>> > not cached, so we will need to cache a few request choices in advance for > >>>> > each RequestStarter. And we will need to devise some way to deal with > >>>> > requests that don't want to be persisted - presumably we'd keep them in RAM. > >>>> > > >>>> It turns out that db4o does indeed unrecoverably self-corrupt when it runs out > >>>> of disk space. (Thanks nextgens for getting me to test this!) > >>>> > >>>> http://amphibian.dyndns.org/bdb4o-test.log > >>>> > >>> > >>> muhahahaha. > >>> > >>> Last time I checked the bdb database was recoverable... Okay > >>> it lost some^wmost of the data in the process but at least it did > >>> attempt to recover! > >> > >> Both BDB (the original C version) and BDB-JE have very bad history on > >> recovering. > >> Why should we lost some data when there are better alternative that don't? > >> > >>>> We will therefore have to keep a fallback. IMHO for the client layer the > >>>> fallback should be downloads.dat.gz. We are careful not to lose that when we > >>>> run out of disk space, and it should only contain what is needed to restart > >>>> requests from the beginning (in practice a lot will come from the store). > >>>> > >>> ... > >>> > >>> While we are at it, what's wrong with bdb-je's persistence framework > >>> again ? > >>> http://www.oracle.com/database/berkeley-db/je/index.html > >>> > >> > >> Another reason to get rid of BDB-JE is memory usage and performance. > >> > >> db4o is usable on J2ME and scalable to terabytes database. I think > >> this say a lot. > >> > >>>> I apologise if the above was presented as a fait accompli, any input on > >>>> databases would be appreciated. On Friday, me and Ian spent a long time > >>>> debating the issue, first and foremost of whether we should even have a > >>>> database; I was initially in favour of not having one at all, or using jdbm's > >>>> persistent hashtable class (HTree). > >>>> > >>>> Personally I think if we have a database it should be a native object database > >>>> i.e. either Perst or db4o. It also should be robust, low overhead, mature, > >>>> open source etc. I will start implementing the new client layer with db4o > >>>> soon, unless convinced to use something else in the meantime. But it seems > >>>> that with BDBJE (which isn't a native object database), you can lose the > >>>> database even by an unclean shutdown... can anyone confirm this from > >>>> experience? Or is it only out of disk space and memory corruption that causes > >>>> this? > >>> > >>> I'm still not convinced that we need a database... as our requirements > >>> are completely different from their typical use-cases... but well, your > >>> immediate concern is to store persistent requests to disk, right? What > >>> about using Hibernate or javax.persistence (from EE) to do that ? > >>> > >> eeeeeee.... > >> Hibernate is just ORM -- You need a sql backend for that. > >> (I am not oppose to the idea of using a sql backend, but then we have > >> to decide which one to use) > >> > >> javax.persistence have Java5 dependency, and you need a J2EE > >> container. just too ugly. > >> > >> -- > >> _______________________________________________ > >> Devl mailing list > >> Devl at freenetproject.org > >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > >> > > > > > > > > -- > > __________________________________________________ > > GnuPG key: (0x48DBFA8A) > > Keyserver: pgpkeys.pca.dfn.de > > Fingerprint: > > 477D F057 1BD4 1AE7 8A54 8679 6690 E2EC 48DB FA8A > > __________________________________________________ > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > > > > -- > Email: ian at uprizer.com > Cell: +1 512 422 3588 > Skype: sanity > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20080519/000a7d88/attachment.pgp>
