Re: [Catalyst] KiokuDB, MongoDB and the NoSQL thing
As an addendum to this thread ... It just occurred to me now, following an interview today with a developer of millions-users social-network games, that I could be embracing this NoSQL thing in an indirect way, taking into account that still I consider it just a reasonable compromise for some applications, but not a general database pancea. Because Muldis D is designed from the start to be implementable over both SQL DBMSs of various capabilities and within general purpose languages, meaning that any functionality a DBMS doesn't natively support can be hoisted to the application, this structure is a natural fit for NoSQL. NoSQL is all about keeping the main DBMS simple, supporting just a few native operations, and massively parallel. And so, one could write Muldis D code and deploy it both over SQL and NoSQL, and in the latter case it would just do a lot of the work on the application side. Essentially, Muldis D takes care of the I believe I did, Bob, so you can use a NoSQL store without doing the write a distributed map-reduce function in Erlang yourself, instead you just say what you want declaratively much like in SQL, and then the right thing happens. Of course, that won't generally give you the consistency/integrity that a relational DBMS backend has over a NoSQL one, but at least you'll get the powerful queries part. You heard it here first. -- Darren Duncan S.A. Kiehn wrote: Thanks for the link Darren. I will read and ponder. Also of interest, I seen that Search::GIN has had a new release http://blogs.perl.org/users/sawyer_x/2010/03/searchgin-004-finally-out.html. The post also mentions some docs explaining queries, but I have not found them yet. Active Search::GIN development would encourage KiokuDB usage I would think. Thanks for the posts regarding this topic. S. Kiehn Darren Duncan wrote: Here's something timely on Ars, in that I just discovered it now around the time of this discussion thread: http://arstechnica.com/business/data-centers/2010/02/-since-the-rise-of.ars It lays out a summary of how the SQL and NoSQL worlds compare, and what sort of trade-offs you get for each choice, and it introduces several specific NoSQL projects. I also learned something, such that what CouchDB and MongoDB specifically represent data with is JSON documents. So basically, what you pick out of today's choices depends on your priorities. ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] KiokuDB, MongoDB and the NoSQL thing
Thanks for the link Darren. I will read and ponder. Also of interest, I seen that Search::GIN has had a new release. The post also mentions some docs explaining queries, but I have not found them yet. Active Search::GIN development would encourage KiokuDB usage I would think. Thanks for the posts regarding this topic. S. Kiehn Darren Duncan wrote: Here's something timely on Ars, in that I just discovered it now around the time of this discussion thread: http://arstechnica.com/business/data-centers/2010/02/-since-the-rise-of.ars It lays out a summary of how the SQL and NoSQL worlds compare, and what sort of trade-offs you get for each choice, and it introduces several specific NoSQL projects. I also learned something, such that what CouchDB and MongoDB specifically represent data with is JSON documents. So basically, what you pick out of today's choices depends on your priorities. ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] KiokuDB, MongoDB and the NoSQL thing
On 4 Mar 2010, at 23:38, Darren Duncan wrote: MongoDB specifically represent data with is JSON documents. Not quite true. They are BSON (B = binary) and have specific types for dates, regexps and binary data___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] KiokuDB, MongoDB and the NoSQL thing
S.A. Kiehn wrote: I do not see many posts regarding uses of KiokuDB within Catalyst so I was curious about the opinion of the community in regards to its usage. Is it still to early within development? Also, I have been reading more about the increase in the NoSQL interest, with a particular interest in the MongoDB database (it seems to be similar in some respects to KiokuDB), but I do not find Perl people in the discussion as much as others (Ruby, PHP). Are there developers in the Catalyst community who lean toward NoSQL concepts over traditional RDMS's, or is best to view as a tool to use at times? How about MongoDB? Am I being suckered by another bandwagon? Thanks, Scott K. Well I happen to be strongly opinionated on this topic, so here goes ... While these other DBMSs have their uses, I believe that anyone is misguided who figures they are superior solutions for most uses of relational databases. I believe that the relational model of data is still the single best general solution that we have come up with for organizing and querying any non-trivial amount of data that is the least bit structured. Especially so when that data needs to be or could possibly be worked with by multiple kinds of users or applications that may have different needs, and need their own views of that data. It is also good for persisting data, but persistence isn't the main point; rather easy and flexible organization and querying is the point; persistence is optional, same as persisting an array of data is optional. That said, while it does the job well enough many times, SQL is deeply flawed and doesn't represent the relational model of data properly, but just approximates it to varying degrees, this variance depends partly on the SQL DBMS in use, which range in features quite a bit. I believe that quite often when people complain about relational databases, they are really complaining about SQL databases, as if those were the same, and so various solutions are proposed like ORMs or these NoSQL concepts. But the problem is that they are throwing out the baby with the bathwater. Yes, SQL is quite flawed, but the relational model it approximates is not (or it is much less flawed, if you want to argue that having something actually flawless is impossible). I believe that the best solution is not to ditch everything, but rather to provide a DBMS that actually implements the full relational model of data, and not just an approximation, nor ditching the concept entirely. If you do that, then a lot of these other kinds of DBMSs like so-called object-relational or object or key-value become redundant, because the full relational model incorporates their features. For example, the full relational model supports having tuple/row and relation/table values as attributes/fields of other ones, and so you can natively model any arbitrarily complex data type that an object could model, without too much indirection (similarly to how many languages support having an array as an element of an array). Hence ORMs are redundant and so are distinct concepts of object stores, or alternately they become a lot thinner. And so, as a model of good community behaviour, I'm not just sitting around talking about what people should do, but I am going out and actually doing it, creating a DBMS that provides the full relational model (both as self-contained implementations as well as implementations over existing DBMSs), and right now. Simultaneously taking what SQL should have been and all the good features of the SQL alternatives, elegantly integrated in one streamlined bloat-free package. This project, with the umbrella name Muldis (see CPAN etc), is mostly pre-alpha right now, but I am filling in the gaps as soon as I can and I am confident it will be viable. In fact, I released updates to 3 of 4 current sub-projects on CPAN earlier today. So, to answer your question, go ahead and explore the alternatives you name, but I will say to anyone that the relational model is the single best general solution for data aggregation and processing. -- Darren Duncan ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] KiokuDB, MongoDB and the NoSQL thing
On Tuesday 02 March 2010 12:07:23 am S.A. Kiehn wrote: I have a couple of production Catalyst/DBIx::Class sites on Debian stable, and then on my personal hobby site I use local::lib to try out new things. Recently I split out my users for this site into a separate model and I thought it a good exercise to learn and use KiokuDB. It was just a couple of simple objects, users roles, but I believe I have a better understanding of how a schema-less data model would work. All I do are lookups based on ID or indexed object values, but doing any type of ordering by dates or titles is a mystery. It seems that the Search::GIN is to provide this sort of functionality, but it is under-documented and has not had an update for awhile. I do not see many posts regarding uses of KiokuDB within Catalyst so I was curious about the opinion of the community in regards to its usage. Is it still to early within development? Also, I have been reading more about the increase in the NoSQL interest, with a particular interest in the MongoDB database (it seems to be similar in some respects to KiokuDB), but I do not find Perl people in the discussion as much as others (Ruby, PHP). Are there developers in the Catalyst community who lean toward NoSQL concepts over traditional RDMS's, or is best to view as a tool to use at times? How about MongoDB? Am I being suckered by another bandwagon? Kioku is a really beautiful idea. I was hoping to use it on a recent project but unfortunately it wasn't the right fit -- one of the most common access patterns I would have to support for this app was unduly difficult in Kioku at this time so I decided to put it off. So, no real first-hand advice from me. Might I suggest hitting #catalyst and #kiokudb on irc.perl.org? I know that there's at least one person using MongoDB with Catalyst on #catalyst, so you can probably get some words of wisdom. Andrew ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] KiokuDB, MongoDB and the NoSQL thing
I have a couple of production Catalyst/DBIx::Class sites on Debian stable, and then on my personal hobby site I use local::lib to try out new things. Recently I split out my users for this site into a separate model and I thought it a good exercise to learn and use KiokuDB. It was just a couple of simple objects, users roles, but I believe I have a better understanding of how a schema-less data model would work. All I do are lookups based on ID or indexed object values, but doing any type of ordering by dates or titles is a mystery. It seems that the Search::GIN is to provide this sort of functionality, but it is under-documented and has not had an update for awhile. I do not see many posts regarding uses of KiokuDB within Catalyst so I was curious about the opinion of the community in regards to its usage. Is it still to early within development? Also, I have been reading more about the increase in the NoSQL interest, with a particular interest in the MongoDB database (it seems to be similar in some respects to KiokuDB), but I do not find Perl people in the discussion as much as others (Ruby, PHP). Are there developers in the Catalyst community who lean toward NoSQL concepts over traditional RDMS's, or is best to view as a tool to use at times? How about MongoDB? Am I being suckered by another bandwagon? Thanks, Scott K. ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] KiokuDB, MongoDB and the NoSQL thing
This is actually a pretty interesting subject, and while I can't help you with any of it as I have no experience (I have tried KiokuDB once for a project I was working on, but found that it's too young for my purposes, from the same reason as you - searching and sorting is next to impossible). Anyway, I hope this subject gets more attention, as the possibilities are huge. Ido. On Tue, Mar 2, 2010 at 8:07 AM, S.A. Kiehn keenli...@ymail.com wrote: I have a couple of production Catalyst/DBIx::Class sites on Debian stable, and then on my personal hobby site I use local::lib to try out new things. Recently I split out my users for this site into a separate model and I thought it a good exercise to learn and use KiokuDB. It was just a couple of simple objects, users roles, but I believe I have a better understanding of how a schema-less data model would work. All I do are lookups based on ID or indexed object values, but doing any type of ordering by dates or titles is a mystery. It seems that the Search::GIN is to provide this sort of functionality, but it is under-documented and has not had an update for awhile. I do not see many posts regarding uses of KiokuDB within Catalyst so I was curious about the opinion of the community in regards to its usage. Is it still to early within development? Also, I have been reading more about the increase in the NoSQL interest, with a particular interest in the MongoDB database (it seems to be similar in some respects to KiokuDB), but I do not find Perl people in the discussion as much as others (Ruby, PHP). Are there developers in the Catalyst community who lean toward NoSQL concepts over traditional RDMS's, or is best to view as a tool to use at times? How about MongoDB? Am I being suckered by another bandwagon? Thanks, Scott K. ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] KiokuDB, MongoDB and the NoSQL thing
On 02/03/10 17:07, S.A. Kiehn wrote: I have a couple of production Catalyst/DBIx::Class sites on Debian stable, and then on my personal hobby site I use local::lib to try out new things. Recently I split out my users for this site into a separate model and I thought it a good exercise to learn and use KiokuDB. It was just a couple of simple objects, users roles, but I believe I have a better understanding of how a schema-less data model would work. All I do are lookups based on ID or indexed object values, but doing any type of ordering by dates or titles is a mystery. It seems that the Search::GIN is to provide this sort of functionality, but it is under-documented and has not had an update for awhile. I do not see many posts regarding uses of KiokuDB within Catalyst so I was curious about the opinion of the community in regards to its usage. Is it still to early within development? Also, I have been reading more about the increase in the NoSQL interest, with a particular interest in the MongoDB database (it seems to be similar in some respects to KiokuDB), but I do not find Perl people in the discussion as much as others (Ruby, PHP). Are there developers in the Catalyst community who lean toward NoSQL concepts over traditional RDMS's, or is best to view as a tool to use at times? How about MongoDB? Am I being suckered by another bandwagon? Also Apache CouchDB. I'm curious to know how these things work out for what I see as real world cases, where you do actually want to do searches on correlated data. For instance, say you wanted to make a phpbb-style forum. You have several forum areas, and within each there are many threads, each containing many posts. Would every post in a thread be a new document, or would the entire thread be one big document? How would they be linked to the forums, by an ID in the document, or do we get into some kind of mega-document encapsulating everything on the board? Say I wanted to do a search for all posts made by users named John in the forum called Linux, what would the syntax look like? cheers, Toby ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
