Thanks Telmo, I'll catch up with you offline. Cheers, ~PM Date: Fri, 8 Nov 2013 23:09:04 +0100 Subject: Re: [agi] Rules + Big Data From: [email protected] To: [email protected]
Hi PM, On Fri, Nov 8, 2013 at 10:18 PM, Piaget Modeler <[email protected]> wrote: No. I have a 1000+ page detailed design for the Mind Server and PAM-P2. we are already coding agents for PAM-P2. But I'm always on the lookout for a better and more expeditious way to do things. So I'm in the middle of performing an architectural alternative analysis on whether to (a) use agents coded in a typical prodecural language (e.g., Java, C++, etc.) calling conventional stored procedures on a very large database (up to a billion records), or (b) implement each agent as a rule, using a rules engine which can fire rules in parallel against a big data store. So I'm seeing whether or not a vendor exists so that I can buy versus build it. We're already coding (a), but I would like to see if there is a vendor out there so that we can buy (b). If not, then in a later phase we might build (b) as time and money and justification permit. My advice, for what is worth, is that you should worry less about pre-packaged solutions at thins point and try to precisely specify the trade-offs. One good high-level starting point would be the CAP theorem: https://en.wikipedia.org/wiki/CAP_theorem Which two sides of the CAP triangle do you choose? Then there are many subtle trade-offs. For example with Riak you get a distributed key-value store where you can write map-reduce jobs that could possibly fit you bill of rules running in parallel: http://basho.com/riak/ One subtle problem with Riak is that it needs at least three nodes to be performant. In my experience, this makes your development and testing environment complicated. If you are willing to pay that price, you could get a lot of scalability and parallelism in return. You should probably go though a list of the more popular nosql databases and understand the trade-offs: http://nosql-database.org/ As it stands, your question is still way too vague. But maybe these pointers are useful. I'm working on a project with some similarities to yours, and I'm currently just using a single local instance of LevelDB (an open source Google key-value store / b-tree). I decided it's better to worry about scalability only once I really need it. I want to convince myself that I can do something worthwhile with Gigabytes of data before moving to Terabytes. Cheers,Telmo. Cheers, ~ PM Date: Fri, 8 Nov 2013 15:23:17 -0500 Subject: Re: [agi] Rules + Big Data From: [email protected] To: [email protected] So you have a high level description of an AGI design and you are asking if someone can fill in the details? On Fri, Nov 8, 2013 at 12:41 PM, Piaget Modeler <[email protected]> wrote: See http://piagetmodeler.tumblr.com for more background. ~PM Date: Fri, 8 Nov 2013 09:06:11 +0100 Subject: Re: [agi] Rules + Big Data From: [email protected] To: [email protected] Hi, This is a bit vague. What kind of data is it and what do you want to do with it? Best,Telmo. On Fri, Nov 8, 2013 at 7:56 AM, Piaget Modeler <[email protected]> wrote: Hi all, Does anyone know of a rules system that can attach to and apply rule patterns over a big data database? Thanks in advance. ~PM AGI | Archives | Modify Your Subscription AGI | Archives | Modify Your Subscription AGI | Archives | Modify Your Subscription -- -- Matt Mahoney, [email protected] AGI | Archives | Modify Your Subscription AGI | Archives | Modify Your Subscription AGI | Archives | Modify Your Subscription ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
