https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5845
--- Comment #4 from Theo Van Dinter <[EMAIL PROTECTED]> 2008-03-15 14:14:41 PST --- Woo, mapreduce! :) The issue with all this stuff, of course, is that it's written to deal with processing a large dataset with processing nodes in close proximity -- ie: a LAN w/ low latency, packet loss, high bandwidth, and probably no need for things like SSL, etc. Our model is very not that. ;) We don't have a single dataset to run over, we have several individual (and unlikely to be shared) datasets. Without sharing the data, we end up immediately reducing our setup to individual processing clusters, where the norm is to only have 1 machine (or at most a small handful of machines). So there's really two sets of issues here: a) We want <distributed clusters> to use a standard set of rules and send in the results. It doesn't really matter if it's net/weekly/per-commit, it's the same request essentially, though there are different ways to implement handling them since two are periodic and long-running, and the other is randomly timed and short-running. b) In each of the distributed clusters, we possibly want to farm out the incoming request to multiple machines for faster processing. Hadoop could (and our current client-server system does) handle issue b, but it really doesn't help for a, imo, which is what I think you're trying to target here. -- Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the assignee for the bug.
