I believe os_process_timeout applies to view servers (default is 5 seconds) so you can configure the timeout for view queries, not sure about CPU/HDD Space (although inspecting _stats might shed some light).
Ta. On 28 November 2014 at 10:12, Peter Grman <[email protected]> wrote: > No, I don't. The program should be for analysing logs (collected by > fluentd) - should be open source and on github, however there isn't much > done yet: https://github.com/logTank/ > > The index rebuilding shouldn't be a problem as CouchDB will be only used > for general stats and the user actually won't see the up to date data, but > always with a delay - another advantage of CouchDB, I can read the queries > without bothering the system, and once the data is outdated, I can update > the index. At least so far the theory, I'll need to run some performance > tests if that actually works, once I'll have a MVP. The other option is to > use MongoDB for ad-hoc queries, but I was thinking that CouchDB will be > more efficient as storage is so cheap. > > As I'm learning every time I look up info about CouchDB something new, and > something becomes more clear, I'm also glad about feedback on the idea in > general, how I want to use CouchDB. > > However I'd be also very happy if I could somehow solve the problem with > the possible DoS attacks :). Maybe there is something in CouchDB or evalcx > which I can configure - maximal runtime of a map/reduce function? > (shouldn't be more than 1ms). Or there are some logged data by CouchDB > about the resources required by views (CPU Time + HDD Space)? > > Cheers > Peter > > On Fri Nov 28 2014 at 2:54:51 AM Alexander Gabriel <[email protected]> > wrote: > > > sorry for being off-topic > > Alex > > > > > > 2014-11-28 2:52 GMT+01:00 Alexander Gabriel <[email protected]>: > > > > > sounds like a very interesting application > > > > > > seems like you dont care if the user has to wait for an index to be > built > > > when the user creates a query > > > > > > Alex > > > > > > > > > 2014-11-28 2:23 GMT+01:00 Peter Grman <[email protected]>: > > > > > >> Hi Alex, > > >> > > >> Yes, the users would be able to import different sets of data, which > > isn't > > >> relational, and use the platform to analyse it. The analysed data > would > > be > > >> in 99% of the cases append only (+ removing old data) and the data can > > be > > >> defined by the user, as well as be hierarchical. > > >> > > >> When I thought about the system in the beginning, CouchDB seemed like > an > > >> awesome choice as there would be only a couple of well defined queries > > and > > >> storage is generally cheap, I thought that CouchDB views and their > > caching > > >> are what I'm looking for. > > >> > > >> The problem is again only with people who want to trick the system. I > > >> would > > >> be also happy with a solution which would detect bad views ones they > > have > > >> been deployed (uses too much space, takes too long to compute) and > > >> deactivates and marks them for me to check. This way I could check > those > > >> few people who try a DoS attack and ban them from the service. > > >> > > >> The additional main problem was, if it is really impossible to get > data > > >> from a different database inside the view and if the user won't be > able > > to > > >> access the underlying system, ..., or if it is just very difficult => > > >> possible, if someone wants to do it they'll find a way. But after > > reading > > >> more and understanding more, how the views are executed using evalcx > I > > >> think the other problems aren't a big concern for me anymore, is that > > >> correct?. > > >> > > >> Although I've found in the code "if possible, use evalcx (not always > > >> available)" - how can I check that evalcx is available on my system? > Or > > is > > >> it just a note for older distributions, nothing to be concerned about > > >> anymore? > > >> > > >> Thank you > > >> > > >> Cheers > > >> Peter > > >> > > >> On Fri Nov 28 2014 at 1:37:57 AM Alexander Gabriel <[email protected]> > > >> wrote: > > >> > > >> > Hi Peter > > >> > > > >> > Will the users create their own datastructures too? > > >> > If not this sounds like sql on relational tables might be a better > > tool > > >> for > > >> > the problem. > > >> > It seems to me you're hitting exactly the weak point of most nosql > > >> > solutions. > > >> > > > >> > Alex > > >> > > > >> > > > >> > 2014-11-28 0:49 GMT+01:00 Peter Grman <[email protected]>: > > >> > > > >> > > Hi, > > >> > > > > >> > > this might sound like a terrible idea to someone who knows > CouchDB, > > >> and > > >> > if > > >> > > that's the case, please just take a minute or two, to explain why, > > >> > > otherwise, if the idea isn't so crazy after all, I hope I'll get > > some > > >> > > solutions to my problem: > > >> > > > > >> > > I'm thinking of creating a platform based on CouchDB, where each > set > > >> of > > >> > > users (group, customer, ...) would get their own CouchDB Database, > > to > > >> > store > > >> > > and query data. I've heard in a podcast, roughly a year ago, that > > >> this is > > >> > > how CouchDB was meant to be - many smaller databases. > > >> > > > > >> > > To query the data, I want to allow them, to define their own > custom > > >> > > queries. Now I could (and want to) create a form which allows to > > >> build a > > >> > > query and translates it to a JS view, but I was thinking about > > >> > > additionally, on top of that, allowing them to define their custom > > >> views > > >> > > directly in JS. They would basically be allowed to define their > > custom > > >> > > Map/Reduce functions. > > >> > > > > >> > > There is a lot which can go wrong with this the worst ones I came > up > > >> > with: > > >> > > - DoS attack with endless loops inside the function > > >> > > - DoS attack by emitting too much data (potentially in a loop > again) > > >> > > > > >> > > As far as I've understood, it's not possible to access other > > Databases > > >> > from > > >> > > within the view, is this understanding of mine correct? > > >> > > > > >> > > Is it possible to access the filesystem or network services in any > > way > > >> > from > > >> > > the CouchDB view or is the JavaScript engine, which is running the > > >> code, > > >> > > limiting enough? > > >> > > > > >> > > Are there any other things which could go wrong? - or did actually > > >> > somebody > > >> > > already use CouchDB like this, and it's perfectly normal? > > >> > > > > >> > > Is there any way I could prevent the problem with endless loops > and > > >> data > > >> > > emitting from happening? - I can run JSLint, which maybe will > detect > > >> an > > >> > > endless loop, but that won't help against a loop with a million > > >> > iterations, > > >> > > which will be called for every item inside CouchDB - still quite > > >> endless. > > >> > > > > >> > > Thank you for your help! > > >> > > > > >> > > Cheers, > > >> > > Peter > > >> > > > > >> > > > >> > > > > > > > > > -- mischa (aka muji).
