Hello Ivan,
Thank you very much for the response.
We do a bulk data load, and then incremental updates that occur in batch
mode. In some systems, the bulk load can be as much as 3 GB, with
incremental batch updates of 20-30 MB.
We prevent access to the system during bulk and batch updates, so no queries
occur.
The problem is that we noticed one of the Sedna systems use 2 G of ram
memory, which basically dragged down an Amazon micro instance and made it
unusable. (We stopped & started the instance and it worked correctly)
We have not discovered the root cause of the memory problem, and are still
investigating other concerns. (The OS and Sedna are the only thing we have
running on the micro instance)
We will start profiling Sedna request in an attempt discover root cause of
some of issues.
Thanks again for the quick response,
Malcolm
From: Ivan Shcheklein [mailto:shchekl...@gmail.com]
Sent: Thursday, July 07, 2011 2:07 PM
To: Malcolm Davis
Cc: sedna-discussion@lists.sourceforge.net
Subject: Re: [Sedna-discussion] Sedna memory requirement and configuration
Hi Malcolm,
I understand the problem of determining memory requirements is more
complicated than just volume. There are the number of nodes to consider,
and the number and type of the indexes.
I was curious if any formulas existed I could utilize to determine minimum
memory requirements.
No, there is no such formula. Apart from the data structure and indexes
there is also one important factor - workload - i.e. queries/updates you
run. Besides, what does enough mean? 100MB is *physically* enough to run any
query on any data.
I believe the only really effective approach to analyze queries. Run them
and look how many blocks they read/write (this information is available in
event log after session is closed).
Ivan Shcheklein,
Sedna Team
------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Sedna-discussion mailing list
Sedna-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sedna-discussion
