On 11 Jul 2008, at 17:12, Brian Aker wrote:
Hi!
Lets figure out a memory policy first. I talked to Jay about this
for a bit today. What I would like to do is sit down at OSCON and
write up a plan for how to tackle memory.
We have (at a least):
1) Short lived, per query.
2) Medium lived, per login session
3) Forever
We also have the issue that some things, like Tables, are not per
session and are cached.
Here is my $0.02...
Table/view/schema object definitions should all be persistent
(forever). This makes it cheap to check if a table exists without
having to hit the disk. An async event notification system should
exist so that a storage engine can inform the server of the creation
of a new table so that the server can request and cache it.
It may be a good idea to keep open storage handlers on a per-session
basis and not shared. Then the problem of sharing/limiting open file
handles is pushed down into the storage engines. Of course, a ceiling
for max number of open handler instances per session is useful... Max
of 63 per connection is reasonable as that is max number of tables in
a join.
Perhaps specify on a per-engine basis what is max number of file
handles that engine may open. Perhaps a small utility helper can be
provided to retrofit this facility like what I did with Amira to
support 32000 "open" file handles.
The current scheme mysqld uses which limits the total number of
handler instances doesn't maximize use of available file handles,
especially with many storage engines which are not file-per-table and
share file handles among multiple handler instances even if they are
file-per-table.
A small added bonus if you do this... some memory managers on some OS
have limited NUMA knowledge and will then try to allocate objects in
memory local or near to the current cpu. As we move to massively
parallel system, we will see more and more NUMA-like architectures. Of
course, the most common NUMA systems today are all those HyperChannel
AMD SMP systems. Having a shared global pool can hurt performance on
them.
Regards,
Antony
I would like to define these out, and create a strategy that is
written down and someone and go to when they ask "how do I allocate
memory?"
And everything should be error checked/type checked.
Until we have a plan though, lets just clean up what we have to make
it a bit easier to deal with when we have a plan.
Cheers,
-Brian
On Jul 11, 2008, at 6:36 AM, Stewart Smith wrote:
On Fri, 2008-07-11 at 12:28 +0100, David LP Travel wrote:
Original 1990ties MySQL allocation always was per query on a
stack. And
when the query finished you reset the stack.
Very fast and very simple. When we added things like stored
procedures
etc that way broke. It looks like drizzle could go back to the
original
way...
/me cheers "talloc! talloc!"
:)
--
Stewart Smith ([EMAIL PROTECTED])
http://www.flamingspork.com/
_______________________________________________
Drizzle mailing list
[EMAIL PROTECTED]
http://lists.tangent.org/mailman/listinfo/drizzle
--
_______________________________________________________
Brian "Krow" Aker, brian at tangent.org
Seattle, Washington
http://krow.net/ <-- Me
http://tangent.org/ <-- Software
_______________________________________________________
You can't grep a dead tree.
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
