MARK CALLAGHAN wrote:
On Wed, Jul 29, 2009 at 10:00 AM, Jay Pipes<[email protected]> wrote:
MARK CALLAGHAN wrote:
On Wed, Jul 29, 2009 at 3:11 AM, Biping MENG<[email protected]> wrote:
Currently the size of pool of threads is fixed to 8 by default
(sysbenches on more than 8 cores platforms should produce nearly the same
results as 8 cores platform does; from 9th on, cores idle for most of the
time). But I think it really should be easy to set when starting drizzled
(like providing a option, say, --pool-size=XXX ), for the size has a
quite
tight relationship with the number of CPU cores available which varies on
different machines. Thoughts?
Earlier threads about your work mentioned rescheduling threads on any
event that requires a long wait (IO, row lock, ...). Is that part of
your patch? For example, how does Drizzle with this patch behave when
the pool size is 8 and 8 sessions block in InnoDB on the auto
increment lock for a table?
I haven't taken a look at Bipeng's code yet (but I'm excited to :) ). But, I
should point out that Drizzle does not modify any storage engine; we instead
wait for the storage engine developers to improve their plugins and we
incorporate their changes almost immediately.
Not sure if you were asking if Bipeng has modified the InnoDB source code or
not. Just wanted to make sure it was clear where the "dividing lines" are
in Drizzle development :)
I want to know how you plan to resolve some of the problems lists at
the URL below if you intend to run with a few threads per core, or
even with a fixed number of threads.
http://mysqlha.blogspot.com/2009/01/no-new-global-mutexes-and-how-to-make.html
Aww, Mark, you know we're all about making you happy! :)
OK, so here is a partial answer to this part of your query:
"get MySQL to listen on multiple ports. Use the pool to handle requests
that arrive on one port. Don't use the pool to handle requests that
arrive on other ports. We already added code to MySQL to make it listen
on a separate port but only accept connections from accounts with SUPER
or REPL_SLAVE privileges. I don't want replication connections to get
blocked waiting for a thread from the pool."
You will be happy to learn that:
1) The new Drizzle replication system (publisher and subscriber plugins)
do *not* listen on the same port as regular client connections.
Here is part of the GPB Message from /drizzled/message/replication.proto
that you may be interested in:
/*
* Describes a server used in replication. This message
* class is passed in the Publisher::registerSubscriber()
* and Subscriber::subscribe() methods.
*/
message Server
{
/* A unique numeric identifier for this server */
required int32 server_id = 1;
/* A valid IP address */
required string ip_address = 2;
/* Port that this server is listening for replication events */
required uint32 port = 3;
/* Optional name for the server */
optional string name = 4;
}
Underlying the Publisher interface, I use the Listen and ListenHandler
plugins from the wonderful Eric Day to abstract away communication
between the subscriber and publisher. As you can see from the above,
each server in the replication topology has it's own IP address, and
port specifier. FYI, I left the ip_address member as string considering
that IPv6 is real now...
When a Subscriber plugin registers itself with a Publisher, it
constructs a GPB Server message and passes connection information about
itself to the Publisher plugin over the regular Drizzle port connection.
When the publisher receives the Server GPB message, it hands a new
instance of drizzled::command::RegisterSubscriber object to the
Publisher plugin, which then establishes a connection to the subscribing
server using the information passed in the Server message.
After the initial registration, the publisher asks the subscriber (over
its newly created Listen interface and separate ports), for information
about the subscriber's known state of the publisher. The subscriber
sends back a GPB SubscriberManifest message describing its known state:
/*
* A subscriber manifest describes the state of
* a subscriber in replication. This message class
* is passed in the replication API in the
* drizzled::plugin::Publisher's findSubscriptionState() call
*/
message SubscriberManifest
{
/* Only need to pass the ID...not the whole Server message */
required int32 server_id = 1;
/*
* The timestamp of the last time this subscriber
* communicated with the publisher
*/
required uint64 last_applied_timestamp = 2;
/* The globally unique transaction ID of last applied command */
required uint64 last_applied_transaction_id = 3;
}
This is just a glimpse of the new interface. I'll be pushing the (very
nascent) work up to LP very shortly.
Cheers, and as always, I look forward to your comments!
Jay
And today's example of a connection pileup in MySQL is
http://bugs.mysql.com/bug.php?id=46459
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
