Hi Emmanuel, 1) One Reactor is preferable. Easier to manage, easier to code, meets all the requirements.
2) Event Processing: Use non-blocking sockets. Accept, connect, read and write won't hang each other up. I have some code that does this. I'll get it and attach to a subsequent email. It might take a couple days. Regards, Chad Beaulac Objective Solutions, Inc. www.objectivesolutions.com [email protected] On Oct 1, 2012, at 6:45 AM, Emmanuel Lécharny wrote: > Hi guys, some thoughts about the TCP server. feel free to comment. > > TCP server MINA 3 > ----------------- > > As we are reworking the server part of MINA, we can review the current > architecture. There are a few problems we can address. > > 1) One vs Many selectors > In MINA 2, we do have at least 2 selectors used : > - one for the OP_ACCEPT event > - One or many for the OP_READ/OP_WRITE > > I don't think that it makes a lot of sense to force such a choice. IMO, we > could perfectly start with one single selector to handle all the events, > assuming we are fast enough to process them. Otherwise, we can also configure > the server to use more selectors, if needed. > > Typically, the acceptor selector will just deal with incoming new > connections, and the created channel will be registred on an IoProcessor > selector on MINA 2. We could register this channel on the acceptor selector. > > In any case, if we do create more than one selector, I suggest that the first > one would always handle OP_ACCEPT events, and that's all. > > The general algorithm will look likes : > > signal started > > while true > nbSelect = select( delay ) > > if nbSelect != 0 > for each selectionKey do > case isAccept // Only if the selector is the first one, otherwise we > don't need to heck this case > create session > > case isRead > process read > > case isWrite > process write > done > > if dispose // In order to stop the server > process dispose > break > done > > The key is to start all the selector workers *before* accepting any > connection, otherwise we may lose some messages. One more thing : each > selector should signal that there have started before entering in the loop, > so that the caller don't have to wait a random period of time for the > selectors to be started. > > 2) Events processing > Now, in order to limit the number of selectors to use, we need to limit the > time it takes to process the read/write/accept events. But even if we have > many selectors, we should try to minimize the risk that one selector is > blocked by a single session blocked somewhere while doing some heavy > prcoessing, as it will block all the other sessions. > > Having more than a selector is one way to mitigate this issue : as we have > many threads (one per selector), we spread the loads on as many threads. > Another solution would be to use an executor in charge of processing the > events, with a queue between the selector and the executor, queue that is > used to process the events as fast as possible on the selector (this is > really important for UDP, as we don't want to lose messages simply because > the OS buffer is full). > > The problem is that we are just not solving the problem of a rogue service > that block a thread for a long time (if we use a limited size executor), or > we may end with so many threads that it may kill the server. But anyway, it > sounds like a better solution, as incoming events that won't require a long > processing will be favored in the long term. > > 3) Write processing > This is a complex issue too : we may not be able to push all the data we want > into the socket, if it becoms full (or was already full). In this case, we > will have to store the data in a queue. The following algorithm describe this > situation and a proposal to solve it > > if there are some data in the writeQueue > then > // We need to enqueue the data, and write the head of the queue > enqueue data > > // Now, we shoudl try to write as much of the queue as we can > while ( queue not empty) > do > poll the data from the queue > nbWritten = channel.write( remaining data ) // We may have already > written some part of the head data > > if nbWritten < data.remaining > then > // the socket is already full, set its OP_WRITE interest, and > don't touch the queue > selectionKey.ops = OP_WRITE > break // We can stop, the socket is full anyway > else > pop the data from the queue // We just remove the head, and > continue with the next data > done > > if queue is empry > then > selectionKeys.ops = ! OP_WRITE // We remoe this flag, it's > ueless now > else > nbWritten = channel.write( remaining data ) // We may have already > written some part of the head data > > if nbWritten < data.remaining > then > // the socket is already full, set its OP_WRITE interest, and add > the data in the queue > selectionKey.ops = OP_WRITE > writeQueue.add( remaining data ) > > 4) Read/Write order > MINA 2 was processing all the reads first, then all the writes. This is not > necessarilly a good idea. We may rather process read and write on a per > session basis. It's perfectly possible that a session has to process some > reads and some write at the same time. Waiting for all the reads to be > processed create some memory load, as we will have to wait until all the > reads are done, storing all the data to be written in the mean time, until we > are done with the last session. > > 5) MINA events > Atm, we are processing the following events : > - messageReceived (when we have read some data from the channel) > - messageSent (when a user message has been sentcompletely) > - exceptionCaugth (when an exception has been caught) > - sessionIdle (if the session has not received or sent anything for a period > of time) > - sessionCreated (when a new session is created) > - sessionOpened > - sessionClosed (when the session has been closed) > - filterWrite > - filterClose > > The sessionOpened, filterWrite and filterClose are a bit mysterious. I don't > see why we need a special event SessionOpened when we alreayd have the > sessionCreated event. That may seems we *may* create a session, but not > accept any incoming or outgoing messages for this session, until it's > initialized. I'm not sure this is necessary. > > The two other events are probably some YAGNY iplementation... > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com >
