As a shameless plug, awhile ago I wrote an asynchronous networking
library called ASync for Java that wraps around NIO, loosely based on
the networking library found in Bamboo, which was in turn inspired by
libasync by David Mazieres. You can find it available at http://
ponyride.sf.net. At the core is a select loop and event queue
management system that services can elect to use -- that is, the
service tells the core what events it would like to schedule or what
sockets it would like to read/write/accept/connect on, and the core
will notify the service when the appropriate action happens. Services
can be dynamically added and removed at run-time, and that services
don't need to be aware of each other. I include two services with it,
ASyncTCP and ASyncUDP, which are connection managers for TCP and UDP
respectively. For each there's a collection of (shared) interfaces
for events, messages, message queues, incoming/outgoing connections,
etc. It's released under a BSD license. Although from what I
understand this is a lot like twisted for Python, but how the hell
was I supposed to know that when I was writing it? ;)
(The goal was, eventually, I was going to write a protocol called
"PonyRide" that would provide TCP-like functionality over UDP --
hence the project name. It would be as a service that uses ASync, and
share the same interfaces. I got fairly far in a unidirectional
implementation, but never found time to finish :( I'd like to one day
soon pick up where I left off...)
- Mike
On Oct 31, 2006, at 11:04 AM, Chaz. wrote:
Alexander,
Thanks for the info on the disk I/O. It certainly explains what I was
seeing. I didn't expect the write to block (unless I had the flush
option set), but I did expect to see the read blocked, which didh't
happen.
Peace,
Chaz
Alexander Pevzner wrote:
Chaz,
in UNIX the disk I/O considered non blocking. It means that disk file
descriptor will be always ready for reading/writing if you will
pass it
to select() or poll(). And although the actual I/O operation may
block,
you will not be able to control this kind of blocking using the
event-driven I/O (multithreaded architecture may help, but may not,
depending on a actual system kernel implementation).
This is not so stupid decision, because 1) disk operations are
usually
backed by the memory cache 2) overlapped disk file I/O looks quite
overcomplicated for a practical usage.
I've heard that Windows allows the reasonable use of the disk file
handles for overlapped I/O, but I'm not 100% sure.
I believe that Python etc follows the UNIX model of life. For
portability across platforms, in particular.
Chaz. wrote:
Sam,
Thanks for the information, things like this always prove the
point. I
was wondering if anyone has information on the use of event
programming
for file I/O? I know in the Python/Twisted world there is a
belief that
event level programming on file i/o is a waste - that you might
just as
well do normal blocking opens/read/write/close since the kernel
will do
most of the heavy lifting. Does any one know if this is true?
Peace,
Chanz
Sam Berlin wrote:
Yes, true polling is very bad, as is any kind of sleeping or
delaying
of receiving events. For a few versions of LimeWire, we had a
Thread.sleep(50) in the selector thread, in order to work around
a few
JDK bugs. This had the side-effect of limiting data transfer in
proportion to the size of the buffer allocated to read the data,
which
ended up being ~120KB/s. Once we removed the sleep and woke up the
selector whenever something became interested in reading, LAN
transfers shot back up to ~5MB/s (or whatever was the saturation
point
of the LAN). Moral of the story: don't artificially limit your
selector.
Sam
On 10/30/06, Daniel Stutzbach <[EMAIL PROTECTED]> wrote:
I think you are confusing polling with calling poll(). Despite
the
name, poll() does not do polling. It blocks until one of a set of
file descriptors becomes ready, then tells you which ones.
select()
does the same thing with slightly different semantics. poll() is
good. Polling, on the other hand, is almost always bad.
On 10/30/06, Lemon Obrien <[EMAIL PROTECTED]> wrote:
"polling" -- polling is a good thing with p2p; LimeWire is
trying to do
polling; it reduces the number of threads...if you do one for
udp,
another
for tcp, and another for network maintance, you already have
three
threads
running; not even counting inteface...or any thread you'll
have to
spawn to
do udp transfers, or searches...
yes i know "polling" is not the attractive way to create a
server...but...in
this case; i believe it's preffered.
just my opinion.
"Chaz." <[EMAIL PROTECTED]> wrote:
I think your are confused about ACE. It might be single
threaded but it
doesn't using polling. Instead it uses event based scheduling
and state
machines. It is a very sophisticated, low overhead system.
Check it out
before "dissing" it.
In the python arena Twisted is a single thread, event passed
framework
and the BitTorrent client is written using it. I have also
written an
application using it that have 5 services running at once and
is a very
sophisticated p2p application (supporting multiple requests at
once).
The reason I mentioned ACE at the start of all this was to
point out
you
can really make a sophisticated, high performance application
without
much effort.
Chaz
Lemon Obrien wrote:
you're speaking to people who have 15 years plus experience...i
persoannly started coding professionally in 92, two years
before the
internet.
abstraction layer. abstraction layer...if it took a whole
team to
create
ACE, Twisted, Whatever,...with years of testing...well; that's
probably
a case where too many cooks are in the kitchen...and after
looking at
ACE architecture...not to be harsh; but it looked
disorganized; and
seemed to suck...plus someone mention "single threaded" p2p and
"single"
threaded systems don't mix.
anyway...i don't believe in following rules laid down by IT
people
who
do nothing but sit in cubicles.
*/Antoine Pitrou /* wrote:
Le jeudi 26 octobre 2006 à 12:28 -0700, Alex Pankratov a écrit :
But average Linux/Windows coder with a good understanding of
language
standards (assuming C/C++) should be able to produce non-UI
abstraction
layer *much* faster than it would take him to learn
something like
ACE.
Wow, it's a joke right?
I can't help thinking how tedious it must be to workaround
all the
various subtleties of each platform's network stack, API,
threading
semantics etc. There is a reason why people decided to write
ACE,
Twisted, apr... in the first place.
Not to mention that software like ACE or Twisted has been in
use and
actively maintained for years, and chances are they make the
right
decisions in a lot of places. Not because their designers are
genious,
but because they have actually been tested and fixed to work
correctly
in a *lot* of situations.
What are the odds that an "average Linux/Windows coder" would be
able to
come up with the right decisions at the first attempt to code an
network
abstraction library? At what cost?
Perhaps "average coder" has a special meaning in your mouth,
because I
can't imagine how your claim can be realistic.
Oh and by saying "Linux/Windows", you already leave out the
BSDs,
MacOS
X, and various other OS flavours (embedded stuff, etc.).
Project leader who pushed for using ACE had no better option
than to suggest purchasing paid support from ACE people.
And how is that a problem exactly? Does your in-house
"average coder"
work for free?
If the same bug had occurred with an in-house developed
library, who
could you have paid to solve the problem?
Answer: nobody, because nobody outside knows your library,
and the
guy
who coded is an "average coder" by your own words, so he
would have a
very hard time debugging bizarre, erratic threading issues.
The very fact that you could find someone to fix that - probably
specific or exotic - problem is highly positive.
regards
Antoine.
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
You don't get no juice unless you squeeze
Lemon Obrien, the Third.
http://www.tamago.us
------------------------------------------------------------------
------
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
You don't get no juice unless you squeeze
Lemon Obrien, the Third.
http://www.tamago.us
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
--
Daniel Stutzbach Computer Science
Ph.D Student
http://www.barsoom.org/~agthorr University
of Oregon
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
