[Flightgear-devel] network IO issues

2010-03-04 Thread John Denker 
I observe the following.  Lots of irrelevant stuff snipped:

Setup:

  fgfs --httpd=5400  
  lynx -source -head http://localhost:5400/sim/intl/locale/strings/

Result:

Making HTTP connection to localhost:5400
Alert!: Unexpected network read error; connection aborted.

WARNING: netBufferChannel: output buffer overflow!

=

This is 100% reproducible chez moi.  I looked at the code,
and this result is entirely predictable, given that the
page that should have been returned is larger than the
buffer the plib wants it to fit in.

There seems to be a deep-seated problem in the design of
plib.  I don't see any way of fixing this without making
substantial changes in multiple places.

Hypotheses that should be considered include
 a) rewriting httpd.cxx to use SGIOChannel::readline, or
 b) switching to something new, such as boost::asio tcp::acceptor

Hypotheses that should IMHO not be seriously considered
include:
 x) using the Berkeley sockets directly -- notoriously 
   non-portable
 y) fussing with plib.   This would be a lot of work, and
   would be a dead end anyway.  Any day that you can 
   remove a plib dependency is a good day.

The boost stuff seems a bit clumsy, but it should be 
portable, and it is cleaner than the simgear readline 
(which is what FGMetar uses) which is in turn much much 
cleaner than the plib stuff.

==

The question of asynchronous IO and thread safety must be 
dealt with.  FGMetar starts a thread of its own, so it
can deal with network IO without blocking the main loop
of FG.  I don't entirely understand what plib does, but 
whatever it is, it doesn't work right.

On 02/08/2010 09:18 AM, Erik Hofman wrote:

 FlightGear is not thread safe, simply because it uses OpenGL. Parts of 
 it can work with threads though, and some already do.

The question for today is:

  Is reading and writing the property tree one of the
  safe parts?  Is a process that does nothing except
  read and write the tree always/sometimes/never
  thread safe?

I can imagine getting into trouble if somebody sets a
listener on some property, and then calls opengl from
the listener, whereby writing to the property tree 
becomes thread-unsafe.  Or are all the listener callbacks
somehow carefully serialized?

There are lots of ways of serializing the network traffic,
but it would be a shame to do that if not necessary.  If
the property tree is thread safe, that makes everything
simpler and better.

==

Bottom line:  Does anybody know the right way to make
this work?

--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] network IO issues

2010-03-04 Thread Tim Moore 
On Thu, Mar 4, 2010 at 10:19 PM, John Denker j...@av8n.com wrote:
...


 The question of asynchronous IO and thread safety must be
 dealt with.  FGMetar starts a thread of its own, so it
 can deal with network IO without blocking the main loop
 of FG.  I don't entirely understand what plib does, but
 whatever it is, it doesn't work right.

 On 02/08/2010 09:18 AM, Erik Hofman wrote:

  FlightGear is not thread safe, simply because it uses OpenGL. Parts of
  it can work with threads though, and some already do.

 The question for today is:

  Is reading and writing the property tree one of the
  safe parts?  Is a process that does nothing except
  read and write the tree always/sometimes/never
  thread safe?

 It depends. The tree part of the property tree is not thread safe at all.
If a node is added to the property tree in one thread while another thread
is parsing a path in the property tree to find a value, the second thread
can see an outdated / corrupted version of the property tree. New property
tree nodes are probably not added that often at runtime, and some code (but
not all) keeps pointers directly to the property nodes it cares about.

Getting and setting property values are not necessarily atomic operations,
particularly where string values are concerned. So all sorts of races are
possible there too. There's no global lock on the property tree and no local
locks on nodes in the tree either.

 I can imagine getting into trouble if somebody sets a
 listener on some property, and then calls opengl from
 the listener, whereby writing to the property tree
 becomes thread-unsafe.  Or are all the listener callbacks
 somehow carefully serialized?

No, and the callbacks will be run in whatever thread sets the property.

OpenGL's thread safety is really neither here nor there. OpenGL calls are
restricted to a few places, and certainly can't happen in a general property
listener. OpenGL calls could end up running in several different possible
threads or even simultaneously if there is more than one graphics context.


 There are lots of ways of serializing the network traffic,
 but it would be a shame to do that if not necessary.  If
 the property tree is thread safe, that makes everything
 simpler and better.

 ==

 Bottom line:  Does anybody know the right way to make
 this work?

 Some of us have thrown around the idea of using the property system as
general inter-thread communications mechanism, but there's nothing concrete.
The approach I've thought of is to double-buffer the values in the property
tree, so that readers see a consistent value for each property. The
challenge is doing this without duplicating the entire property tree every
frame, and also supporting property tree writes by more than one thread.

Tim
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel