Re: [Flightgear-devel] Nasal Garbage Collector (was Stuttering at 1 Hz rate)

[Robert]

>
> In the case of nasal, I believe the garbage collection pass must be done in
> a single atomic step, otherwise it would leave the heap in
> an inconsistent state and adversely affect the scripts.
>

I completely agree with you. Now I understand the whole thing much better.
I originally thought about a non-mutual-exclusive thread ( completely
asynchronous ), but then it would potentially leave a mess in Nasal heap,
right?


It might be possible to use a GC module from a GPL:d Java vm or
> similar.
>

That's a good idea.
As Curtis Olson said the GC should have an atomic step character (or don't
produce too much delay).

Does anybody of you know how garbage collectors of Java and Python and other
GPL'd script engines compare to each other? (atomic step character)
--
Forrester Wave Report - Recovery time is now measured in hours and minutes
not days. Key insights are discussed in the 2010 Forrester Wave Report as
part of an in-depth evaluation of disaster recovery service providers.
Forrester found the best-in-class provider in terms of services and vision.
Read this report now!  http://p.sf.net/sfu/ibm-webcastpromo___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] Nasal Garbage Collector (was Stuttering at 1 Hz rate)

[Anders Gidenstam]

On Sun, 10 Apr 2011, Arnt Karlsen wrote:

>> On Sun, Apr 10, 2011 at 2:15 PM, Anders Gidenstam wrote:
>
>>> The small changes needed to add a separate GC thread are available
>>> here: http://www.gidenstam.org/FlightGear/misc/test/sg-gc-2.diff
>>> http://www.gidenstam.org/FlightGear/misc/test/fg-gc-1.diff
>
> ..how do I apply these 2 in my git tree?

Hi,

With patch -p1 < ../the/patch/file.diff  from the top-level of the 
respective source directories.

However, the result is very similar to the standard FG since even if the 
GC is run on a different thread it still takes the same amount of time 
and while it is running the main thread will be blocked waiting for it as 
soon as it tries to run some Nasal code.

Note that there is a spurious change in simgear/nasal/code.c that makes
variable creation without the var keyword illegal. I use that to make sure 
my Nasal scripts are clean in this regard but there are plenty of scripts 
in fgdata that do not pass that test (and, well, they are still valid 
Nasal). I'll update the patch file ASAP but please revert that file to 
the standard one in the meanwhile.

Cheers,

Anders
-- 
---
Anders Gidenstam
WWW: http://www.gidenstam.org/FlightGear/

--
Xperia(TM) PLAY
It's a major breakthrough. An authentic gaming
smartphone on the nation's most reliable network.
And it wants your games.
http://p.sf.net/sfu/verizon-sfdev
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] Nasal Garbage Collector (was Stuttering at 1 Hz rate)

[Arnt Karlsen]

On Sun, 10 Apr 2011 15:03:47 -0500, Curtis wrote in message 
:

> Just for the record, I would be pretty strongly opposed to adding a
> thread for no particular benefit.  In fact I think the threshold of
> gained benefit needs to be pretty high to add another thread to the
> code.  Threads might seem simple at first, but they can hide nasty
> bugs that are almost impossible to find.  And even if the person
> submitting the code has validated everything and it's perfect --
> inevitably, someone will come and make changes somewhere in the code
> that affect the thread in some subtle way and we are back to having
> almost impossible to find bugs.  That doesn't mean we can't make a
> big mess of non-threaded code too, but threads are 100 times easier
> to screw up without realizing it until weeks or months later ... and
> when there is a problem, the concurrency of threads can make it
> extremely difficult to isolate where the bug actually is.

..ok, I volunteer for this stylish crash-n-burn 
so we can get back to the drawing board. ;o) 


> Regards,
> 
> Curt.
> 
> 
> On Sun, Apr 10, 2011 at 2:15 PM, Anders Gidenstam wrote:

> > The small changes needed to add a separate GC thread are available
> > here: http://www.gidenstam.org/FlightGear/misc/test/sg-gc-2.diff
> > http://www.gidenstam.org/FlightGear/misc/test/fg-gc-1.diff

..how do I apply these 2 in my git tree?

-- 
..med vennlig hilsen = with Kind Regards from Arnt Karlsen
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.

--
Xperia(TM) PLAY
It's a major breakthrough. An authentic gaming
smartphone on the nation's most reliable network.
And it wants your games.
http://p.sf.net/sfu/verizon-sfdev
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] Nasal Garbage Collector (was Stuttering at 1 Hz rate)

[Curtis Olson]

Just for the record, I would be pretty strongly opposed to adding a thread
for no particular benefit.  In fact I think the threshold of gained benefit
needs to be pretty high to add another thread to the code.  Threads might
seem simple at first, but they can hide nasty bugs that are almost
impossible to find.  And even if the person submitting the code has
validated everything and it's perfect -- inevitably, someone will come and
make changes somewhere in the code that affect the thread in some subtle way
and we are back to having almost impossible to find bugs.  That doesn't mean
we can't make a big mess of non-threaded code too, but threads are 100 times
easier to screw up without realizing it until weeks or months later ... and
when there is a problem, the concurrency of threads can make it extremely
difficult to isolate where the bug actually is.

Regards,

Curt.


On Sun, Apr 10, 2011 at 2:15 PM, Anders Gidenstam wrote:

> On Sun, 10 Apr 2011, Curtis Olson wrote:
>
> > I'm not an expert in nasal garbage collection, but I think the problem is
> > that garbage collection is not something we can divide up into chunks
> (which
> > is essentially what threading would do.)  In addition, threading adds a
> lot
> > of potential order dependent bugs.  I've got a whole soapbox speech on
> > threading I could deliver which I would summarize as "only if absolutely
> no
> > other solution is viable."
> >
> > In the case of nasal, I believe the garbage collection pass must be done
> in
> > a single atomic step, otherwise it would leave the heap in
> > an inconsistent state and adversely affect the scripts.
>
> There are algorithms for incremental and/or concurrent and/or parallel
> garbage collection out there. They most likely not easy to implement and
> as far as I have seen so far would require (at least for concurrent and
> /or parallel GC) all writes of pointers to the Nasal heap (and possibly
> reads) to be redirected via wrapper functions (also known as
> (GC) read/write barriers).
>
> This will not be an easy task but in my opinion it would be a promising
> option. It might be possible to use a GC module from a GPL:d Java vm or
> similar.
>
> Btw, just running the normal (mutually exclusive) Nasal GC in another
> thread than the main loop is not hard - but since it is mutually exclusive
> to executing Nasal functions it doesn't help much when it comes to
> reducing the worst case latency.
>
> The small changes needed to add a separate GC thread are available here:
> http://www.gidenstam.org/FlightGear/misc/test/sg-gc-2.diff
> http://www.gidenstam.org/FlightGear/misc/test/fg-gc-1.diff
>
> Cheers,
>
> Anders
> --
> ---
> Anders Gidenstam
> WWW: http://www.gidenstam.org/FlightGear/
>
>
> --
> Xperia(TM) PLAY
> It's a major breakthrough. An authentic gaming
> smartphone on the nation's most reliable network.
> And it wants your games.
> http://p.sf.net/sfu/verizon-sfdev
> ___
> Flightgear-devel mailing list
> Flightgear-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/flightgear-devel
>



-- 
Curtis Olson:
http://www.atiak.com - http://aem.umn.edu/~uav/
http://www.flightgear.org -
http://gallinazo.flightgear.org
--
Xperia(TM) PLAY
It's a major breakthrough. An authentic gaming
smartphone on the nation's most reliable network.
And it wants your games.
http://p.sf.net/sfu/verizon-sfdev___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] Nasal Garbage Collector (was Stuttering at 1 Hz rate)

[Anders Gidenstam]

On Sun, 10 Apr 2011, Curtis Olson wrote:

> I'm not an expert in nasal garbage collection, but I think the problem is
> that garbage collection is not something we can divide up into chunks (which
> is essentially what threading would do.)  In addition, threading adds a lot
> of potential order dependent bugs.  I've got a whole soapbox speech on
> threading I could deliver which I would summarize as "only if absolutely no
> other solution is viable."
>
> In the case of nasal, I believe the garbage collection pass must be done in
> a single atomic step, otherwise it would leave the heap in
> an inconsistent state and adversely affect the scripts.

There are algorithms for incremental and/or concurrent and/or parallel 
garbage collection out there. They most likely not easy to implement and
as far as I have seen so far would require (at least for concurrent and 
/or parallel GC) all writes of pointers to the Nasal heap (and possibly 
reads) to be redirected via wrapper functions (also known as 
(GC) read/write barriers).

This will not be an easy task but in my opinion it would be a promising 
option. It might be possible to use a GC module from a GPL:d Java vm or 
similar.

Btw, just running the normal (mutually exclusive) Nasal GC in another 
thread than the main loop is not hard - but since it is mutually exclusive 
to executing Nasal functions it doesn't help much when it comes to 
reducing the worst case latency.

The small changes needed to add a separate GC thread are available here:
http://www.gidenstam.org/FlightGear/misc/test/sg-gc-2.diff
http://www.gidenstam.org/FlightGear/misc/test/fg-gc-1.diff

Cheers,

Anders
-- 
---
Anders Gidenstam
WWW: http://www.gidenstam.org/FlightGear/

--
Xperia(TM) PLAY
It's a major breakthrough. An authentic gaming
smartphone on the nation's most reliable network.
And it wants your games.
http://p.sf.net/sfu/verizon-sfdev
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] Nasal Garbage Collector (was Stuttering at 1 Hz rate)

[Curtis Olson]

On Sun, Apr 10, 2011 at 1:43 PM, Robert wrote:

> As discussed in "Stuttering at 1 Hz rate" we now know that regular and
> unpleasant stuttering is caused by Nasals garbage collector.
> So I thought about possibilities to improve it.
> What if we could decouple the following function as a separate thread, so
> that it runs *asynchronously* from the main thread?
> This way it would not interfere (or much less) with the main thread and our
> fps would be more consistent.
>
> This is the function causing the jitter:
> In "simgear/nasal/gc.c"
> static void garbageCollect()
>
> The thread will need to share some of the global variables from the main
> thread.
> Unfortunately I have no experience with thread programming. But I'm sure
> there are some skilled programmers in the FG-community.
>

Danger danger Will Robinson! :-)

I'm not an expert in nasal garbage collection, but I think the problem is
that garbage collection is not something we can divide up into chunks (which
is essentially what threading would do.)  In addition, threading adds a lot
of potential order dependent bugs.  I've got a whole soapbox speech on
threading I could deliver which I would summarize as "only if absolutely no
other solution is viable."

In the case of nasal, I believe the garbage collection pass must be done in
a single atomic step, otherwise it would leave the heap in
an inconsistent state and adversely affect the scripts.

Regards,

Curt.
-- 
Curtis Olson:
http://www.atiak.com - http://aem.umn.edu/~uav/
http://www.flightgear.org -
http://gallinazo.flightgear.org
--
Xperia(TM) PLAY
It's a major breakthrough. An authentic gaming
smartphone on the nation's most reliable network.
And it wants your games.
http://p.sf.net/sfu/verizon-sfdev___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

[Flightgear-devel] Nasal Garbage Collector (was Stuttering at 1 Hz rate)

[Robert]

As discussed in "Stuttering at 1 Hz rate" we now know that regular and
unpleasant stuttering is caused by Nasals garbage collector.
So I thought about possibilities to improve it.
What if we could decouple the following function as a separate thread, so
that it runs *asynchronously* from the main thread?
This way it would not interfere (or much less) with the main thread and our
fps would be more consistent.

This is the function causing the jitter:
In "simgear/nasal/gc.c"
static void garbageCollect()

The thread will need to share some of the global variables from the main
thread.
Unfortunately I have no experience with thread programming. But I'm sure
there are some skilled programmers in the FG-community.
--
Xperia(TM) PLAY
It's a major breakthrough. An authentic gaming
smartphone on the nation's most reliable network.
And it wants your games.
http://p.sf.net/sfu/verizon-sfdev___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel