Re: [Flightgear-devel] Fixing fgfs-construct crashes
I think the solution to this whole issue is to bring fgfs-construct .btg
generation closer to how the genapt works - keeping the material
information around with each poly through the clipping process.
Hi Peter,
I thought that was already the case? I was fooling around the other day,
modifying the BTG loader to use a material definition for each feature
(fan/strip/triangle), thus loading a different texture from the texture-set
instead of only one per tile. Really the only modification I made was to store
the geometry along with the material while loading, instead of loading a
material and then storing all features with that material in the same bin.
That seemed to bring the performance down to a crawl (to be expected since
otherwise this code would probably have been there from day 1). If it's any
use, I'll attach here a diff of my experiments.
Cheers,
Adrian
diff --git a/simgear/scene/tgdb/obj.cxx b/simgear/scene/tgdb/obj.cxx
index d080ff3..ba56c0e 100644
--- a/simgear/scene/tgdb/obj.cxx
+++ b/simgear/scene/tgdb/obj.cxx
@@ -68,7 +68,7 @@
using namespace simgear;
-typedef std::mapstd::string,SGTexturedTriangleBin SGMaterialTriangleMap;
+typedef std::multimapstd::string,SGTexturedTriangleBin SGMaterialTriangleMap;
typedef std::listSGLightBin SGLightListBin;
typedef std::listSGDirectionalLightBin SGDirectionalLightListBin;
@@ -190,7 +190,7 @@ struct SGTileGeometryBin {
}
static void
- addTriangleGeometry(SGTexturedTriangleBin triangles,
+ addTriangleGeometry(SGMaterialTriangleMap triangleMap, std::string material_name,
const std::vectorSGVec3d vertices,
const std::vectorSGVec3f normals,
const std::vectorSGVec2f texCoords,
@@ -203,11 +203,11 @@ struct SGTileGeometryBin {
// If the normal indices do not match, they should be inmplicitly
// the same than the vertex indices. So just call ourselves again
// with the matching index vector.
- addTriangleGeometry(triangles, vertices, normals, texCoords,
+ addTriangleGeometry(triangleMap, material_name, vertices, normals, texCoords,
tris_v, tris_v, tris_tc, tcScale);
return;
}
-
+SGTexturedTriangleBin triangles;
for (unsigned i = 2; i tris_v.size(); i += 3) {
SGVertNormTex v0;
v0.vertex = toVec3f(vertices[tris_v[i-2]]);
@@ -223,10 +223,11 @@ struct SGTileGeometryBin {
v2.texCoord = getTexCoord(texCoords, tris_tc, tcScale, i);
triangles.insert(v0, v1, v2);
}
+triangleMap.insert(std::pairstd::string, SGTexturedTriangleBin (material_name, triangles));
}
static void
- addStripGeometry(SGTexturedTriangleBin triangles,
+ addStripGeometry(SGMaterialTriangleMap triangleMap, std::string material_name,
const std::vectorSGVec3d vertices,
const std::vectorSGVec3f normals,
const std::vectorSGVec2f texCoords,
@@ -239,11 +240,11 @@ struct SGTileGeometryBin {
// If the normal indices do not match, they should be inmplicitly
// the same than the vertex indices. So just call ourselves again
// with the matching index vector.
- addStripGeometry(triangles, vertices, normals, texCoords,
+ addStripGeometry(triangleMap, material_name, vertices, normals, texCoords,
strips_v, strips_v, strips_tc, tcScale);
return;
}
-
+SGTexturedTriangleBin triangles;
for (unsigned i = 2; i strips_v.size(); ++i) {
SGVertNormTex v0;
v0.vertex = toVec3f(vertices[strips_v[i-2]]);
@@ -262,10 +263,11 @@ struct SGTileGeometryBin {
else
triangles.insert(v0, v1, v2);
}
+triangleMap.insert(std::pairstd::string, SGTexturedTriangleBin (material_name, triangles));
}
static void
- addFanGeometry(SGTexturedTriangleBin triangles,
+ addFanGeometry(SGMaterialTriangleMap triangleMap, std::string material_name,
const std::vectorSGVec3d vertices,
const std::vectorSGVec3f normals,
const std::vectorSGVec2f texCoords,
@@ -278,11 +280,11 @@ struct SGTileGeometryBin {
// If the normal indices do not match, they should be implicitly
// the same than the vertex indices. So just call ourselves again
// with the matching index vector.
- addFanGeometry(triangles, vertices, normals, texCoords,
+ addFanGeometry(triangleMap, material_name, vertices, normals, texCoords,
fans_v, fans_v, fans_tc, tcScale);
return;
}
-
+SGTexturedTriangleBin triangles;
SGVertNormTex v0;
v0.vertex = toVec3f(vertices[fans_v[0]]);
v0.normal = normals[fans_n[0]];
@@ -299,6 +301,7 @@ struct SGTileGeometryBin {
triangles.insert(v0, v1, v2);
v1 = v2;
}
+triangleMap.insert(std::pairstd::string, SGTexturedTriangleBin (material_name, triangles));
}
SGVec2f getTexCoordScale(const
Re: [Flightgear-devel] Fixing fgfs-construct crashes
I think the solution to this whole issue is to bring fgfs-construct .btg generation closer to how the genapt works - keeping the material information around with each poly through the clipping process. Ignore my previous e-mail on this issue, I misread the each poly part. Adrian -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Hello Maxime, I had also noticed more instances of the grey poly with clipper than GPC. As an experiment, on Saturday, I spent several hours gutting fgfs-construct down to the bones. Although the grey polys didn't disappear completely, I got Clipper producing as good results as GPC did. The areas I still see problems are on tile boundaries, and osm roads. I'm at work right now, but will download your tarball tonight to if it's useful. I have also been trying to reproduce the issue on the simplest possible case. This could save some time. I think the grey polys are the default material. I was going to verify this tonight by assigning bogus materials to some well known land-class. I think the solution to this whole issue is to bring fgfs-construct .btg generation closer to how the genapt works - keeping the material information around with each poly through the clipping process. This is a necessary requirement to be able to generate good texture coordinates for linear data as well. I think I will work on this in parallel with the texture mapped roads / streams. Pete On Sun, Nov 20, 2011 at 4:31 PM, Maxime Guillaud m...@mguillaud.net wrote: Hi Peter, Any news on the topic of the gray polygons ? I have been experimenting more with this, and I can confirm that it occurs only when clipper is enabled. If this can help, I have isolated a simple dataset (much simpler than last time) that triggers the bug. It is just one landmass polygon with a handful of vertices. This data is here: http://www.mguillaud.net/fg/2794338/2794338.tgz (I also include the fitted elevation, since I don't know if this data interacts with the bug). The bug occurs with fgfs-construct built from the code in papillon's tg850 branch (i.e. with clipper), with or without my attempts at adjusting the epsilon constants that can be found here and there in the code. I run fgfs-construct --work-dir=[..] --output-dir=[...] --tile-id=2794338 SRTM-3 Landmass , and the gray area is around -9.40025/51.5005 degrees. Any insight on this issue is appreciated... Maxime On Sat, 12 Nov 2011 21:52:56 -0500 Peter Sadrozinski psadrozin...@gmail.com wrote: I have verified that the problem lies in using clipper. Using the same simple data with GPC works. It appears clipper can sometimes produce clockwise winded polys. I'm working on a fix now. Pete On Sat, Nov 12, 2011 at 7:59 PM, Peter Sadrozinski psadrozin...@gmail.comwrote: Hi Martin, I've been reluctant to move to the official repository mostly because of very little understanding of GIT. I'm a bit more confident, now, so I don't see it as a big problem. I think most of the work we are doing (alternate clipping library, 850 format) should be considered experimental, however. I'm pretty sure we want to keep the main branch concentrated on fixing problems with detailed landclass. We seem to be breaking terragear pretty good as of late :) Maxime: I have an experiment for you to try - could you take the UFO under those grey sections of scenery, and look up at them? I think the normals have swapped. Chris mentioned this happening with the skirt around the airport, and I hadn't seen it until a recent update. Looks like something broke recently. Pete On Sat, Nov 12, 2011 at 2:38 PM, Martin Spott martin.sp...@mgras.net wrote: Hi Maxime and others, Maxime Guillaud wrote: You can find my code here [...] I'm just starting to recover from a couple of _really_ tight days (including a nice PostgreSQL conference PGConf.DE where I gave a talk about our geodata collection and in-database processing), therefore I'm not yet ready to provide comprehensive responses. Anyhow there's one point I'd like to emphasize: I know it's really cool to maintain private source repositories ;-) but for increasing the overall success of building FlightGear scenery I'd think it would be really beneficial to keep the various development efforts in close relation and sync. Thus I'd invite everyone who's serious about improving the TerraGear toolchain to create a branch in the main repository in order to develop their specific features/changes there - not only but also because this would simplify tracking of the various changes. The former terragear-cs main repository at MapServer was already open to (almost) everybody who asked and now that it's moved over to Gitorious there's really no more excuse not to create branches inside the main repo :-) Best regards, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Hi Peter,
Any news on the topic of the gray polygons ? I have been experimenting more
with this,
and I can confirm that it occurs only when clipper is enabled.
If this can help, I have isolated a simple dataset (much simpler than last
time) that
triggers the bug. It is just one landmass polygon with a handful of vertices.
This data
is here: http://www.mguillaud.net/fg/2794338/2794338.tgz
(I also include the fitted elevation, since I don't know if this data interacts
with the
bug). The bug occurs with fgfs-construct built from the code in papillon's
tg850 branch
(i.e. with clipper), with or without my attempts at adjusting the epsilon
constants that
can be found here and there in the code.
I run fgfs-construct --work-dir=[..] --output-dir=[...] --tile-id=2794338
SRTM-3
Landmass , and the gray area is around -9.40025/51.5005 degrees.
Any insight on this issue is appreciated...
Maxime
On Sat, 12 Nov 2011 21:52:56 -0500
Peter Sadrozinski psadrozin...@gmail.com wrote:
I have verified that the problem lies in using clipper.
Using the same simple data with GPC works. It appears clipper can
sometimes produce clockwise winded polys.
I'm working on a fix now.
Pete
On Sat, Nov 12, 2011 at 7:59 PM, Peter Sadrozinski
psadrozin...@gmail.comwrote:
Hi Martin,
I've been reluctant to move to the official repository mostly because of
very little understanding of GIT.
I'm a bit more confident, now, so I don't see it as a big problem.
I think most of the work we are doing (alternate clipping library, 850
format) should be considered experimental, however. I'm pretty sure we
want to keep the main branch
concentrated on fixing problems with detailed landclass.
We seem to be breaking terragear pretty good as of late :)
Maxime: I have an experiment for you to try - could you take the UFO under
those grey sections of scenery, and look up at them? I think the normals
have swapped.
Chris mentioned this happening with the skirt around the airport, and I
hadn't seen it until a recent update. Looks like something broke recently.
Pete
On Sat, Nov 12, 2011 at 2:38 PM, Martin Spott martin.sp...@mgras.netwrote:
Hi Maxime and others,
Maxime Guillaud wrote:
You can find my code here [...]
I'm just starting to recover from a couple of _really_ tight days
(including a nice PostgreSQL conference PGConf.DE where I gave a talk
about our geodata collection and in-database processing), therefore I'm
not yet ready to provide comprehensive responses. Anyhow there's one
point I'd like to emphasize:
I know it's really cool to maintain private source repositories ;-)
but for increasing the overall success of building FlightGear
scenery I'd think it would be really beneficial to keep the various
development efforts in close relation and sync. Thus I'd invite
everyone who's serious about improving the TerraGear toolchain to
create a branch in the main repository in order to develop their
specific features/changes there - not only but also because this would
simplify tracking of the various changes.
The former terragear-cs main repository at MapServer was already open
to (almost) everybody who asked and now that it's moved over to
Gitorious there's really no more excuse not to create branches inside
the main repo :-)
Best regards,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--
--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
--
sent from my armchair
--
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Jason Cox wrote: With GRASS how ever there is no doco and so it would take people such as myself a long time to come up to speed. Nobody will force you to use a GRASS-based Scenery toolchain, especially not if you're happy with the current one. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
I am all for changing the generation tools but have one issue with the use of GRASS and that is how to use it. Currently there is some good doco's on building scenery via the current tools and also some scripts that people have put together to automate all or part of it. With GRASS how ever there is no doco and so it would take people such as myself a long time to come up to speed. Would some be able to supply a nice howto doco for this process that even I could follow? I think we also need some doco covering how to setup a PostGIS setup with TG for building as i have several TB of GIS data that I use (under CC License ) and think that PostGIS might speed up my processing times Jason On Sun, 2011-11-13 at 10:58 +, Martin Spott wrote: Curtis Olson wrote: So this all sounds good (I think) except we now have to compute a point inside each material region. Easy, right? Well, except that all the published algorithms can tell you if a random point is inside a polygon or not, but they don't tell you how to manufacture a point that is guaranteed to be inside a polygon. Well, there _is_ OpenSource software which is capable of reliably creating points in polygons (GRASS GIS, just to present a prominent example, they call them centroids) and terragear-cs did have such a thing in the past decade, but Ralf Gerlich finally decided to back it out again because later processing steps in TerraGear were designed to not having to deal with certain corner-cases for valid ! points in polygons. For more information, read thie fine comment in this diff: http://mapserver.flightgear.org/git/gitweb.pl?p=terragear-cs;a=commitdiff;h=8e0e2b6cef93ac689f611e93ca6250d33f243fbe;hp=e813c093fc444d9c36bb992768e37a1abb8c47d4 Therefore I'm still in favour of the long-term idea to drop the entire poly-preprocessing from TerraGear and, instead, add a frontend to read the entire land cover and roads from a GRASS database. GRASS is capable of doing all the clipping and sliver removal (v.clean tool=rmsa -c) in a topologically consistent and reliable way. Cheers, Martin. -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Peter Sadrozinski psadrozin...@gmail.com wrote: I would experiment with removing the code that removes the node. You'll be trading one error for another, but it may help track this down. Actually, this is exactly what one of my commits is doing: https://gitorious.org/~maximeguillaud/papillon81/maxime-terragear-cs/commit/81a78602e09b67edd58b2cedacf09079239b0177 Bad nodes are detected but not removed. I put a minimal set of data that triggers the bug online here: http://www.mguillaud.net/fg/2794354/polygons-2794354.tgz Download it and run fgfs-construct with the following arguments: fgfs-construct --work-dir=. --output-dir=/storage2/work_europe/tmp/output --tile-id=2794354 Landmass SRTM-3 Road6.test You can have a look at what I get when I run my modified version of construct on this data here: http://www.mguillaud.net/fg/2794354/log-2794354.txt Maxime: I have an experiment for you to try - could you take the UFO under those grey sections of scenery, and look up at them? I think the normals have swapped. From below it looks... black. If you want to take a look for yourself, here are the terrain files: http://www.mguillaud.net/fg/2794354/terrain-2794354.tgz Just navigate to 51.751 lat/-9.333 lon. Maxime -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Hi Martin,
Martin Spott martin.sp...@mgras.net wrote:
I know it's really cool to maintain private source repositories ;-)
but for increasing the overall success of building FlightGear
scenery I'd think it would be really beneficial to keep the various
development efforts in close relation and sync.
Same as Peter, I was reluctant to do it so far since I have no prior experience
with Git,
and given the complexity of the tool, the probability of breaking something is
just too
high.
Thus I'd invite
everyone who's serious about improving the TerraGear toolchain to
create a branch in the main repository in order to develop their
specific features/changes there - not only but also because this would
simplify tracking of the various changes.
Point well taken !
Maxime
--
sent from my armchair
--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Peter Sadrozinski wrote: I think most of the work we are doing (alternate clipping library, 850 format) should be considered experimental, however. I'm pretty sure we want to keep the main branch concentrated on fixing problems with detailed landclass. Sure, I didn't propose to put the experimental stuff into the main _branch_ :-) I'm just a bit unhappy that I'd have to check various _repositories_ (_plus_ various branches therein) in order to get an update about what peple are actually doing about TerraGear. Experimental stuff is pretty much what Git branches (or branches in general) are for - not making use of this feature is probably the first step on the way to duplicate work which would be quite a stupid incident in a volunteer project where all participants are aiming at the same goal (at least that's what I assume). Thus, if you'd be willing to put your stuff into a branch at the main repo, please go ahead - call us if you don't have write access. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Maxime Guillaud wrote: The only problem that prevents me from calling it final and generating custom scenery for all of Europe can be seen in the last screenshot: http://www.mguillaud.net/fg/scenery-gallery/fgfs-screen-008.png I've seen a couple of these when preparing the terrain which is now on the Base Package. The Bay Area was built using additional custom land cover by Christian Schmitt and the cases I had to fix (manually) in order to prevent these grey patches usually looked like this: line Polygon A / / =/== / Polygon B / / / / [...] . with just a short segment of the line reaching into Polygon A, whereas Polygon A showed up as being grey in FlightGear. After trimming down these line segments in Polygon A back to the polygon boundary, all polys got textured properly. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
On 13 Nov 2011, at 10:04, Martin Spott wrote: I'm just a bit unhappy that I'd have to check various _repositories_ (_plus_ various branches therein) in order to get an update about what peple are actually doing about TerraGear. Experimental stuff is pretty much what Git branches (or branches in general) are for - not making use of this feature is probably the first step on the way to duplicate work which would be quite a stupid incident in a volunteer project where all participants are aiming at the same goal (at least that's what I assume). Thus, if you'd be willing to put your stuff into a branch at the main repo, please go ahead - call us if you don't have write access. Agreed 100% - Maxime, Peter, just ping me or Martin with your Gitorious key for access, and I'm happy to work you through the Git steps, on email or IRC, if you're worried about doing something bad. We have enough git experts here, we can fix most dumb things ... speaking as someone who has done *many* dumb things ;) James -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Curtis Olson wrote: So this all sounds good (I think) except we now have to compute a point inside each material region. Easy, right? Well, except that all the published algorithms can tell you if a random point is inside a polygon or not, but they don't tell you how to manufacture a point that is guaranteed to be inside a polygon. Well, there _is_ OpenSource software which is capable of reliably creating points in polygons (GRASS GIS, just to present a prominent example, they call them centroids) and terragear-cs did have such a thing in the past decade, but Ralf Gerlich finally decided to back it out again because later processing steps in TerraGear were designed to not having to deal with certain corner-cases for valid ! points in polygons. For more information, read thie fine comment in this diff: http://mapserver.flightgear.org/git/gitweb.pl?p=terragear-cs;a=commitdiff;h=8e0e2b6cef93ac689f611e93ca6250d33f243fbe;hp=e813c093fc444d9c36bb992768e37a1abb8c47d4 Therefore I'm still in favour of the long-term idea to drop the entire poly-preprocessing from TerraGear and, instead, add a frontend to read the entire land cover and roads from a GRASS database. GRASS is capable of doing all the clipping and sliver removal (v.clean tool=rmsa -c) in a topologically consistent and reliable way. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
James Turner wrote: Looks very good to me - especially when it's combined with some of the shader / materials work that is coming! Ah, well, I'm having mixed feelings about the shaders because they're imposing a huge performance hit onto our bigger multi-monitor setup Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Hi, Martin Spott wrote: Thus, if you'd be willing to put your stuff into a branch at the main repo, please go ahead - call us if you don't have write access. I don't feel too good about adding even more branches directly into the main TG repo. The reason is this: In the process of the 850 development we have changed and rebased the history multiple times against TG master (mainly because of the cmake addition that came along and that I added to our files as well). This as a consequence changes the git objects and needs a forced push. That's allright for testing. I'd rather do the dirty work seperately in an own repo than polluting the TG main repo with old and unused git objects (which git does not throw away by itself). And no, I'd not use git merge for this, as this as it complicates the history and leads to all kind of problems later on. To make life a bit easier, I have moved the TG 850 repo to be a clone of TG now, so that at least activity in our clone shows up on the main FG page. This should already improve the situation quite a bit. Chris -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
James Turner wrote: Okay, this all sounds like good news indeed. Martin, Chris, Peter - I think the steps need to be as follows - get a branch of terragear with the clipper changes, and probably the epsilon changes Maxime describes (because I've always worried, they were only needed due to GPC problems). I like the way of getting the changes into master in tiny chunks. And I already tested the epsilon changes of Maxime with the NL scenery I created for FSWeekend. So if you all agree, I can create a clipper/epsilon branch against master to get this tested first. Chris -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Christian Schmitt wrote: So if you all agree, I can create a clipper/epsilon branch against master to get this tested first. Yes, please, maybe even one for clipper and another with Maxime's epsilon changes applied. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
[Flightgear-devel] Fixing fgfs-construct crashes
James Turner zakal...@mac.com wrote: Onwards with fixing the fgfs-construct crashes! I have had pretty good luck building complex scenery with a modified version of fgfs-construct. Here is what I did: Following advice by Peter Sadrozinski here on the list, I started from the code in papillon's terragear repo where GPC is replaced by clipper. I then applied a few patches, reducing various epsilon constants in the code which I think were too loose and caused inconsistencies in the clipper. You can find my code here (in the branch custom_scenery_clean): https://gitorious.org/~maximeguillaud/papillon81/maxime-terragear-cs/commits/custom_scenery_clean More specifically, the patches relevant to fgfs-construct are in commits 81a78602e09b67edd58b2cedacf09079239b0177 and 8601f66cf7cf00ffda38f57cee77ae991c42dab8. The second one uses the constant SG_EPSILON defined in Simgear. I reduced the value of this constant in simgear/constants.h: #define SG_EPSILON (DBL_EPSILON*10) (Note that I am using a different installation of Simgear for terragear and for fgfs, so I do not know if this change would have consequences on fgfs). With the above settings, I was able to process large amounts of scenery without any crash. So far I have tried a band covering Europe between 12 and 14 degrees East, i.e. going through Sicily, Italy, the Austrian Alps, the Czech Republic and Germany, Sweden and Norway, up to 69 degrees North. My scenery includes detailed terrain (fitted with 15m accuracy), the CORINE land cover, and OSM rail and roads up to the secondary network. The latest update in the BTG format seem to have fixed the swirlies. You can see a few screenshots here: http://www.mguillaud.net/fg/scenery-gallery/ The only problem that prevents me from calling it final and generating custom scenery for all of Europe can be seen in the last screenshot: http://www.mguillaud.net/fg/scenery-gallery/fgfs-screen-008.png Sometimes, one polygon will have the correct shape but the material is wrong, and it appears all gray. This seems to occur only around roads. Any advice on this is appreciated. I had a chat with Martin Spott (hi Martin !) a few days ago and he mentioned that the bug is known, and proceeded to explain it to me, however I have not been able to relate his explanation to the code in fgfs-construct. I can provide a minimal set of files (3 roads) that trigger this problem, in case anyone can help with the debugging. Maxime -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Hi Maxime, Final material assignment is done when the data is triangulated. (All the points connected into triangles.) The algorithm needs a set of points, and it also needs a set of edges (boundaries) that show the division between material shapes. Finally for each material type, it needs a point inside each unique area. The algorithm starts at each given point and finds the triangle that encloses it assigns it the material type of that point. Then works it's way to all the neighbor triangles and assigns them the same type. It keeps going until it hits an edge (which we provide) or it runs out of triangles. Then it continues on to the next point and the next and the next. So this all sounds good (I think) except we now have to compute a point inside each material region. Easy, right? Well, except that all the published algorithms can tell you if a random point is inside a polygon or not, but they don't tell you how to manufacture a point that is guaranteed to be inside a polygon. With very complex convex shapes, possibility of holes, and all kinds of extremely tiny numbers (or points that are extremely close to each other) this becomes a bit more of a challenge. We have an approach with some heuristics to try to find a good point inside a polygon, but it sounds like this is failing. Also I haven't had my fingers in the code for quite some time and there have been some substantial changes. So what you want to do is identify the mistaken areas and determine the point we generate inside those areas. Then try to figure out what's wrong with the picture? Is the point in the area? Is it too close to an edge? Did we miss an edge such that the classification algorithm loses contain and spills out into a neighboring region. The answer here is to get down and dirty and look at the intermediate results ... it's painstaking, drudgery work, but someone needs to dig in to each specific case and figure out exactly why it's breaking, and then either fix the problem directly, or back up to an earlier stage in the pipeline and try to catch/avoid/fix the situation before it causes trouble. There are direct bugs, and then there are difficult situations with the data that arise. Best regards, Curt. On Sat, Nov 12, 2011 at 7:59 AM, Maxime Guillaud m...@mguillaud.net wrote: James Turner zakal...@mac.com wrote: Onwards with fixing the fgfs-construct crashes! I have had pretty good luck building complex scenery with a modified version of fgfs-construct. Here is what I did: Following advice by Peter Sadrozinski here on the list, I started from the code in papillon's terragear repo where GPC is replaced by clipper. I then applied a few patches, reducing various epsilon constants in the code which I think were too loose and caused inconsistencies in the clipper. You can find my code here (in the branch custom_scenery_clean): https://gitorious.org/~maximeguillaud/papillon81/maxime-terragear-cs/commits/custom_scenery_clean More specifically, the patches relevant to fgfs-construct are in commits 81a78602e09b67edd58b2cedacf09079239b0177 and 8601f66cf7cf00ffda38f57cee77ae991c42dab8. The second one uses the constant SG_EPSILON defined in Simgear. I reduced the value of this constant in simgear/constants.h: #define SG_EPSILON (DBL_EPSILON*10) (Note that I am using a different installation of Simgear for terragear and for fgfs, so I do not know if this change would have consequences on fgfs). With the above settings, I was able to process large amounts of scenery without any crash. So far I have tried a band covering Europe between 12 and 14 degrees East, i.e. going through Sicily, Italy, the Austrian Alps, the Czech Republic and Germany, Sweden and Norway, up to 69 degrees North. My scenery includes detailed terrain (fitted with 15m accuracy), the CORINE land cover, and OSM rail and roads up to the secondary network. The latest update in the BTG format seem to have fixed the swirlies. You can see a few screenshots here: http://www.mguillaud.net/fg/scenery-gallery/ The only problem that prevents me from calling it final and generating custom scenery for all of Europe can be seen in the last screenshot: http://www.mguillaud.net/fg/scenery-gallery/fgfs-screen-008.png Sometimes, one polygon will have the correct shape but the material is wrong, and it appears all gray. This seems to occur only around roads. Any advice on this is appreciated. I had a chat with Martin Spott (hi Martin !) a few days ago and he mentioned that the bug is known, and proceeded to explain it to me, however I have not been able to relate his explanation to the code in fgfs-construct. I can provide a minimal set of files (3 roads) that trigger this problem, in case anyone can help with the debugging. Maxime -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now
Re: [Flightgear-devel] Fixing fgfs-construct crashes
On 12 Nov 2011, at 13:59, Maxime Guillaud wrote: I have had pretty good luck building complex scenery with a modified version of fgfs-construct. Here is what I did: snip Okay, this all sounds like good news indeed. Martin, Chris, Peter - I think the steps need to be as follows - get a branch of terragear with the clipper changes, and probably the epsilon changes Maxime describes (because I've always worried, they were only needed due to GPC problems). Then we can all test on our favourite datasets, and see if there's any regressions, and hopefully a big improvement in not crashing! (I'm also glad to hear the BTG changes have really fixed the swirlies) You can see a few screenshots here: http://www.mguillaud.net/fg/scenery-gallery/ Looks very good to me - especially when it's combined with some of the shader / materials work that is coming! The only problem that prevents me from calling it final and generating custom scenery for all of Europe can be seen in the last screenshot: http://www.mguillaud.net/fg/scenery-gallery/fgfs-screen-008.png Personally, I'd could live with the issue to get the world scenery rebuilt, but certainly to have the code on an official branch for testing, and investigation of this bug makes sense, I think. James -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
With this minimal data set, do you see bad node print messages during construct? found a bad node = xxx When a node touches a line segment on the same contour, the bad node is removed, as odd/even clipper algorithms blow up when this occurs. I would experiment with removing the code that removes the node. You'll be trading one error for another, but it may help track this down. Take a look at reduce_contour_degeneracy() in poly_support.cxx If this is the cause, then we'll have to figure out a different way to handle touching segments. As Curt said, pushing nodes around may not be any better than removing the offending node all together. Pete On Sat, Nov 12, 2011 at 8:59 AM, Maxime Guillaud m...@mguillaud.net wrote: James Turner zakal...@mac.com wrote: Onwards with fixing the fgfs-construct crashes! I have had pretty good luck building complex scenery with a modified version of fgfs-construct. Here is what I did: Following advice by Peter Sadrozinski here on the list, I started from the code in papillon's terragear repo where GPC is replaced by clipper. I then applied a few patches, reducing various epsilon constants in the code which I think were too loose and caused inconsistencies in the clipper. You can find my code here (in the branch custom_scenery_clean): https://gitorious.org/~maximeguillaud/papillon81/maxime-terragear-cs/commits/custom_scenery_clean More specifically, the patches relevant to fgfs-construct are in commits 81a78602e09b67edd58b2cedacf09079239b0177 and 8601f66cf7cf00ffda38f57cee77ae991c42dab8. The second one uses the constant SG_EPSILON defined in Simgear. I reduced the value of this constant in simgear/constants.h: #define SG_EPSILON (DBL_EPSILON*10) (Note that I am using a different installation of Simgear for terragear and for fgfs, so I do not know if this change would have consequences on fgfs). With the above settings, I was able to process large amounts of scenery without any crash. So far I have tried a band covering Europe between 12 and 14 degrees East, i.e. going through Sicily, Italy, the Austrian Alps, the Czech Republic and Germany, Sweden and Norway, up to 69 degrees North. My scenery includes detailed terrain (fitted with 15m accuracy), the CORINE land cover, and OSM rail and roads up to the secondary network. The latest update in the BTG format seem to have fixed the swirlies. You can see a few screenshots here: http://www.mguillaud.net/fg/scenery-gallery/ The only problem that prevents me from calling it final and generating custom scenery for all of Europe can be seen in the last screenshot: http://www.mguillaud.net/fg/scenery-gallery/fgfs-screen-008.png Sometimes, one polygon will have the correct shape but the material is wrong, and it appears all gray. This seems to occur only around roads. Any advice on this is appreciated. I had a chat with Martin Spott (hi Martin !) a few days ago and he mentioned that the bug is known, and proceeded to explain it to me, however I have not been able to relate his explanation to the code in fgfs-construct. I can provide a minimal set of files (3 roads) that trigger this problem, in case anyone can help with the debugging. Maxime -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Hi Maxime and others, Maxime Guillaud wrote: You can find my code here [...] I'm just starting to recover from a couple of _really_ tight days (including a nice PostgreSQL conference PGConf.DE where I gave a talk about our geodata collection and in-database processing), therefore I'm not yet ready to provide comprehensive responses. Anyhow there's one point I'd like to emphasize: I know it's really cool to maintain private source repositories ;-) but for increasing the overall success of building FlightGear scenery I'd think it would be really beneficial to keep the various development efforts in close relation and sync. Thus I'd invite everyone who's serious about improving the TerraGear toolchain to create a branch in the main repository in order to develop their specific features/changes there - not only but also because this would simplify tracking of the various changes. The former terragear-cs main repository at MapServer was already open to (almost) everybody who asked and now that it's moved over to Gitorious there's really no more excuse not to create branches inside the main repo :-) Best regards, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
Hi Martin,
I've been reluctant to move to the official repository mostly because of
very little understanding of GIT.
I'm a bit more confident, now, so I don't see it as a big problem.
I think most of the work we are doing (alternate clipping library, 850
format) should be considered experimental, however. I'm pretty sure we
want to keep the main branch
concentrated on fixing problems with detailed landclass.
We seem to be breaking terragear pretty good as of late :)
Maxime: I have an experiment for you to try - could you take the UFO under
those grey sections of scenery, and look up at them? I think the normals
have swapped.
Chris mentioned this happening with the skirt around the airport, and I
hadn't seen it until a recent update. Looks like something broke recently.
Pete
On Sat, Nov 12, 2011 at 2:38 PM, Martin Spott martin.sp...@mgras.netwrote:
Hi Maxime and others,
Maxime Guillaud wrote:
You can find my code here [...]
I'm just starting to recover from a couple of _really_ tight days
(including a nice PostgreSQL conference PGConf.DE where I gave a talk
about our geodata collection and in-database processing), therefore I'm
not yet ready to provide comprehensive responses. Anyhow there's one
point I'd like to emphasize:
I know it's really cool to maintain private source repositories ;-)
but for increasing the overall success of building FlightGear
scenery I'd think it would be really beneficial to keep the various
development efforts in close relation and sync. Thus I'd invite
everyone who's serious about improving the TerraGear toolchain to
create a branch in the main repository in order to develop their
specific features/changes there - not only but also because this would
simplify tracking of the various changes.
The former terragear-cs main repository at MapServer was already open
to (almost) everybody who asked and now that it's moved over to
Gitorious there's really no more excuse not to create branches inside
the main repo :-)
Best regards,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--
--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Fixing fgfs-construct crashes
I have verified that the problem lies in using clipper.
Using the same simple data with GPC works. It appears clipper can
sometimes produce clockwise winded polys.
I'm working on a fix now.
Pete
On Sat, Nov 12, 2011 at 7:59 PM, Peter Sadrozinski
psadrozin...@gmail.comwrote:
Hi Martin,
I've been reluctant to move to the official repository mostly because of
very little understanding of GIT.
I'm a bit more confident, now, so I don't see it as a big problem.
I think most of the work we are doing (alternate clipping library, 850
format) should be considered experimental, however. I'm pretty sure we
want to keep the main branch
concentrated on fixing problems with detailed landclass.
We seem to be breaking terragear pretty good as of late :)
Maxime: I have an experiment for you to try - could you take the UFO under
those grey sections of scenery, and look up at them? I think the normals
have swapped.
Chris mentioned this happening with the skirt around the airport, and I
hadn't seen it until a recent update. Looks like something broke recently.
Pete
On Sat, Nov 12, 2011 at 2:38 PM, Martin Spott martin.sp...@mgras.netwrote:
Hi Maxime and others,
Maxime Guillaud wrote:
You can find my code here [...]
I'm just starting to recover from a couple of _really_ tight days
(including a nice PostgreSQL conference PGConf.DE where I gave a talk
about our geodata collection and in-database processing), therefore I'm
not yet ready to provide comprehensive responses. Anyhow there's one
point I'd like to emphasize:
I know it's really cool to maintain private source repositories ;-)
but for increasing the overall success of building FlightGear
scenery I'd think it would be really beneficial to keep the various
development efforts in close relation and sync. Thus I'd invite
everyone who's serious about improving the TerraGear toolchain to
create a branch in the main repository in order to develop their
specific features/changes there - not only but also because this would
simplify tracking of the various changes.
The former terragear-cs main repository at MapServer was already open
to (almost) everybody who asked and now that it's moved over to
Gitorious there's really no more excuse not to create branches inside
the main repo :-)
Best regards,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--
--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
