Re: [Flightgear-devel] Upcoming Random Buildings changes
On Wed, Sep 18, 2013 at 4:16 PM, Stuart Buchanan wrote: This is to give a heads up on some changes that I'm planning for Random Buildings for the V3.0 release, and to allow for comments/suggestions/ideas. I've now got the new system broadly working on a private build: http://www.nanjika.co.uk/flightgear/buildings.jpg I've managed to minimize apparent tiling by randomly choosing between building sizes and random objects at each point (subject to dimension restrictions). I need to sort out the random objects which need to be rotated by 90 degrees (see the bottom left where an orange school building goes across the road). As expected, the downside is that creating the object masks takes quite a bit longer. It's taken me about 4 hours to generate the mask used here for the city1.png texture. Generating the buildings at runtime is also a bit slower so I expect to implement the PagedLoD next. -Stuart In 2.12.0 the algorithm for the placement of random buildings is a rather scattergun approach. materials.xml sets a building density (amongst other building attributes) which is used to generate a number of random points on each triangle that makes up the scenery. These random points are checked against a mask to ensure they aren't being placed in the middle of roads, and then rotated to the correct orientation to match the texture (using the red channel of the mask). This broadly works, but the placement is fairly unrealistic: - The mask is quite broad, and some buildings quite large, so they can still stick out into the roads, and generally don't line right up against the street. - The spacing of buildings has to be quite conservative. So, in V3.0 I plan to change this. Instead of using a scatter-gun approach to placement and a mask, random building location will be read directly from the mask, defined by a single pixel. The color (actually blue value from 0-255) will define the size of building (small medium, large), and the red channel the rotation. So instead of a material designer blocking out a large area for random buildings to be placed within, they will define the specific location for each random building. Creating masks is going to require quite a bit more work in the new world, but the end result should be better. There is an open question of how to handle random object placement, which also use the blue channel of the object mask: - For areas that don't use random buildings, I'm tempted just to leave it as-is for the moment, as it reduces the number of object masks that need to be re-created. - For areas with both random buildings and random objects I'm planning to make groups of objects to specific blue values. So, a blue value of 234 might equate to placing a silo or a farmhouse, while a blue value of 233 might place a church, or tower block. The mapping will be defined in the materials.xml file. Command, questions and offers of help to update some object masks are all welcome :) -Stuart -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134071iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
On Fri, 11 Oct 2013, Stuart Buchanan wrote: On Wed, Sep 18, 2013 at 4:16 PM, Stuart Buchanan wrote: This is to give a heads up on some changes that I'm planning for Random Buildings for the V3.0 release, and to allow for comments/suggestions/ideas. I've now got the new system broadly working on a private build: http://www.nanjika.co.uk/flightgear/buildings.jpg That looks realy nice Stuart! Good job! Is it possble to place buildings manually and have the random buildings fill around them without interference? g. -- Proud owner of F-15C 80-0007 http://www.f15sim.com - The only one of its kind. http://www.diy-cockpits.org/coll - Go Collimated or Go Home. Some people collect things for a hobby. Geeks collect hobbies. ScarletDME - The red hot Data Management Environment A Multi-Value database for the masses, not the classes. http://scarlet.deltasoft.com - Get it _today_! -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134071iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
Hi Tim, On Fri, Oct 11, 2013 at 12:55 PM, Tim Moore wrote: Cool stuff. Are those all individual models, or instanced geometry, or what? :) The vast majority are instanced geometries, though there are some random models interspersed. As mentioned previously, I'd like to move back to the previous implementation from (IIRC 2.8.0) where the buildings were collected into massive objects as it provides collision detection and more variety. On Fri, Oct 11, 2013 at 1:54 PM, geneb wrote: Is it possble to place buildings manually and have the random buildings fill around them without interference? Not yet, but that's a known bug that I would dearly like to fix. I'm hoping that the refactoring required to implement the PagedLOD will provide access to the data required. At present the static objects (e.g. manually placed buildings) are only ingested after the geometry and random objects have already been generated. With the PagedLOD approach the random objects will be generated only when the aircraft gets into range, so I'm hoping I can simply check against the bounding box of each static object. -Stuart -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134071iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
On Thu, Sep 19, 2013 at 3:07 PM, Renk Thorsten wrote: Which would in fact not make them random buildings at all :-) It'll make them less random, certainly :) However, while the placement and size class (small, medium, large) will now be determined by the texture, there will be still be randomness for each building in terms of width, depth, number of floors and texture. I need to prototype this to understand whether that's sufficient to fool the eye or not. The other avenue I'm looking at is whether the texture scale is correct. IIRC the city1 texture is 1024x1024 and covers a 1kmx1km area. I need to measure some street widths, as I suspect that might be a bit small. It may be that we can increase the texture coverage without increasing the texture size, which would reduce the repetition. Or just decide that cities are important enough to be worth a larger texture. May I propose to think about something more ambitious? One thing which bugs me is that we have now well-working de-tiling strategies for terrain, but not really for city textures and agriculture. Forest is effectively de-tiled by placing sufficiently many random trees on top - but we still get huge tiling artefacts for large city areas or fields. Deterministic 'random' building on top of that will make the tiling problem worse, because you get repeating patterns now in both texture and objects on top, which wasn't the case with the scattergun. I think one could at least partially solve this by subdividing the terrain into individual coordinate cells and give each city texture inside the cell a random orientation of the texture. The template would be the sparse dot noise function introduced in terrain-haze-ultra.frag , but instead of creating the dot position and radius randomly, the routine would just create a rotation angle randomly. I believe this would improve city and agriculture, but for this to really work, the 'random' buildings and trees would have to be shifted by the same function - probaby at placement time. Would it be technically hard to design the system in such a way that this could be introduced? I don't have a function ready yet, but it's something I wanted to give a try at some point with running a distinct shader for 'random' and 'civilized' terrain which needs different types of overlay. At the moment, we use a shader instantiate the buildings and forests to the correct locations. That transformation could be made more complicated to allow for texture rotation. Where it would have more difficulty is in handling the random objects, which are masked as well and would need to be moved. It would also be a bit tricky to make the edges of each coordinate cell look. At the moment, the textures edges tile nicely. I guess we could mirror the translation in the C-code, but it feels a bit wrong. On Thu, Sep 19, 2013 at 5:05 PM, Adrian Musceac wrote: Stuart, this is totally unrelated, but what would be the price to pay to have collision with generated buildings, and Rembrandt shadows? Actually, this isn't completely unrelated... The original random buildings implementation (2.8.0) use basic OSG primitives, and so had collision detection and Rembrandt shadows for free. In 2.10.0 this was changed to a shader-based instance approach based on the tree scheme to reduce the memory footprint which was ridiculously high in places like LA or Tehran. With the shader-based approach, collision detection isn't possible as the building doesn't really exist in the scenegraph. Rembrandt should be possible at the cost of running another shader IIRC. I think I had Rembrandt working for trees, but the cost was absolutely huge. Now, the basic OSG primitive approach has some advantages: - doesn't rely on shader support - allows for more variety in the buildings as one isn't instantiating a small set of buildings multiple times. - the code is simpler - more flexibility in adding complicated buildings such as signage, extensions. If I implement a PagedLOD approach, it might reduce the memory footprint sufficiently that we could switch back to the OSG primitives. Now, all I need to do is find the time to code all of this up!. -Stuart -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
On Friday, September 20, 2013 13:04:32 Stuart Buchanan wrote: If I implement a PagedLOD approach, it might reduce the memory footprint sufficiently that we could switch back to the OSG primitives. I think one of the problems is that most of our objects have no concept of different LOD's. If we had objects with say 3 LOD, it would ease memory and graphics load. It should be possible to take an object and generate multiple LOD's for it automatically, but I have almost no knowledge about 3D objects. Might be worth checking it out. Cheers, Adrian -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
On Fri, Sep 20, 2013 at 11:15 AM, Adrian Musceac wrote: I think one of the problems is that most of our objects have no concept of different LOD's. If we had objects with say 3 LOD, it would ease memory and graphics load. It should be possible to take an object and generate multiple LOD's for it automatically, but I have almost no knowledge about 3D objects. Might be worth checking it out. While a good LoD system helps with graphics load, it doesn't help with overall memory load (unless it's a PagedLOD), which was the problem with the buildings. We have fairly good LoD solutions for both trees and buildings to help with the graphics loads. For example we display the large buildings are further range than the medium or small. Hopefully people never notice it because it's quite subtle. -Stuart -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
On 20 Sep 2013, at 11:04, Stuart Buchanan stuar...@gmail.com wrote: The original random buildings implementation (2.8.0) use basic OSG primitives, and so had collision detection and Rembrandt shadows for free. In 2.10.0 this was changed to a shader-based instance approach based on the tree scheme to reduce the memory footprint which was ridiculously high in places like LA or Tehran. With the shader-based approach, collision detection isn't possible as the building doesn't really exist in the scenegraph. Rembrandt should be possible at the cost of running another shader IIRC. I think I had Rembrandt working for trees, but the cost was absolutely huge. Now, the basic OSG primitive approach has some advantages: - doesn't rely on shader support - allows for more variety in the buildings as one isn't instantiating a small set of buildings multiple times. - the code is simpler - more flexibility in adding complicated buildings such as signage, extensions. If I implement a PagedLOD approach, it might reduce the memory footprint sufficiently that we could switch back to the OSG primitives. That would be my hope too, BTW. Ideally we'd set a total memory use (or vertex count, which is a proxy metric for the same) for trees+buildings and setup the PagedLOD to keep things loading (really, generating) and unloading based on that target. Then it becomes a fairly clear memory-burn vs popping tradeoff which I think most people would accept as reasonable. You don't like the pops, you buy more [V]RAM :) Instancing would help the memory burn, but OSG makes it very complex, we need to bypass large chunks of the PrimitiveSet and Arrays APIs until they get a real API internally, possibly not until OSG 3.4 (This is on the assumption that even though the buildings are random, across a tile you could have quite a few which differ only in transform and some other shader uniforms, but have the same width/breadth/height/roof-pitch/number-of-floors, and hence could be drawn using the same instance data) Regards, James -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
So, in V3.0 I plan to change this. Instead of using a scatter-gun approach to placement and a mask, random building location will be read directly from the mask, defined by a single pixel. The color (actually blue value from 0-255) will define the size of building (small medium, large), and the red channel the rotation. So instead of a material designer blocking out a large area for random buildings to be placed within, they will define the specific location for each random building. Which would in fact not make them random buildings at all :-) May I propose to think about something more ambitious? One thing which bugs me is that we have now well-working de-tiling strategies for terrain, but not really for city textures and agriculture. Forest is effectively de-tiled by placing sufficiently many random trees on top - but we still get huge tiling artefacts for large city areas or fields. Deterministic 'random' building on top of that will make the tiling problem worse, because you get repeating patterns now in both texture and objects on top, which wasn't the case with the scattergun. I think one could at least partially solve this by subdividing the terrain into individual coordinate cells and give each city texture inside the cell a random orientation of the texture. The template would be the sparse dot noise function introduced in terrain-haze-ultra.frag , but instead of creating the dot position and radius randomly, the routine would just create a rotation angle randomly. I believe this would improve city and agriculture, but for this to really work, the 'random' buildings and trees would have to be shifted by the same function - probaby at placement time. Would it be technically hard to design the system in such a way that this could be introduced? I don't have a function ready yet, but it's something I wanted to give a try at some point with running a distinct shader for 'random' and 'civilized' terrain which needs different types of overlay. * Thorsten -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
On Wednesday, September 18, 2013 18:45:09 James Turner wrote: (BTW Adrian proposed something which achieves a similar end result back in February, the problem with his approach is it doesn't work via the OSG paging mechanism, and hence I don't think we should pursue it - ultimately we need to be letting the OSG Pager handle all our loading and unloading, instead of trying to fit our own LOD systems above / below it) Regards, James I had a private exchange about that in December with Mathias, his method is way better, hence I didn't pursue this any further. I'm still using my code here for the memory benefits but that's related to my radio code. PagedLOD is the way to go, case closed :) Cheers, Adrian -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
On Wednesday, September 18, 2013 19:05:22 Stuart Buchanan wrote: I did take a look at the PagedLOD approach - Matthias' code made a great template to work from. -Stuart Stuart, this is totally unrelated, but what would be the price to pay to have collision with generated buildings, and Rembrandt shadows? Adrian -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
On Wed, Sep 18, 2013 at 4:45 PM, James Turner wrote: It would really help memory use if we could defer tree/object/building generation until the tile is close to the viewer. Right now you generate lots of LOD nodes so we don't pay the rendering cost for distant tiles, but we're still paying a memory cost and loading time hit. I did take a look at the PagedLOD approach - Matthias' code made a great template to work from. One thing I wasn't clear on is whether this would move generation of the trees/objects/buildings from a scenery loading thread into some other thread. I'm not sufficiently familiar with our threading to know if there would be any impact. -Stuart -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Upcoming Random Buildings changes
On 18 Sep 2013, at 17:05, Stuart Buchanan stuar...@gmail.com wrote: I did take a look at the PagedLOD approach - Matthias' code made a great template to work from. Yes, that is exactly the model I would follow, and I am sure Mathias can be asked for additional hints on the best way to integrate it. His SPT system replaces the tile-scheduling logic with the osgDB Pager, which improve various things, since osgDB would be fully in control of deciding what is loaded, and unloaded when, based on the camera(s) - no more arbitrary dealing with rings of tiles based on visibility parameters. One thing I wasn't clear on is whether this would move generation of the trees/objects/buildings from a scenery loading thread into some other thread. I'm not sufficiently familiar with our threading to know if there would be any impact. The ReaderWriters run in the osgDB pager thread, which is exactly where the current ReaderWriterSTG runs (which ultimately does the current tree/object/building placement, I think) - so the threading should not change at all. Indeed, the more I think on it, the more it feels like this should be a very small restructuring of the code, just requiring some slightly delicate PagedLOD plumbing to make it work. We're already doing the right work (building an osg::Node) and doing it in the right thread, we just need to change *when* we do it. BTW, I would also hope this means the fixed osg::LOD elements in the quadtrees could go away, or at least the layering reduced (to one layer of LODs instead of two) My gut feeling is the LOD-tree is preventing sufficiently large batching for both trees and random buildings. But of course the optimal batch size is very hardware dependant. Regards, James -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
