[osg-users] Distributed rendering (was: RE: [3rdparty] Cloud node kit)
OSG doesn't prevent you from doing distributed rendering. If you have a copy of your scene graph on multiple machines and your appuses a socket or othermechanism to communicate, you have distributed rendering. -Paul I m in need of distributed rendering and i thought OSG supported this natively. 2-If OSG is sufficent for distributed rendering, is there an example how to achieve it? ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
[osg-users] Distributed Rendering
Dear all, I'm thinking in using OSG (via Delta3D) for a flight simulator but I have one little problem. How to set up OSG to perform Distributed Rendering? Usually we use a commercial tool that provides Distributed Rendering functionality, but we would like to move to OSG (because it's better) but I'm afraid we won't be able to have multiple channels using several pc. An example would be 3 Image Generator to render 180º horizontal FOV, 60º each, synchronized. Could I just have 3 OSG applications in 3 different machines having a camera attitude offset? Can someone provide me some insight on this issue, and possible solutions? I would really love to use OSG in this project. Thank you for your kind attention. Regards, Raul -- Read this topic online here: http://forum.openscenegraph.org/viewtopic.php?p=18473#18473 ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed Rendering
Hi Raul, have you tried osgcluster's example,for example: for the master: osgcluster.exe -m cow.osg for the slaves: osgclusterd.exe -s -o 15 cow.osg osgclusterd.exe -s -o 0 cow.osg osgclusterd.exe -s -o -15 cow.osg Best regards, Jagovic Dear all, I'm thinking in using OSG (via Delta3D) for a flight simulator but I have one little problem. How to set up OSG to perform Distributed Rendering? Usually we use a commercial tool that provides Distributed Rendering functionality, but we would like to move to OSG (because it's better) but I'm afraid we won't be able to have multiple channels using several pc. An example would be 3 Image Generator to render 180º horizontal FOV, 60º each, synchronized. Could I just have 3 OSG applications in 3 different machines having a camera attitude offset? Can someone provide me some insight on this issue, and possible solutions? I would really love to use OSG in this project. Thank you for your kind attention. Regards, Raul ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed Rendering
Hi Jagoviv, Many thanks for your reply. Indeed osgCluster is a good starting point to implement the visual system. It also provides synchronization mechanism through software. Next steep is to load a large database (aprox. 50 Gb). I will look through the examples and forum to find out how to do it. Once again thank you for your suggestion. Regards, Raul -- Read this topic online here: http://forum.openscenegraph.org/viewtopic.php?p=18492#18492 ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
Benjamin, Jan, Let me jump in with some definite answers on Equalizer. On 1. Mar 2008, at 20:54, Benjamin Eikel wrote: On Saturday 01 March 2008 at 20:18:19, Jan Ciger wrote: Hello Benjamin, Benjamin Eikel wrote: Hello Jan, thank you for your suggestion. Equalizer looks like a prebuilt framework for simple, high-performance parallel rendering. This is not what we are searching for, because we want to/have to implement our own specialized rendering system. As far as I remember, you can plug in your own rendering and compositing stages into Equalizer. Hello Jan, okay, this might be something that we could use. I have only skimmed through the web page and the programming guide so maybe I missed something. The problem that I see is that maybe other functions which are provided by OpenSceneGraph are not supported by Equalizer (loading from different file formats, picking/dragging, animation pathes). Equalizer is a parallel rendering framework, in the same sense as MPI is a parallel programming framework. File loading, picking, etc. are application-specific problema, and outside of the scope of Equalizer. Equalizer does not prevent implementing these, and where it makes sense it will aid these functions. File loading for example would be done, as I understand, on your application node. The resulting scenegraph will then be distributed to your render clients. This is something we have already done with another scene graph. Picking and event handling in general are areas where Equalizer helps the application. For example, events are sent to the application node by default. Data distribution for OSG is something which has been requested a number of times, just nobody stepped forward and did it so far (or paid for it). I am sure this work would give you a good reward, and I can help with the details. Equalizer gives you a framework into which you plug your applications' (rendering) code. If you have something which doesn't fit, let us know and we'll find a solution. Furthermore if we want to write our own rendering and composition algorithm what is left for Equalizer to do for us? What exactly do you have in mind? It seems that if you want to implement your own parallel rendering algorithms, the right way would be to modify Equalizer. You are of course welcome to contribute. I think the effort getting used to it and integrating it in our system would be too high in comparison to the earnings. The effort to relearn the lessons we learned by doing two parallel rendering frameworks is high as well. Sometimes it is better to start from something working than from scratch, even though it is tempting to reinvent the wheel (I am/was in academia too). Yes that is right. That is why we want to use OpenSceneGraph. An integration of OSG with Equalizer is a new thing. Furthermore it builds new value (rather than redoing existing things), and would add value to both projects. Cross-posting to eq-dev. Feel free to direct Equalizer-specific questions to this list. Cheers, Stefan. -- http://www.eyescale.ch http://www.equalizergraphics.com http://www.linkedin.com/in/eilemann ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
HI Benjamin, Thanks for the more structured explanation. I'm not sure that the overall approach will be the most efficient way to tackle the task for a wide range of usages, so I'd recommend exploring the possibilities of different ways to implement that various usages rather than just sticking to one approach - different approaches are very likely to be suitable for different usages. o What is my question at all? I am searching for a interface where I can grab these visible nodes and put them into a list. I need them so I can analyse them to run the balancing algorithm and send the data to the rendering nodes. I think that it might be possible to use the DrawImplementation of the Drawable nodes. What it sounds like you are suggesting/looking for is a distributed render graph. The OSG's render graph is the rendering back end that the OSG produces from its cull traversal. The render graph containts osgUtil::RenderStage/Bin, a osgUtil::StateGraph and osgUtil::RenderLeaf. The RenderLeaf contain pointers to the projection and modelview matrices, the drawable and the point on the StateGraph that specifies the accumulated state that must applied for that drawable. Once the the OSG's cull traversal creates the render graph, the draw traversal then traverses this render graph in a controlled order that ensures that the scene builds up in the correct way - for instances opaque objects first, then depth sorted transparent objects. Now you could implement a system where rather than the local machine traversing the render graph to do the rendering, you could broadcast this rendering graph to the cluster and get them all render a portion of it, then to collate the result back of the frame buffer chunks. I don't think you should be doing things at as low level as trying to overriding the OpenGL functions that the OSG uses in its Drawables and StateAttributes. To go this low level you might as well use a distributed GL implementation and save yourself a lot of time. You mention transforms and drawables in your text, whereas you actually need all OpenGL state that the OSG manages as well, you really do need the whole render graph, yes it is more complex but its what is actually required to get things to render properly. There are also issues of performance - you want to be distributing as little data as possible, and doing a small a compute on the master as possible, this means differing the most work you can to the slaves, and minimizing the amount of the data that you have to distribute to the slaves and get back. Low level distributed GL loads the network and the master CPU massively, high level distributed application/IG has the lowest network and master CPU load, and since you system is only as fast as the slowest bottleneck then the high level system will always provide the greatest peformance. Your distributed render graph system is somewhere in the middle. Robert. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
Hello Robert, On Sunday 02 March 2008 at 11:42:24, Robert Osfield wrote: HI Benjamin, Thanks for the more structured explanation. I'm not sure that the overall approach will be the most efficient way to tackle the task for a wide range of usages, so I'd recommend exploring the possibilities of different ways to implement that various usages rather than just sticking to one approach - different approaches are very likely to be suitable for different usages. You have a point there. But we want to implement a very specialized system for only our kind of factory scenes. Of course our system will perform very bad with other kind of scenes (nature for example). o What is my question at all? I am searching for a interface where I can grab these visible nodes and put them into a list. I need them so I can analyse them to run the balancing algorithm and send the data to the rendering nodes. I think that it might be possible to use the DrawImplementation of the Drawable nodes. What it sounds like you are suggesting/looking for is a distributed render graph. The OSG's render graph is the rendering back end that the OSG produces from its cull traversal. The render graph containts osgUtil::RenderStage/Bin, a osgUtil::StateGraph and osgUtil::RenderLeaf. The RenderLeaf contain pointers to the projection and modelview matrices, the drawable and the point on the StateGraph that specifies the accumulated state that must applied for that drawable. Thank you for that insight. That was something I did not know and that was the part that was missing in my puzzle. This sound exactly like the thing I am searching. So if I want to get this RenderStage object I could get it from the SceneView after doing the update and cull traversal? Once the the OSG's cull traversal creates the render graph, the draw traversal then traverses this render graph in a controlled order that ensures that the scene builds up in the correct way - for instances opaque objects first, then depth sorted transparent objects. Now you could implement a system where rather than the local machine traversing the render graph to do the rendering, you could broadcast this rendering graph to the cluster and get them all render a portion of it, then to collate the result back of the frame buffer chunks. Okay right. We will only have one render bin which eases the thing a little bit because we will not support transparent objects. I don't think you should be doing things at as low level as trying to overriding the OpenGL functions that the OSG uses in its Drawables and StateAttributes. To go this low level you might as well use a distributed GL implementation and save yourself a lot of time. You mention transforms and drawables in your text, whereas you actually need all OpenGL state that the OSG manages as well, you really do need the whole render graph, yes it is more complex but its what is actually required to get things to render properly. Yes you are right. I did not know that this render graph does exist and only looked at the documentation and the code. But I did not get this insight that you gave me by your explanations above. So I assumend that overwriting the DrawImplementation for getting the information about the visible nodes would be a possible way. It might be possible, but as you said, not a good approach. Yes you are right, in general we need the whole state, but as we do not want to implement texturing I hope the state does not change that often and it might be possible to use the same state for the whole scene (except for the model-view matrix and vertex data of course). There are also issues of performance - you want to be distributing as little data as possible, and doing a small a compute on the master as possible, this means differing the most work you can to the slaves, and minimizing the amount of the data that you have to distribute to the slaves and get back. Low level distributed GL loads the network and the master CPU massively, high level distributed application/IG has the lowest network and master CPU load, and since you system is only as fast as the slowest bottleneck then the high level system will always provide the greatest peformance. Your distributed render graph system is somewhere in the middle. Yes that is clear. We have to see how the system performs with that single central node which has to do the culling etc. Thank you very much for your time and help. Regards, Benjamin Robert. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
On Sun, Mar 2, 2008 at 11:41 AM, Benjamin Eikel [EMAIL PROTECTED] wrote: Thanks for the more structured explanation. I'm not sure that the overall approach will be the most efficient way to tackle the task for a wide range of usages, so I'd recommend exploring the possibilities of different ways to implement that various usages rather than just sticking to one approach - different approaches are very likely to be suitable for different usages. You have a point there. But we want to implement a very specialized system for only our kind of factory scenes. Of course our system will perform very bad with other kind of scenes (nature for example). Even for you factory scene what approach to use will depend upon what you are viewing - its relatively small section then a distributed approach will be slower than a local rendering. You have to bare in mind the overheads and bottlenecks inherit with a distributed system, throwing lots of hardware at the problem won't help if you are introducing greater bottlenecks in the process of using the cluster. What it sounds like you are suggesting/looking for is a distributed render graph. The OSG's render graph is the rendering back end that the OSG produces from its cull traversal. The render graph containts osgUtil::RenderStage/Bin, a osgUtil::StateGraph and osgUtil::RenderLeaf. The RenderLeaf contain pointers to the projection and modelview matrices, the drawable and the point on the StateGraph that specifies the accumulated state that must applied for that drawable. Thank you for that insight. That was something I did not know and that was the part that was missing in my puzzle. This sound exactly like the thing I am searching. So if I want to get this RenderStage object I could get it from the SceneView after doing the update and cull traversal? You get the root RenderStage after cull traversal. osgViewer::Renderer has the SceneView that you can get the RenderStage from. The root RenderStage has everything you need. Might I recommend using osgViewer as your base and then just having a custom renderingTraversals method that you can override the cull and draw traversals if need be. If need be is the key part as you may well find using standard local rendering will often be better than a distributed implementation. Okay right. We will only have one render bin which eases the thing a little bit because we will not support transparent objects. Much of modern rendering is done with multiple pass techniques that go well beyond just the need for transparent bins. Don't forget there techniques like using of the occlussion query extension requires multipass, and its one technique that is well suited for complex CAD models. Paul Martz was even commission recently to add support for this in the OSG for the purpose of better handling CAD models. So, while limiting what types of rendering you can use to just standard opaque models might make your tasks easier, there is a real danger of excluding modern rendering techniques that help solve the performance problems in a better way. I don't think you should be doing things at as low level as trying to overriding the OpenGL functions that the OSG uses in its Drawables and StateAttributes. To go this low level you might as well use a distributed GL implementation and save yourself a lot of time. You mention transforms and drawables in your text, whereas you actually need all OpenGL state that the OSG manages as well, you really do need the whole render graph, yes it is more complex but its what is actually required to get things to render properly. Yes you are right. I did not know that this render graph does exist and only looked at the documentation and the code. But I did not get this insight that you gave me by your explanations above. So I assumend that overwriting the DrawImplementation for getting the information about the visible nodes would be a possible way. It might be possible, but as you said, not a good approach. Yes you are right, in general we need the whole state, but as we do not want to implement texturing I hope the state does not change that often and it might be possible to use the same state for the whole scene (except for the model-view matrix and vertex data of course). If I was to tackle render graph distribution (I have been contemplating this approach for number of years) I'd sereailize the whole render graph and distribute this to the cluster. One would take care to cache objects like osg::StateSet's, osg::StateAttribute and osg::Drawables locally, and only update them any time they are updated. osgIntrospection could potentially be used for serialization. The slaves would then listen to render graph dispatch then cull the render graph down to just the required viewing volume then rendering, then dispatch back their portion of the colour and depth (if required) buffers.
Re: [osg-users] Distributed rendering of scene graph
Hello Robert, thank you again for your helpful advice. I will meet with our team tomorrow and talk with them and decide what approach we will implement in the end. Regards, Benjamin On Sunday 02 March 2008 at 14:58:41, Robert Osfield wrote: On Sun, Mar 2, 2008 at 11:41 AM, Benjamin Eikel [EMAIL PROTECTED] wrote: Thanks for the more structured explanation. I'm not sure that the overall approach will be the most efficient way to tackle the task for a wide range of usages, so I'd recommend exploring the possibilities of different ways to implement that various usages rather than just sticking to one approach - different approaches are very likely to be suitable for different usages. You have a point there. But we want to implement a very specialized system for only our kind of factory scenes. Of course our system will perform very bad with other kind of scenes (nature for example). Even for you factory scene what approach to use will depend upon what you are viewing - its relatively small section then a distributed approach will be slower than a local rendering. You have to bare in mind the overheads and bottlenecks inherit with a distributed system, throwing lots of hardware at the problem won't help if you are introducing greater bottlenecks in the process of using the cluster. What it sounds like you are suggesting/looking for is a distributed render graph. The OSG's render graph is the rendering back end that the OSG produces from its cull traversal. The render graph containts osgUtil::RenderStage/Bin, a osgUtil::StateGraph and osgUtil::RenderLeaf. The RenderLeaf contain pointers to the projection and modelview matrices, the drawable and the point on the StateGraph that specifies the accumulated state that must applied for that drawable. Thank you for that insight. That was something I did not know and that was the part that was missing in my puzzle. This sound exactly like the thing I am searching. So if I want to get this RenderStage object I could get it from the SceneView after doing the update and cull traversal? You get the root RenderStage after cull traversal. osgViewer::Renderer has the SceneView that you can get the RenderStage from. The root RenderStage has everything you need. Might I recommend using osgViewer as your base and then just having a custom renderingTraversals method that you can override the cull and draw traversals if need be. If need be is the key part as you may well find using standard local rendering will often be better than a distributed implementation. Okay right. We will only have one render bin which eases the thing a little bit because we will not support transparent objects. Much of modern rendering is done with multiple pass techniques that go well beyond just the need for transparent bins. Don't forget there techniques like using of the occlussion query extension requires multipass, and its one technique that is well suited for complex CAD models. Paul Martz was even commission recently to add support for this in the OSG for the purpose of better handling CAD models. So, while limiting what types of rendering you can use to just standard opaque models might make your tasks easier, there is a real danger of excluding modern rendering techniques that help solve the performance problems in a better way. I don't think you should be doing things at as low level as trying to overriding the OpenGL functions that the OSG uses in its Drawables and StateAttributes. To go this low level you might as well use a distributed GL implementation and save yourself a lot of time. You mention transforms and drawables in your text, whereas you actually need all OpenGL state that the OSG manages as well, you really do need the whole render graph, yes it is more complex but its what is actually required to get things to render properly. Yes you are right. I did not know that this render graph does exist and only looked at the documentation and the code. But I did not get this insight that you gave me by your explanations above. So I assumend that overwriting the DrawImplementation for getting the information about the visible nodes would be a possible way. It might be possible, but as you said, not a good approach. Yes you are right, in general we need the whole state, but as we do not want to implement texturing I hope the state does not change that often and it might be possible to use the same state for the whole scene (except for the model-view matrix and vertex data of course). If I was to tackle render graph distribution (I have been contemplating this approach for number of years) I'd sereailize the whole render graph and distribute this to the cluster. One would take care to cache objects like osg::StateSet's, osg::StateAttribute and osg::Drawables locally, and only update them any time
Re: [osg-users] Distributed rendering of scene graph
Hi Benjamin, On Sat, Mar 1, 2008 at 12:24 AM, Benjamin Eikel [EMAIL PROTECTED] wrote: Have you explored multiple CPU/GPU set ups on a single machine? We have two CPUs but only one GPU on each node in the cluster. Have you fully explored scene graph/OpenGL optimization? I think you mean that we do not needed to use a cluster at all, because the scenes can be rendered on one computer. I think I can answer that with no. We are trying to create a parallel rendering algorithm and want to check how well it scales with the number of cluster nodes used. In the end it will hopefully be able to render really massive scenes (some hundred millions polygons) You can see it as some kind of research. Ahh, understood, its research rather than just trying to solve a specific performance issue. BTW, modern GPU's can do hundreds of millions of polygons per second right now ;-) As its research might I suggest benchmarking different types of scenes between a single workstation with a single GPU, multiple GPU's and the cluster. I strongly suspect it's only a small set of scenes that will favour a cluster. So it is no option to only use a single big workstation. We do not want to use the rendering of OpenSceneGraph (or Chromium which was suggested in another e-mail) because we want to implement some algorithm ourselves to be able to handle these massive scenes (dynamical LOD with loose octrees for big models for example). When writing these rendering ourselves, we have more freedom in doing so. Of course we could write the whole system without using OpenSceneGraph. But as stated before, it would be very nice to use some of its features on our central computer so we do not have to implement things like picking, dragging, culling and so on. We already began the implementation and now the question is, as stated in my first mail, how we can use the power of OpenSceneGraph on the front end to ease our lives. I have already used OSG in my Bachelor's thesis to implement adaptive animation algorithms and was very pleased by it's features. But I am not quite sure if the way I described in my first mail is the right one to follow. I do not know if it might be better to put the interface between this front end node and the rendering slaves somewhere else. So further hints are greatly appreciated. There are different levels of distributed rendering. Low level like - distributed GL like Chomium, through to high level distributed IG where all nodes have a local copy of all the data with high level state like camera matrices etc. sync'd from the master. What level of granularity are you aiming for? Will you have the opportunity to test/develop various levels? Robert. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robert Osfield wrote: There are different levels of distributed rendering. Low level like - distributed GL like Chomium, through to high level distributed IG where all nodes have a local copy of all the data with high level state like camera matrices etc. sync'd from the master. What level of granularity are you aiming for? Will you have the opportunity to test/develop various levels? There is also Equalizer, which looks like it is designed to do exactly what these guys want: http://www.equalizergraphics.com/ Jan -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mandriva - http://enigmail.mozdev.org iD8DBQFHyVW5n11XseNj94gRAjzoAKC7CyPo3PcT1MdM1KBfnM+8QIm6vgCdHbpZ hTUxZ6iQIihSEOPWx1pZRXM= =SprR -END PGP SIGNATURE- ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
Hello Robert, On Saturday 01 March 2008 at 13:22:35, Robert Osfield wrote: Hi Benjamin, On Sat, Mar 1, 2008 at 12:24 AM, Benjamin Eikel [EMAIL PROTECTED] wrote: Have you explored multiple CPU/GPU set ups on a single machine? We have two CPUs but only one GPU on each node in the cluster. Have you fully explored scene graph/OpenGL optimization? I think you mean that we do not needed to use a cluster at all, because the scenes can be rendered on one computer. I think I can answer that with no. We are trying to create a parallel rendering algorithm and want to check how well it scales with the number of cluster nodes used. In the end it will hopefully be able to render really massive scenes (some hundred millions polygons) You can see it as some kind of research. Ahh, understood, its research rather than just trying to solve a specific performance issue. exactly. BTW, modern GPU's can do hundreds of millions of polygons per second right now ;-) Okay, but we do not have such modern GPUs and furthermore we just could increase the complexity of the scene then and the GPU cannot cope with the scene anymore. As its research might I suggest benchmarking different types of scenes between a single workstation with a single GPU, multiple GPU's and the cluster. I strongly suspect it's only a small set of scenes that will favour a cluster. We want to visualize scenes which are generated by a simulation of a factory. The simulation is running on the cluster too. We want to do the rendering on the cluster because we can get messages from the simulation faster (10 GBit/s InfiniBand, lower latency than Ethernet, DMA transfers from one node to another) and are able to cope with big scenes by just using more rendering nodes (at least that is our goal). For example models of some machines are very complex because they are generated by CAD programs and we do not want to do an extensive preprocessing because the models may change over time. So it is no option to only use a single big workstation. We do not want to use the rendering of OpenSceneGraph (or Chromium which was suggested in another e-mail) because we want to implement some algorithm ourselves to be able to handle these massive scenes (dynamical LOD with loose octrees for big models for example). When writing these rendering ourselves, we have more freedom in doing so. Of course we could write the whole system without using OpenSceneGraph. But as stated before, it would be very nice to use some of its features on our central computer so we do not have to implement things like picking, dragging, culling and so on. We already began the implementation and now the question is, as stated in my first mail, how we can use the power of OpenSceneGraph on the front end to ease our lives. I have already used OSG in my Bachelor's thesis to implement adaptive animation algorithms and was very pleased by it's features. But I am not quite sure if the way I described in my first mail is the right one to follow. I do not know if it might be better to put the interface between this front end node and the rendering slaves somewhere else. So further hints are greatly appreciated. There are different levels of distributed rendering. Low level like - distributed GL like Chomium, through to high level distributed IG where all nodes have a local copy of all the data with high level state like camera matrices etc. sync'd from the master. What level of granularity are you aiming for? Will you have the opportunity to test/develop various levels? We get ids for the machine models from the simulation. We then want to load the meshes for that machine inside the scene graph (for this we can use the already existing reader/writer plugins of OSG and do not have to write our own). So the scene graph knows the bounding boxes and models and can do culling, picking, moving machines with grabbers and so on. Furthermore we want to use the animation capabilities for some special models (e. g. forklifts, packages on a conveyor). When a frame should be rendererd we want to know the nodes (e. g. Geodes) which are visible and their transformation matrix (which should be known to OSG after it's update and culling traversals). There is one central node with a scene graph for every user connected to our simulation/rendering system (so we are able to have different views into the scene and can distribute the culling for every user to an own node). After that we know the visible machines (and are able to estimate the costs for the rendering). We then do a load balancing between the different users viewing the simulation and the different rendering nodes to create a partitioning of the set of render nodes. Each subset of render nodes is assigned to one of the central nodes. After each of this central nodes knows these rendering nodes which can be used, it instructs these nodes by
Re: [osg-users] Distributed rendering of scene graph
On Saturday 01 March 2008 at 14:10:17, Jan Ciger wrote: Robert Osfield wrote: There are different levels of distributed rendering. Low level like - distributed GL like Chomium, through to high level distributed IG where all nodes have a local copy of all the data with high level state like camera matrices etc. sync'd from the master. What level of granularity are you aiming for? Will you have the opportunity to test/develop various levels? There is also Equalizer, which looks like it is designed to do exactly what these guys want: http://www.equalizergraphics.com/ Hello Jan, thank you for your suggestion. Equalizer looks like a prebuilt framework for simple, high-performance parallel rendering. This is not what we are searching for, because we want to/have to implement our own specialized rendering system. Regards, Benjamin Jan ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
Hi Benjamin, Could you possibly repost the below text, but broken down into shorter paragraphs as long paragraphs are really hard to follow. Try to keep one major point per paragraph. You are tackling a big topic in terms of performance optimization, so its worth taking it bit by bit. Robert. On Sat, Mar 1, 2008 at 2:26 PM, Benjamin Eikel [EMAIL PROTECTED] wrote: Hello Robert, On Saturday 01 March 2008 at 13:22:35, Robert Osfield wrote: Hi Benjamin, On Sat, Mar 1, 2008 at 12:24 AM, Benjamin Eikel [EMAIL PROTECTED] wrote: Have you explored multiple CPU/GPU set ups on a single machine? We have two CPUs but only one GPU on each node in the cluster. Have you fully explored scene graph/OpenGL optimization? I think you mean that we do not needed to use a cluster at all, because the scenes can be rendered on one computer. I think I can answer that with no. We are trying to create a parallel rendering algorithm and want to check how well it scales with the number of cluster nodes used. In the end it will hopefully be able to render really massive scenes (some hundred millions polygons) You can see it as some kind of research. Ahh, understood, its research rather than just trying to solve a specific performance issue. exactly. BTW, modern GPU's can do hundreds of millions of polygons per second right now ;-) Okay, but we do not have such modern GPUs and furthermore we just could increase the complexity of the scene then and the GPU cannot cope with the scene anymore. As its research might I suggest benchmarking different types of scenes between a single workstation with a single GPU, multiple GPU's and the cluster. I strongly suspect it's only a small set of scenes that will favour a cluster. We want to visualize scenes which are generated by a simulation of a factory. The simulation is running on the cluster too. We want to do the rendering on the cluster because we can get messages from the simulation faster (10 GBit/s InfiniBand, lower latency than Ethernet, DMA transfers from one node to another) and are able to cope with big scenes by just using more rendering nodes (at least that is our goal). For example models of some machines are very complex because they are generated by CAD programs and we do not want to do an extensive preprocessing because the models may change over time. So it is no option to only use a single big workstation. We do not want to use the rendering of OpenSceneGraph (or Chromium which was suggested in another e-mail) because we want to implement some algorithm ourselves to be able to handle these massive scenes (dynamical LOD with loose octrees for big models for example). When writing these rendering ourselves, we have more freedom in doing so. Of course we could write the whole system without using OpenSceneGraph. But as stated before, it would be very nice to use some of its features on our central computer so we do not have to implement things like picking, dragging, culling and so on. We already began the implementation and now the question is, as stated in my first mail, how we can use the power of OpenSceneGraph on the front end to ease our lives. I have already used OSG in my Bachelor's thesis to implement adaptive animation algorithms and was very pleased by it's features. But I am not quite sure if the way I described in my first mail is the right one to follow. I do not know if it might be better to put the interface between this front end node and the rendering slaves somewhere else. So further hints are greatly appreciated. There are different levels of distributed rendering. Low level like - distributed GL like Chomium, through to high level distributed IG where all nodes have a local copy of all the data with high level state like camera matrices etc. sync'd from the master. What level of granularity are you aiming for? Will you have the opportunity to test/develop various levels? We get ids for the machine models from the simulation. We then want to load the meshes for that machine inside the scene graph (for this we can use the already existing reader/writer plugins of OSG and do not have to write our own). So the scene graph knows the bounding boxes and models and can do culling, picking, moving machines with grabbers and so on. Furthermore we want to use the animation capabilities for some special models (e. g. forklifts, packages on a conveyor). When a frame should be rendererd we want to know the nodes (e. g. Geodes) which are visible and their transformation matrix (which should be known to OSG after it's update and culling traversals). There is one central node with a scene graph for every user connected to our simulation/rendering system (so we are able to have different views into the scene and
Re: [osg-users] Distributed rendering of scene graph
Hello Robert, On Saturday 01 March 2008 at 18:50:16, Robert Osfield wrote: Hi Benjamin, Could you possibly repost the below text, but broken down into shorter paragraphs as long paragraphs are really hard to follow. Try to keep one major point per paragraph. of course. o What do we want to display? We want to visualize scenes which are generated by a simulation of a factory. o Why do we want to use a cluster? The simulation is running on a cluster. We want to do the rendering on the cluster because - we can get messages from the simulation faster (10 GBit/s InfiniBand, lower latency than Ethernet, DMA transfers from one node to another) - we are able to cope with big scenes by just using more rendering nodes (at least that is our goal). - we have the computation power there. o Why do we have big scenes? Models of some machines are very complex because they are generated by CAD programs and we do not want to do an extensive preprocessing because the models may change over time. o What is the architecture of the system? - We have a central node for each user which is connected to our system. On that node we want to use OpenSceneGraph (reasons below). So each of these nodes has an own view into the scene. - We have a set of rendering nodes. These rendering nodes are partitioned by a balancing algorithm and each central node gets some of them for rendering. We want to balance between different users and between the rendering nodes one central node is able to render with. Example: User A sees only a small part of the factory = needs only few rendering nodes. User B sees some very complex machines = needs lots of rendering nodes. Example: Central node A can render with 20 rendering nodes. It has 20 objects with 1 million polygons and so it gives each rendering node one object. o How to we get the scene data from the simulation? We get a identification number for each machine mesh from the simulation. We then want to load the meshes for that machine inside the scene graph. o Why do we want to use OpenSceneGraph? - We can use the already existing reader/writer plugins of OSG and do not have to write our own. - OSG can do culling, picking, moving machines with grabbers for us. - We want to use the animation capabilities of OSG for some special models (e. g. forklifts, packages on a conveyor). o How will OpenSceneGraph be connected to the rendering nodes? When a frame should be rendererd we want to know the nodes (e. g. Geodes) which are visible and their transformation matrix (which should be known to OSG after it's update and culling traversals). With this knowledge (list of visible nodes and transformations) the central nodes instruct their rendering nodes by sending them the mesh ids, the transformations of the meshes and the camera transformation. o How can we optimize the rendering on the rendering nodes? We want to exploit the fact that there may be the same machine (= same mesh) multiple times inside the factory. So we could load the mesh of such a machine as VBO and let one rendering node render the machines with different transformations multiple times. o What is my question at all? I am searching for a interface where I can grab these visible nodes and put them into a list. I need them so I can analyse them to run the balancing algorithm and send the data to the rendering nodes. I think that it might be possible to use the DrawImplementation of the Drawable nodes. Regards, Benjamin ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
Benjamin Eikel wrote: Hello Robert, On Saturday 01 March 2008 at 18:50:16, Robert Osfield wrote: Hi Benjamin, Could you possibly repost the below text, but broken down into shorter paragraphs as long paragraphs are really hard to follow. Try to keep one major point per paragraph. of course. o What do we want to display? We want to visualize scenes which are generated by a simulation of a factory. o Why do we want to use a cluster? The simulation is running on a cluster. We want to do the rendering on the cluster because - we can get messages from the simulation faster (10 GBit/s InfiniBand, lower latency than Ethernet, DMA transfers from one node to another) - we are able to cope with big scenes by just using more rendering nodes (at least that is our goal). - we have the computation power there. o Why do we have big scenes? Models of some machines are very complex because they are generated by CAD programs and we do not want to do an extensive preprocessing because the models may change over time. o What is the architecture of the system? - We have a central node for each user which is connected to our system. On that node we want to use OpenSceneGraph (reasons below). So each of these nodes has an own view into the scene. - We have a set of rendering nodes. These rendering nodes are partitioned by a balancing algorithm and each central node gets some of them for rendering. We want to balance between different users and between the rendering nodes one central node is able to render with. Example: User A sees only a small part of the factory = needs only few rendering nodes. User B sees some very complex machines = needs lots of rendering nodes. Example: Central node A can render with 20 rendering nodes. It has 20 objects with 1 million polygons and so it gives each rendering node one object. o How to we get the scene data from the simulation? We get a identification number for each machine mesh from the simulation. We then want to load the meshes for that machine inside the scene graph. o Why do we want to use OpenSceneGraph? - We can use the already existing reader/writer plugins of OSG and do not have to write our own. - OSG can do culling, picking, moving machines with grabbers for us. - We want to use the animation capabilities of OSG for some special models (e. g. forklifts, packages on a conveyor). o How will OpenSceneGraph be connected to the rendering nodes? When a frame should be rendererd we want to know the nodes (e. g. Geodes) which are visible and their transformation matrix (which should be known to OSG after it's update and culling traversals). With this knowledge (list of visible nodes and transformations) the central nodes instruct their rendering nodes by sending them the mesh ids, the transformations of the meshes and the camera transformation. o How can we optimize the rendering on the rendering nodes? We want to exploit the fact that there may be the same machine (= same mesh) multiple times inside the factory. So we could load the mesh of such a machine as VBO and let one rendering node render the machines with different transformations multiple times. o What is my question at all? I am searching for a interface where I can grab these visible nodes and put them into a list. I need them so I can analyse them to run the balancing algorithm and send the data to the rendering nodes. I think that it might be possible to use the DrawImplementation of the Drawable nodes. this is not an answer to your questions about your interesting project, only some thoughts about engineering stuff and simulation. http://www.realityprime.com/articles/scenegraphs-past-present-and-future#tomorrow http://www.caelinux.com/CMS/ what i didn´t find is a realtime grid for the balancing stuff. are there more infos about the project, maybe on a website? best wishes for your success! ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello Benjamin, Benjamin Eikel wrote: Hello Jan, thank you for your suggestion. Equalizer looks like a prebuilt framework for simple, high-performance parallel rendering. This is not what we are searching for, because we want to/have to implement our own specialized rendering system. As far as I remember, you can plug in your own rendering and compositing stages into Equalizer. Sometimes it is better to start from something working than from scratch, even though it is tempting to reinvent the wheel (I am/was in academia too). Just a suggestion, though. Jan -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mandriva - http://enigmail.mozdev.org iD8DBQFHyav7n11XseNj94gRAkoDAJ4vATGD/4u3EdKq9+8TxjXNKKB0OACfQBjN o2UVnz0F80uTt313QaSF5cc= =7pdg -END PGP SIGNATURE- ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
On Saturday 01 March 2008 at 20:06:02, [EMAIL PROTECTED] wrote: Benjamin Eikel wrote: Hello Robert, On Saturday 01 March 2008 at 18:50:16, Robert Osfield wrote: Hi Benjamin, Could you possibly repost the below text, but broken down into shorter paragraphs as long paragraphs are really hard to follow. Try to keep one major point per paragraph. of course. o What do we want to display? We want to visualize scenes which are generated by a simulation of a factory. o Why do we want to use a cluster? The simulation is running on a cluster. We want to do the rendering on the cluster because - we can get messages from the simulation faster (10 GBit/s InfiniBand, lower latency than Ethernet, DMA transfers from one node to another) - we are able to cope with big scenes by just using more rendering nodes (at least that is our goal). - we have the computation power there. o Why do we have big scenes? Models of some machines are very complex because they are generated by CAD programs and we do not want to do an extensive preprocessing because the models may change over time. o What is the architecture of the system? - We have a central node for each user which is connected to our system. On that node we want to use OpenSceneGraph (reasons below). So each of these nodes has an own view into the scene. - We have a set of rendering nodes. These rendering nodes are partitioned by a balancing algorithm and each central node gets some of them for rendering. We want to balance between different users and between the rendering nodes one central node is able to render with. Example: User A sees only a small part of the factory = needs only few rendering nodes. User B sees some very complex machines = needs lots of rendering nodes. Example: Central node A can render with 20 rendering nodes. It has 20 objects with 1 million polygons and so it gives each rendering node one object. o How to we get the scene data from the simulation? We get a identification number for each machine mesh from the simulation. We then want to load the meshes for that machine inside the scene graph. o Why do we want to use OpenSceneGraph? - We can use the already existing reader/writer plugins of OSG and do not have to write our own. - OSG can do culling, picking, moving machines with grabbers for us. - We want to use the animation capabilities of OSG for some special models (e. g. forklifts, packages on a conveyor). o How will OpenSceneGraph be connected to the rendering nodes? When a frame should be rendererd we want to know the nodes (e. g. Geodes) which are visible and their transformation matrix (which should be known to OSG after it's update and culling traversals). With this knowledge (list of visible nodes and transformations) the central nodes instruct their rendering nodes by sending them the mesh ids, the transformations of the meshes and the camera transformation. o How can we optimize the rendering on the rendering nodes? We want to exploit the fact that there may be the same machine (= same mesh) multiple times inside the factory. So we could load the mesh of such a machine as VBO and let one rendering node render the machines with different transformations multiple times. o What is my question at all? I am searching for a interface where I can grab these visible nodes and put them into a list. I need them so I can analyse them to run the balancing algorithm and send the data to the rendering nodes. I think that it might be possible to use the DrawImplementation of the Drawable nodes. this is not an answer to your questions about your interesting project, only some thoughts about engineering stuff and simulation. http://www.realityprime.com/articles/scenegraphs-past-present-and-future#to morrow http://www.caelinux.com/CMS/ what i didn´t find is a realtime grid for the balancing stuff. are there more infos about the project, maybe on a website? Hello Hannes (I suppose), the website of our project group is here: http://wwwhni.uni-paderborn.de/alg/lehre/ebkfs/remosim/ But it is only available in German I am sorry. Another project which is a little bit connected to ours is this (English description available): http://wwwhni.uni-paderborn.de/en/alg/projects/avipasia/ best wishes for your success! Thank you very much. Regards, Benjamin ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
On Saturday 01 March 2008 at 20:18:19, Jan Ciger wrote: Hello Benjamin, Benjamin Eikel wrote: Hello Jan, thank you for your suggestion. Equalizer looks like a prebuilt framework for simple, high-performance parallel rendering. This is not what we are searching for, because we want to/have to implement our own specialized rendering system. As far as I remember, you can plug in your own rendering and compositing stages into Equalizer. Hello Jan, okay, this might be something that we could use. I have only skimmed through the web page and the programming guide so maybe I missed something. The problem that I see is that maybe other functions which are provided by OpenSceneGraph are not supported by Equalizer (loading from different file formats, picking/dragging, animation pathes). Furthermore if we want to write our own rendering and composition algorithm what is left for Equalizer to do for us? I think the effort getting used to it and integrating it in our system would be too high in comparison to the earnings. But I will read the programming guide tomorrow and see if my first impression is wrong. Sometimes it is better to start from something working than from scratch, even though it is tempting to reinvent the wheel (I am/was in academia too). Yes that is right. That is why we want to use OpenSceneGraph. Just a suggestion, though. I am thankful for every serious suggestion. Regards, Benjamin Jan ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
Hello Benjamin, we want to use OpenSceneGraph for a parallel rendering system. The scene graph should be located on a central computer which gets the user input, modifies the scene graph (with draggers) and displays the rendered image. The rendering should be done on other computers. We want to use a sort-last approach, where the geometry is distributed to these other computers and they send back an image with depth information that is composed on the central computer. Perhaps not the answer you want, but before doing all this work yourself, have you looked at Chromium? From what I can see, it can do this out of the box. I have never used it (attempts to get it working on Windows have failed) but it could be a good idea to try. Good luck, J-S -- __ Jean-Sebastien Guay[EMAIL PROTECTED] http://www.cm-labs.com/ http://whitestar02.webhop.org/ ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
[osg-users] Distributed rendering of scene graph
Hello OpenSceneGraph community, we want to use OpenSceneGraph for a parallel rendering system. The scene graph should be located on a central computer which gets the user input, modifies the scene graph (with draggers) and displays the rendered image. The rendering should be done on other computers. We want to use a sort-last approach, where the geometry is distributed to these other computers and they send back an image with depth information that is composed on the central computer. My question is: How can I get the nodes which are determined visible after culling? I just want to know these nodes, so that I can get their vertex, color, ... data and send it together with the current model-view matrix via MPI to the other nodes. The other nodes should be able to render the objects and they should not need an own scene graph themselves. One solution I thought of is setting the DrawCallback on all Drawable nodes inside the scenegraph. I think this own DrawCallback can read out the vertex, color, ... arrays (for example if it is a Geometry node) and get the current model-view matrix for that node and send the data. Do you think this own DrawCallback is the right approach? Regards, Benjamin Eikel ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
Hi Benjamin, Before heading off in any deep technical discussion I'd suggest answer oneself a few questions. The aim is to find an easy route to solving whatever problem you have, going the cluster route might be worthwhile, but then it might not be needed at all. What is your motivation for going the cluster route? Have you explored multiple CPU/GPU set ups on a single machine? Have you fully explored scene graph/OpenGL optimization? Robert. On Fri, Feb 29, 2008 at 7:37 PM, Benjamin Eikel [EMAIL PROTECTED] wrote: Hello OpenSceneGraph community, we want to use OpenSceneGraph for a parallel rendering system. The scene graph should be located on a central computer which gets the user input, modifies the scene graph (with draggers) and displays the rendered image. The rendering should be done on other computers. We want to use a sort-last approach, where the geometry is distributed to these other computers and they send back an image with depth information that is composed on the central computer. My question is: How can I get the nodes which are determined visible after culling? I just want to know these nodes, so that I can get their vertex, color, ... data and send it together with the current model-view matrix via MPI to the other nodes. The other nodes should be able to render the objects and they should not need an own scene graph themselves. One solution I thought of is setting the DrawCallback on all Drawable nodes inside the scenegraph. I think this own DrawCallback can read out the vertex, color, ... arrays (for example if it is a Geometry node) and get the current model-view matrix for that node and send the data. Do you think this own DrawCallback is the right approach? Regards, Benjamin Eikel ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Distributed rendering of scene graph
Am Freitag, 29. Februar 2008 20:54:08 schrieb Jean-Sébastien Guay: Hello Benjamin, we want to use OpenSceneGraph for a parallel rendering system. The scene graph should be located on a central computer which gets the user input, modifies the scene graph (with draggers) and displays the rendered image. The rendering should be done on other computers. We want to use a sort-last approach, where the geometry is distributed to these other computers and they send back an image with depth information that is composed on the central computer. Perhaps not the answer you want, but before doing all this work yourself, have you looked at Chromium? From what I can see, it can do this out of the box. I have never used it (attempts to get it working on Windows have failed) but it could be a good idea to try. Good luck, J-S Hello Jean-Sébastien, thank you for your answer. But we do not want to use an already working parallel rendering system because we are working in a project group at University and want to implement our own rendering algorithm which is (hopefully) optimized for the scenes we are going to display. We want to use OpenSceneGraph only as help so that it does things like picking, dragging etc. for us to speed up the implementation. Regards, Benjamin ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
