Dear Pablo, dear all,
Please let me give some comments on the clustering issue as its a topic I
have a strong interest for and Ive a bit of experience on.
Its not strictly related to OpenSG but I hope it could be of interest
anyway.
Distributing data over the network you will always face bandwidth problems,
no matter what software technology you use to do it. From our personal
experience we found out that there is no way (yet) to send at a reasonable
framerate more than few thousand vertices per frame.
We have a setup where a master node is broadcasting (very much like
Chromium but its a completely custom software) OpenGL commands over the a
Gigabit Ethernet. What we do is simple: every frame we take all the
GLCommands and related data, pack them in a series of BYTE chunks so that
once sent they will not exceed the UDP payload (about 64kb), compress each
of them to save on bandwidth using some fast Huffman compressor, and
broadcast them to the clients. On the client side we do similar things in
the reverse order.
We have done several tests (different compressor, different payload size,
different segmentation algorithms, different Ethernet capacity) and we
squeezed out of the framework, I believe, more or less all we could (we are
still working on heavy multithreading for master-clients with quad core
processors, but that is a bit extreme). Our latest optimizations just give
us increases of 3-4% each time and, unless there is something that we really
did not think about, I dont think well be getting much more.
Our conclusion is that a Gigabit link is perfect as long as your graphics
engine is mostly using static geometry (display lists or vertex arrays), but
if you have some CPU deformed meshes (as I think its your case) you will
not get a reasonable framerate for more than lets say 100 ktris per
frame.
So, if you really want to have complex deformed geometry you only have 2
reasonably directions to explore:
1) Use a faster LAN (10G maybe)
2) Perform the same deformations on all the slaves.
Solution 2 is very tempting, but somewhat hard to achieve in a general way.
You will have to replicate your deformation code on all the nodes and keep
all the nodes in sync not only on the graphics but also on the deformation
algorithm. This has a very strong requirement: your code need to be
deterministic, or computation on different clients may diverge. Also,
distributing CPU code can be almost impossible on a C++ framework (but you
might consider using a Virtual Machine and some scripting language).
A neat and clean solution could then be to have your deformation code to be
purely implemented on the GPU. Shaders are practically a very, very
efficient scripting language with a very powerful VM implemented already in
the OpenGL video driver. Distributing deformation shaders can be much more
efficient (and simpler) than distributing deformation code. Obviously there
are limitations in what you can do with a shader right now, and this might
not be solution in your case, but you might want to consider this option.
This said, I know that clustering is a much more complex topic than this,
and I hope you will excuse the shortcuts I took to express my opinion.
Just my 2 cents,
Franco
---
Eng. Franco Tecchia, Ph.D.
PERCRO - Scuola Superiore S.Anna
Polo Sant'Anna Valdera
Viale Rinaldo Piaggio, 34
56025 - Pontedera (Pisa), Italy
<http://www.percro.org/> http://www.percro.org/
Tel: +39 050 883 481
Fax: +39 050 883 333
Da: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Per conto di Pablo
Carneiro Elias
Inviato: 14 May 2007 18:31
A: [email protected]
Oggetto: Re: [Opensg-users] About OpenSG clustering
> Can you give us a little intro to your problem?
Hi Dirk.
Well My project is a bit complex. It actually goes much farther than the
scene graph. We´re working on a BIG system that basically consists on a
browser that can render many kinds of things: Engineering Objects (such as
Ships and Big constructs), Scientific Objects (such as Seismic data) and
Many, many other things, all together within the same scene (eg. a ship at
ocean with all engineering info, and the soil below with seismic info.. and
many other things.) It is a long story... but I will focus on the scene
graph part of the thing that is what I need to study by now:
Among many other things, We´ll use VTK to make many calculations for
scientific objects. But, we´re going to render Everything using OpenSG and
clusters. So, we need somehow to make an integration between VTK geometry
nodes and OpenSG, as OpenSG must have full control of the geometry data and
scene data so it can make the hard work of putting everything on cluster
transparently.
A part from the vtk-openSG integration problem (which is not solved yet, but
I´m assuming that everything has already been placed at OpenSG correctly for
now), as VTK data will possibly be very large (the engineering data will be
very large too, but it will not change so it is not a problem) and it is
going to change many times due to numerical calculations, we need a way to
handle this very efficiently using clusters.
The task that is assigned to me is to study everything about OpenSG
clustering (at low level and high level details), so I can make a preview of
what will happen in this kind of situation just mentioned above. I have to
make a good report explaining about how OpenSG clustering support works, how
data serialization is done, what are the possible cluster strategies
supported by OpenSG (SortLast and SortFirst) and how to use them (including
source code of examples working). I must explain how everything works from
Top to bottom. ;) The High leve part its quite simple to understand and use,
since everything is very well projected and organized on OpenSG. But I´m
still trying to understand the low level part of things....
By now I already have a good picture of how basic things work behind the
scene in OpenSG, I´ve already read about Threading, Aspects, ChangeLists and
RemoteAspects, which, I think, are the base classes for Threading support
and for clustering support too.
But I don´t have much information yet about how fields are serialized over
the network (there is any magical strategy of sending them or it is normal
bit transfer to the network?)
Thats it.. the SceneGraph related part of the project is basically that, and
we´ve chose OpenSG ;) Now I´m searching for info until tomorrow night in
order to finish the report so We can start developing things soon (after a
good analysis if the report) ;)
thanks for all help ;)
2007/5/14, Dirk Reiners <[EMAIL PROTECTED]>:
Hi Pablo,
Pablo Carneiro Elias wrote:
> Thanks Dirk and Marcus (Sorry about the mistake at your name Marcus).
>
> more questions,
>
> 1. using the ProxyGroup I´ll solve a memory issue when using LARGE
> geometry, thats fine (and thats my case). But what If I may need to
> change something on geometry (physical simulations). As the application
> lies at client side, and the client side has only a proxy to the data,
> not the data itself, who can I do that?
Right now there is no good way to do that, sorry.
> 2. My real problem is that I have to deal with very large geometry data
> and it is changing every time. What I want to do is to minimize the need
> for sending the whole gometry every time. I want to send the least data
> possible to the network.Everything should run at real time (I know its a
> difficult task). Is there any known or recommended way to that with
OpenSG?
If you're recalculating large parts of your scene for every frame you
need to do that on the slaves, to utilize all the processors. OpenSG
doesn't directly help you with that at this point.
You can use the OpenSG infrastructure for it, but you would have to
write your own NodeCore that does the calculations when it's asked to
rerender itself, and you would need to be able to calculate the counding
box of the slaves' contributions on the master, so that the SortLast
compositors can schedule the data transfer correctly. You would also
have to add some code to the SortLast cluster logic to recognize your
new Nodes.
So it's not that trivial right now. It's something that I'm very
interested in, but have not been able to spend time on. Can you give us
a little intro to your problem?
Yours
Dirk
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Opensg-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensg-users
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Opensg-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensg-users
