Dear Maxim, Actually I already set the border to be 0 to avoid the padding problem. So I set the range to [0 0.248] didn't help.
When comparing the output, I found that only difference is that when range is [0 1], the TD.D = 1, and when range is [0, 0.25], TD.D = 4. I set the TD.D = 1. , the rendering is fast now, but the scaling is wrong. Anywhere else the TD.D is used in rendering? Maybe that affect the performance? Jinghua On Wed, Feb 18, 2009 at 8:38 AM, Maxim Makhinya <[email protected]> wrote: > > Hello Jinghua, > > Now you did correct test, and as I suspected in my first reply > one of your machines performs bad, as they should render same > amount of data, but the data should be different for different > machines. > > Regarding this difference, when you use range [0 0.25], I could > suspect following: > > 1) 3D textures should have size of power of 2, so if you give > size which is not it will be padded to power of 2 and some > space on your graphics card is wasted. > > 2) eVolve creates small borders of 1-2 voxels on the edges > between two slabs of volume for correct blending between > slabs. > > Form 1) and 2) - if your [0 0.25] is exactly power of 2, 3D > texture will be actually much bigger (twice in fact) because of > the border and that texture overkill. When you render it as > range [0 1] it fits perfectly on GPU and renders faster because > borders are not added. If this is the case - simple solution > would be to render a bit smaller total volume, taking in to > account these borders. > > Check from the source code which borders there are added and > try to render slightly smaller volume than exact power of 2 > (or use smaller ranges, say [0 0.248] or so). > > > Best regards, > > Makhinya Maxim > > > > > On Feb 18, 2009, at 3:18 PM, jinghua wrote: > > > > > Dear Maxim and Stefan, > > > > I am doing tests today, and found out sth. which I believe is my > > problem. > > > > When I do 1 node test, I used to change the .vhf file d value to be > > 256. (I > > change d=2048 for 8 node test and d = 1024 for 4 node test) > > > > w=1024 > > h=1024 > > d=256 > > > > and in my config file the range is [0 1] > > > > It renders at 16 fps. > > > > But today I decided that I don't want to change the .vhf file all > > the time, > > instead I change the config file. So I set d=1024 in .vhf file. For > > 4 node > > test, each node get range [0, 0.25], [0.25 0.5], [0.5, 0.75], [0.75, > > 1.0]. > > With no compund it renders at 6.7 fps. Then I did the 1 node test, > > where the > > range I set as [0 0.25]. it also renders at 6.7 fps! > > > > So that must be the problem right? I am checking my rawVolModel.cpp > > file > > now, see if I did something when loading the data (I don't believe > > so tho). > > Why setting the range to be [0. 0.25] make the 1 node render slow? > > > > Yesterday I sent this post with two attachments of the 4 node and 1 > > node > > statistics screenshots. Somehow it didn't get through. This time I am > > omitting the attachment. > > > > Thanks! > > > > Jinghua > > > > > > > > Maxim Makhinya wrote: > >> > >> > >> Hello Jinghua, > >> > >> Your config file seem to be right, there should be no additional > >> transfers as long as you don't have any input-output frames. > >> > >> Do you have your data locally cached? Do you load it only once > >> per-node? Might be a silly question, but, are you using > >> CXXFLAGS=-O to compile it in release not debug? =) > >> > >> What you can do else is: > >> > >> 1) check what rendering statistics show: > >> > http://www.equalizergraphics.com/documents/design/statisticsOverlay.html > >> (triggered by 's' key, in eVolve and eqPly; one of your nodes > >> should be > >> described as appNode in config file, see "2-node.DB.eqc" example) > >> > >> 2) try "latancy 2" on 3 in your config{ } file > >> http://www.equalizergraphics.com/documents/design/fileFormat.html > >> > >> 3) try to render very small portion of your data "range [ 0 . > >> 001 ]", or > >> something like that, to check equalizer's overhead for your setup. > >> Rendering should be very-very fast, the only thing you will see on > >> statics is equalizer's communications. > >> > >> 4) check your network performance with "netperf" tool. > >> > >> > >> Best regards, > >> > >> Makhinya Maxim > >> > >> > >> On Feb 17, 2009, at 2:31 AM, Jinghua Ge wrote: > >> > >>> Dear Maxim, > >>> > >>> I did some tests today. Since two of my cluster nodes are not > >>> working properly, I just test with single node, 2 nodes, and 4 > >>> nodes. Turns out I was wrong about 4 nodes performance before, in > >>> retrospect, I think I set the window size to be small when I did the > >>> 4nodes test. Anyway, the result I got today is: > >>> > >>> single node: 16 fps > >>> 2nodes: 10 fps > >>> 4nodes: 6 fps > >>> > >>> I also found that in my test, DS compound doesn't improve overall > >>> performance. > >>> > >>> I tried to remove the compound by commenting out all the inputframe, > >>> outframe lines in my config file. (I did remove the whole compound > >>> at first, but found out I must set the range info for each node, > >>> otherwise the data weren't distributed. ) > >>> > >>> The compound looks like this: > >>> > >>> compound > >>> { > >>> channel "channel0" > >>> buffer [ COLOR DEPTH ] > >>> > >>> wall > >>> { > >>> bottom_left [ -.5 -.5 -.75 ] > >>> bottom_right [ .5 -.5 -.75 ] > >>> top_left [ -.5 .5 -.75 ] > >>> } > >>> > >>> compound > >>> { > >>> range [ 0 .25 ] > >>> } > >>> compound > >>> { > >>> channel "channel1" > >>> range [ .25 .5 ] > >>> #outputframe {} > >>> } > >>> compound > >>> { > >>> channel "channel2" > >>> range [ .5 .75 ] > >>> #outputframe {} > >>> } > >>> compound > >>> { > >>> channel "channel3" > >>> range [ .75 1 ] > >>> #outputframe {} > >>> } > >>> #inputframe { name "frame.channel1" } > >>> #inputframe { name "frame.channel2" } > >>> #inputframe { name "frame.channel3" } > >>> } > >>> > >>> The result is about 7fps. I believe the way I did the compound > >>> there are still network transfers going around, just no final > >>> compositing. But I am not sure how to disable all of the network > >>> traffic by editing the config file. Please give more advice here. > >>> Thanks!! > >>> > >>> JInghua > >>> > >>> > >>> > >>> On Mon, Feb 16, 2009 at 10:22 AM, Jinghua Ge <[email protected]> > >>> wrote: > >>> Hi Maxim, > >>> > >>> These tests you suggested in your email really make a lot sense. I > >>> will try them today and hopefully find the bottleneck. Thanks so > >>> much! > >>> > >>> Jinghua > >>> > >>> > >>> On Mon, Feb 16, 2009 at 9:58 AM, Maxim Makhinya > >>> <[email protected]> wrote: > >>> > >>> Hello Jinghua, > >>> > >>> > >>> That sound weird. Are you sure the problem is not with one of your > >>> machines? > >>> I think you should figure out first where is your performance > >>> bottleneck, and > >>> why this happens. You could try following and write back what you > >>> will > >>> get: > >>> > >>> 1) remove all compositing paths, i.e. leave only rendering. As all > >>> nodes will > >>> render the same amount of data without compositing it should not > >>> really > >>> matter how many you use - 4 nodes for 1 Gb or 8 nodes for 2 Gb. > >>> Speed > >>> should remain roughly the same as there is no pictures > >>> transferred. > >>> > >>> 2) try to split config in to two independent parts - i.e. 4 nodes > >>> for > >>> first > >>> 1 Gb of data, another 4 nodes for second Gb, without final > >>> compositing of > >>> this two parts. Again, it should be symmetric and speed shouldn't > >>> change > >>> much. > >>> > >>> 3) if in the second path you will get your 10-12 fps, than just > >>> try to > >>> combine > >>> those results in one additional compositing on top. > >>> > >>> > >>> Best regards, > >>> > >>> Makhinya Maxim > >>> > >>> > >>> On Feb 16, 2009, at 4:01 PM, jinghua wrote: > >>> > >>>> > >>>> Dear Stefan, > >>>> > >>>> I have tried Equalizer's eVolve volume renderer to render a > >>>> 1kx1kx2k, ubyte > >>>> volume over a remote 8-node cluster. I have changed the evolve code > >>>> and > >>>> shader to read in the original volume with one byte per voxel, and > >>>> it all > >>>> worked fine. My cluster has a Nvidia GeForce-9500 card with 1G > >>>> memory on > >>>> each node, and infiband private network among the nodes. I used > >>>> direct send > >>>> compound. Each node get 1kx1kx256 subvolume. Each node renders the > >>>> 256M > >>>> volume locally at 16-20fps. When I used 4nodes to render the 1G > >>>> volume, the > >>>> overall performance is about 10-12 fps. But when I used 8nodes to > >>>> render the > >>>> whole 2G volume, the frame rate drops down to 2fps. I have tried to > >>>> use both > >>>> DB and direct send compound, with ethernet and IB network, it's all > >>>> very > >>>> consistent 2fps performance. Are there something I can do to > >>>> improve > >>>> the > >>>> performance? Thanks a lot! > >>>> > >>>> Jinghua http://n2.nabble.com/file/n2335264/test-8node.res.infi > >>>> test-8node.res.infi > >>>> > >>>> Attached is my config file. > >>>> -- > >>>> View this message in context: > >>> > http://n2.nabble.com/eVolve-render-2G-volume-on-8nodes-at-2fps-tp2335264p2335264.html > >>>> Sent from the Equalizer - Parallel Rendering mailing list archive > >>>> at > >>>> Nabble.com. > >>>> > >>>> > >>>> _______________________________________________ > >>>> eq-dev mailing list > >>>> [email protected] > >>>> http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev > >>>> http://www.equalizergraphics.com > >>>> > >>> > >>> > >>> _______________________________________________ > >>> eq-dev mailing list > >>> [email protected] > >>> http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev > >>> http://www.equalizergraphics.com > >>> > >>> > >>> > >>> -- > >>> Jinghua Ge, Ph.D > >>> Visualization Consultant, CCT > >>> 331 Frey Computing Services Center > >>> Louisiana State University > >>> Phone: (225) 578-7789 > >>> Fax: (225) 334-2061 > >>> > >>> > >>> > >>> > >>> > >>> -- > >>> Jinghua Ge, Ph.D > >>> Visualization Consultant, CCT > >>> 331 Frey Computing Services Center > >>> Louisiana State University > >>> Phone: (225) 578-7789 > >>> Fax: (225) 334-2061 > >>> > >>> > >>> _______________________________________________ > >>> eq-dev mailing list > >>> [email protected] > >>> http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev > >>> http://www.equalizergraphics.com > >> > >> > >> _______________________________________________ > >> eq-dev mailing list > >> [email protected] > >> http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev > >> http://www.equalizergraphics.com > >> > >> > > > > -- > > View this message in context: > http://n2.nabble.com/eVolve-render-2G-volume-on-8nodes-at-2fps-tp2335264p2347050.html > > Sent from the Equalizer - Parallel Rendering mailing list archive at > > Nabble.com. > > > > > > _______________________________________________ > > eq-dev mailing list > > [email protected] > > http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev > > http://www.equalizergraphics.com > > > > > _______________________________________________ > eq-dev mailing list > [email protected] > http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev > http://www.equalizergraphics.com > -- Jinghua Ge, Ph.D Visualization Consultant, CCT 331 Frey Computing Services Center Louisiana State University Phone: (225) 578-7789 Fax: (225) 334-2061
_______________________________________________ eq-dev mailing list [email protected] http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev http://www.equalizergraphics.com
