Is there any future work that you plan to do for the first case? If you plan to work on this what will be the priority?
Thank you for your answers. -- Hamza On 11/23/05, Rob Ross <[EMAIL PROTECTED]> wrote: > You understand correctly. There are two aspects to the problem: > - defining how much data goes to each datafile > - placing the datafiles on the servers that you want > > The varstrip_dist handles the first of these two aspects. Their custom > patch handles the second. We haven't worked out a general way to place > datafiles at this time. > > Rob > > Hamza Kaya wrote: > > Hmm, then I misunderstood the situation. So "varstrip_dist" means > > you're able to define only 'variable' distribution sizes. > > "0:512K;1:3M;2:4K" means select a datanode [we don't know which one!] > > and send the first datafile - which contains 512K of the original > > file- to this node. And then select another one and send the 3M > > datafile to this node... and so on. So there is no way to set the > > physical location of a file except the patch that Prof. Ludwig > > mentioned. > > > > On 11/23/05, Rob Ross <[EMAIL PROTECTED]> wrote: > > > >>The "0" and "1" refer to the datafiles that make up the file, not the > >>servers themselves. > >> > >>Rob > >> > >>Hamza KAYA wrote: > >> > >>>Hi, > >>>We have a 4 node test cluster. One node acts as metadata server and > >>>the others acts as data servers. All of the nodes are clients. On each > >>>data node some programs will be executed and this programs will use > >>>the files stored on pvfs2. However if I ensure that the programs use > >>>the data that is stored on local disk, I will gain a great performance > >>>increase. In order to do this I need to send the data to specific > >>>physical locations [e.g. write file1 to node1, write first N MB of > >>>file2 to node2 etc.] > >>> > >>>After some tests I realized that when a node requests a file it first > >>>contacts to metadata server and than it directly contacts to the > >>>server where data is stored - metadata server involves no more- So if > >>>the data is stored on the local disk of this machine than copying a > >>>file will cost approximately the same as a local copy. > >>> > >>>I can't figure out why the system accepts "0:4M" and does not accept > >>>"1:4M"! What's the difference between these two strips? If the strip > >>>string is "N:<size>" than why does PVFS2 need to send data to all > >>>nodes 0..N-1 and then node N? Is this a design choice? > >>> > >>>Thanks > >>> > >>>-- > >>>Hamza > >>> > >>>On 11/22/05, Thomas Ludwig <[EMAIL PROTECTED]> wrote: > >>> > >>> > >>>>Hamza, > >>>> > >>>>the varstrip distribution was developed in my group. We also have a > >>>>patch that controls where the data goes. However, it's currently only > >>>>available for version 1.0.1 . Hipolito Vasquez takes care of this and > >>>>will give you further info. We urgently want to have that in place such > >>>>that we can control where the data goes. > >>>> > >>>>Let's keep in contact if you have a good application for this. Thanks. > >>>> > >>>> Thomas > >>>> > >>>> > >>>>-- > >>>>Prof. Dr. habil. Thomas Ludwig | Tel: +49-6221-54-4968 > >>>>Ruprecht-Karls-Universität Heidelberg | or: +49-6221-54-4973 > >>>>Institut für Informatik +--------------+ Fax: +49-6221-54-4404 > >>>>Im Neuenheimer Feld 348 | Office 018 | [EMAIL PROTECTED] > >>>>69120 Heidelberg, Germany | http://pvs.informatik.uni-heidelberg.de/ > >>>>_______________________________________________ > >>>>PVFS2-users mailing list > >>>>[email protected] > >>>>http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > >>>> > >>> > >>> > >>>_______________________________________________ > >>>PVFS2-users mailing list > >>>[email protected] > >>>http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > >>> > >> > > > _______________________________________________ PVFS2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
