Chuck: Phil Carnes believes that increasing the config file size limit is not a problem, as you probably have already experienced. The OFS team will add an item to our list of things to be done. Thanks for letting us know about the problem.
Becky On Fri, Sep 28, 2012 at 9:22 AM, Becky Ligon <[email protected]> wrote: > Chuck: > > I think that increasing the limits are the only way, right now, for the > system to handle a large config file. I have a question out to Phil Carnes > to see if this limit change effects BMI. As soon as I hear from him, I'll > let you know. I believe the 65k limit was just so we could send the config > file in one message to the client, which is needed when a client starts up. > We have ways of sending larger messages but that mechanism is not > currently being used for the config file. BTW: PVFS can handle as many > servers as you want; it is just the size of the config file that is posing > the problem. > > We will definitely work on a better way to send a large config file, and > as soon as I hear from Phil, I will let you know. > > Thanks, > Becky > > > On Wed, Sep 26, 2012 at 9:49 AM, Chuck Cranor <[email protected]> wrote: > >> hi- >> >> I'm using OrangeFS 2.8.4 to help test out a large emulab-based cluster >> that we are putting together ( http://newmexicoconsortium.org/probe ). >> >> I am trying to bring up a 512 node PVFS cluster using pvfs2-genconfig >> to generate the configuration file (non-interactively, from a script). >> The problem I am having is that pvfs2-server starts up OK, but then dies >> with an assert in getconfig_init() because of >> PVFS_REQ_LIMIT_CONFIG_FILE_BYTES: >> >> > static PINT_sm_action getconfig_init( >> > struct PINT_smcb *smcb, job_status_s *js_p) >> > { >> > struct PINT_server_op *s_op = PINT_sm_frame(smcb, >> PINT_FRAME_CURRENT); >> > struct server_configuration_s *user_opts = >> get_server_config_struct(); >> > assert(user_opts); >> > >> > /* TODO: handle this properly later */ >> >> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv >> > assert(user_opts->fs_config_buflen <= >> PVFS_REQ_LIMIT_CONFIG_FILE_BYTES); >> >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> >> I looked at the configuration file generated by pvfs2-genconfig and it >> is 68922 bytes. Looking at my startup log file I found: >> >> Warning: Generated config file: /etc/pvfs2-fs.conf >> has size: 68741, which is larger than the current PVFS request: 65536 >> Increase the value of PVFS_REQ_LIMIT_CONFIG_FILE_BYTES in >> src/proto/pvfs2-req-proto.h >> >> >> >> My questions are: >> >> 1. does anyone currently use OrangeFS with 512 or more servers? >> If so, is the only solution to edit the source code and recompile >> the world, or are there other things I should consider? for >> example, >> are there ways to make smaller config files using something other >> than >> pvfs2-genconfig? >> >> 2. pvfs2-req-proto.h says if you change >> PVFS_REQ_LIMIT_CONFIG_FILE_BYTES >> you should also change $req_limit in pvfs2-genconfig as well. >> is that all I need to change to run on 512 nodes, or are there other >> things I need to change too, before recompiling? >> >> 3. if I increase the number of nodes to 1024 or larger, are there other >> gotchas that I should be aware of with OrangeFS? What is the >> biggest >> number of nodes you can put into an OrangeFS system? >> >> >> >> chuck >> >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> > > > > -- > Becky Ligon > OrangeFS Support and Development > Omnibond Systems > Anderson, South Carolina > > > -- Becky Ligon OrangeFS Support and Development Omnibond Systems Anderson, South Carolina
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
