The problem you describe is very specific to glfs_new(), and not to gfapi in general. I guess we can handle this in glfs_new by initializing an appropriate value into THIS (save old_THIS and restore it before returning from glfs_new). That should avoid the need for all those new macros?
Thanks On Wed, Jan 21, 2015, 23:37 Raghavendra Bhat <[email protected]> wrote: > > Hi, > > In glusterfs at the time of the process coming up, it creates 5 pthread > keys (for saving THIS, syncop, uuid buf, lkowner buf and syncop ctx). > Even gfapi does the same thing in its glfs_new function. But with User > Serviceable Snapshots (where a glusterfs process spawns multiple gfapi > instances for a snapshot) this will lead to more and more consumption of > pthread keys. In fact the old keys are lost (as same variables are used > for creating the keys) and eventually the process will run out of > pthread keys after 203 snapshots (maximum allowed number is 1024 per > process.). So to avoid it, pthread keys creation can be done only once > (using pthread_once in which, the globals_init function is called). > > But now a new problem arises. Say, from snapview-server xlator glfs_new > was called (or glfs_init etc). Now gfapi wants calls THIS for some of > its operations such properly accounting the memory within the xlator > while allocating a new structure. But when gfapi calls THIS it gets > snapview-server's pointer. Since snapview-server does not know about > gfapi's internal structure, it asserts at the time of allocating. > > For now, a patch has been sent to handle the issue by turning off > memory-accounting for snapshot daemon. > (http://review.gluster.org/#/c/9430). > > But if memory-accounting has to be turned on for snapshot daemon, then > the above problem has to be fixed. > 2 ways that can be used for fixing the issue are: > > 1) Add the datastructures that are used by gfapi to libglusterfs (and > hence their mem-types as well), so that any xlator that is calling gfapi > functions (such as snapview-server as of now) will be aware of the > memory types used by gfapi and hence will not cause problems, when > memory accounting has to be done as part of allocations and frees. > > OR > > 2) Properly manage THIS by introducing a new macro similar to STACK_WIND > (for now it can be called STACK_API_WIND). The macro will be much > simpler than STACK_WIND as it need not create new frames before handing > over the call to the next layer. Before handing over the call to gfapi > (any call, such as glfs_new or fops such as glfs_h_open), saves THIS in > a variable and calls the gfapi function given as an argument. After the > function returns it again sets THIS back the value before the gfapi > function was called. > > Ex: > > STACK_API_WIND (this, fn, ret, params....) > do { > xlator_t *old_THIS = NULL; > > old_THIS = this; > ret = fn (params); > THIS = old_THIS; > } while (0); > > a caller (as of now snapview-server xlator) would call the macro like this > STACK_API_WIND (this, glfs_h_open, glfd, fs, object, flags); > > > Please provide feedback and any suggestions or solutions to handle the > mentioned issue are welcome. > > Regards, > Raghavendra Bhat > > _______________________________________________ > Gluster-devel mailing list > [email protected] > http://www.gluster.org/mailman/listinfo/gluster-devel >
_______________________________________________ Gluster-devel mailing list [email protected] http://www.gluster.org/mailman/listinfo/gluster-devel
