Re: [Gluster-devel] How can add one fop after another fop?
Yes, I know the architecture of translators, of course I dont want to create a new translator. If I understand that my idea is correct I will create one page in uder_review tab, But I am not sure, I should test. First I need to know which function firstly execute when a create operation is running? On Thu, Mar 30, 2017 at 4:10 PM, Niels de Vos <nde...@redhat.com> wrote: > On Thu, Mar 30, 2017 at 03:52:39PM +0430, Tahereh Fattahi wrote: > > Thanks for your answer. > > For example I want to save the number of files in directory as one > extended > > attribute. > > After every create operation, this number should be added and set as > > attribute, So after create I need a setxattr. > > But I dont know where this create is completed, I tested in dht > translator > > and it was false because the creation is not complete in this translator. > > Each xlator receives the create procedure (and its callback). The > "Glustr way" of doing this, could be to write your own xlator. Because > this is a functionality we would like to see too, it would be a great > opportunity to contribute such an xlator (or other approach?). > > My take on the functionality, is that the number of entries in a > directory can be represented as (struct stat)->st_size of a directory > inode. The Gluster internal representation could be stored as an xattr. > > You seem to be interested in implementing such a feature. If you write > an email to this list with your proposal, we can work with you to get > this included in one of the next versions. > > Once the approach has been settled upon, you can write a little design > document for the glusterfs-specs repository [0]. > > HTH, > Niels > > > 0. https://github.com/gluster/glusterfs-specs (use Gerrit to post docs) > > > > > > On Thu, Mar 30, 2017 at 3:44 PM, Niels de Vos <nde...@redhat.com> wrote: > > > > > On Thu, Mar 30, 2017 at 07:03:28AM +0430, Tahereh Fattahi wrote: > > > > Hi > > > > I want after create any file, do one setxattr automatically , where > and > > > how > > > > I should add this operation, and in which translator? > > > > > > We would like to add support for compound (or composite) FOPs through > > > libgfapi. This is currently not available yet, and I do not think > anyone > > > started working on it. > > > > > > Could you explain a little more about your use case for this? Concrete > > > examples will help in understanding and pointing you to potential > > > alternative solutions. > > > > > > Thanks, > > > Niels > > > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] How can add one fop after another fop?
Thank you very much, I check all of the alternative and see that xlator/mount/fuse/src/fuse-bridge fuse_create is the first On Thu, Mar 30, 2017 at 5:18 PM, Niels de Vos <nde...@redhat.com> wrote: > On Thu, Mar 30, 2017 at 04:59:02PM +0430, Tahereh Fattahi wrote: > > Yes, I know the architecture of translators, of course I dont want to > > create a new translator. > > A new xlator, or an extension to an existing one (I can't think of any > appropriate one though) would be the best solution. > > > If I understand that my idea is correct I will create one page in > > uder_review tab, But I am not sure, I should test. > > First I need to know which function firstly execute when a create > operation > > is running? > > There are create functions in the different 'master xlator' (which is > the xlator that starts the top of the xlator-graph). For gfapi access > this is the pub_glfs_creat() or pub_glfs_open() function. There are also > functions in xlators/mount/fuse/src/fuse-bridge.c and under > xlators/nfs/server/. If you place the function addition in its own > xlator, it will be used by all access methods, and there would not be a > need to make modifications everywhere. > > Niels > > > > > > On Thu, Mar 30, 2017 at 4:10 PM, Niels de Vos <nde...@redhat.com> wrote: > > > > > On Thu, Mar 30, 2017 at 03:52:39PM +0430, Tahereh Fattahi wrote: > > > > Thanks for your answer. > > > > For example I want to save the number of files in directory as one > > > extended > > > > attribute. > > > > After every create operation, this number should be added and set as > > > > attribute, So after create I need a setxattr. > > > > But I dont know where this create is completed, I tested in dht > > > translator > > > > and it was false because the creation is not complete in this > translator. > > > > > > Each xlator receives the create procedure (and its callback). The > > > "Glustr way" of doing this, could be to write your own xlator. Because > > > this is a functionality we would like to see too, it would be a great > > > opportunity to contribute such an xlator (or other approach?). > > > > > > My take on the functionality, is that the number of entries in a > > > directory can be represented as (struct stat)->st_size of a directory > > > inode. The Gluster internal representation could be stored as an xattr. > > > > > > You seem to be interested in implementing such a feature. If you write > > > an email to this list with your proposal, we can work with you to get > > > this included in one of the next versions. > > > > > > Once the approach has been settled upon, you can write a little design > > > document for the glusterfs-specs repository [0]. > > > > > > HTH, > > > Niels > > > > > > > > > 0. https://github.com/gluster/glusterfs-specs (use Gerrit to post > docs) > > > > > > > > > > > > > > On Thu, Mar 30, 2017 at 3:44 PM, Niels de Vos <nde...@redhat.com> > wrote: > > > > > > > > > On Thu, Mar 30, 2017 at 07:03:28AM +0430, Tahereh Fattahi wrote: > > > > > > Hi > > > > > > I want after create any file, do one setxattr automatically , > where > > > and > > > > > how > > > > > > I should add this operation, and in which translator? > > > > > > > > > > We would like to add support for compound (or composite) FOPs > through > > > > > libgfapi. This is currently not available yet, and I do not think > > > anyone > > > > > started working on it. > > > > > > > > > > Could you explain a little more about your use case for this? > Concrete > > > > > examples will help in understanding and pointing you to potential > > > > > alternative solutions. > > > > > > > > > > Thanks, > > > > > Niels > > > > > > > > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] How can add one fop after another fop?
Thanks for your answer. For example I want to save the number of files in directory as one extended attribute. After every create operation, this number should be added and set as attribute, So after create I need a setxattr. But I dont know where this create is completed, I tested in dht translator and it was false because the creation is not complete in this translator. On Thu, Mar 30, 2017 at 3:44 PM, Niels de Vos <nde...@redhat.com> wrote: > On Thu, Mar 30, 2017 at 07:03:28AM +0430, Tahereh Fattahi wrote: > > Hi > > I want after create any file, do one setxattr automatically , where and > how > > I should add this operation, and in which translator? > > We would like to add support for compound (or composite) FOPs through > libgfapi. This is currently not available yet, and I do not think anyone > started working on it. > > Could you explain a little more about your use case for this? Concrete > examples will help in understanding and pointing you to potential > alternative solutions. > > Thanks, > Niels > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] How can add one fop after another fop?
I think that api folder is a good place for adding this operation but I can not find the function that is execute for create. for example pub_glfs_create in glfs-fops.c does not execute because I can not see the printed log that I added in start of function. or I had thought maybe default_create is a good point (in libglusterfs/default) but I can not find the body of this function. can any one tell me which function is the input point for doing a fop? in api folder? or elsewhere? On Thu, Mar 30, 2017 at 7:03 AM, Tahereh Fattahi <t28.fatt...@gmail.com> wrote: > Hi > I want after create any file, do one setxattr automatically , where and > how I should add this operation, and in which translator? > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] How can add one fop after another fop?
Hi I want after create any file, do one setxattr automatically , where and how I should add this operation, and in which translator? ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] About inode table: client, server and inconsistency
Thank you very much. Is it possible to change something in server inode table during a fop from client? (I want to change the dht_layout of a directory when create a file in that directory, but I dont know how send the changed layout to servers) On Sat, Mar 18, 2017 at 6:36 PM, Amar Tumballi <atumb...@redhat.com> wrote: > > > On Thu, Mar 16, 2017 at 10:30 PM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> Hi >> Is it correct that each brick has one inode table for itself and each >> client has one inode table that stores anything that is stored in bricks >> inode table? >> >> For a given inode, the contents on client side and server side would be > very much different between how the volume graph is. > > >> >> Does all inode tables store in RAM all the time? >> > > Client (mainly fuse) inode table will be in memory all the time, until > kernel sends a FORGET. Brick side we have limited number of inodes in > memory. (There is an option called 'lru-limit'). > > >> >> >> When and how client's inode table update (how solve the inconsistency >> problem between clinet and brick inode table that is because of rebalance >> or other client fops) ? >> >> > All the translators are designed to handle the consistency check in their > 'lookup()' code, and they should send a response up with error saying its a > stale inode (ESTALE), upon receiving which, the client inode table > refreshes its inode, and does a fresh lookup again. This allows us to keep > the inode table in consistency. > > Hope that answers the question. > > -Amar > > > >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > > > -- > Amar Tumballi (amarts) > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] About DHT layout
Thanks a lot, but I need more details. Can you tell me exactly which function assigns the hash to directories on bricks? at the end of dht_function the dht_layout->list does not have the value of hash, so where this assignment is done? On Wed, Mar 15, 2017 at 8:11 PM, Ankit Raj <an...@redhat.com> wrote: > > > On Wed, Mar 15, 2017 at 8:43 PM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> Hi >> > > DHT creates directories on all the bricks in the volume. And, their > range varies from to 0x and and each brick is assigned a > specific subset of this range. Also, the directory creation and the >setting of layout is part of mkdir operation. > > I want to know when and where the layout of a directory is created? >> > > >> I read more and the code of dht_mkdir and the functions were called in >> this function but I could not find the primary part (setting the list field >> in dht_layout structure), can anyone help me? >> >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] About inode table: client, server and inconsistency
a university project! On Sun, Mar 19, 2017 at 11:31 PM, Nithya Balachandran <nbala...@redhat.com> wrote: > > > On 18 March 2017 at 21:42, Tahereh Fattahi <t28.fatt...@gmail.com> wrote: > >> Thank you very much. >> Is it possible to change something in server inode table during a fop >> from client? (I want to change the dht_layout of a directory when create a >> file in that directory, but I dont know how send the changed layout to >> servers) >> > > Why do you want to change the layout when files are created? > > Regards, > Nithya > >> >> On Sat, Mar 18, 2017 at 6:36 PM, Amar Tumballi <atumb...@redhat.com> >> wrote: >> >>> >>> >>> On Thu, Mar 16, 2017 at 10:30 PM, Tahereh Fattahi <t28.fatt...@gmail.com >>> > wrote: >>> >>>> Hi >>>> Is it correct that each brick has one inode table for itself and each >>>> client has one inode table that stores anything that is stored in bricks >>>> inode table? >>>> >>>> For a given inode, the contents on client side and server side would be >>> very much different between how the volume graph is. >>> >>> >>>> >>>> Does all inode tables store in RAM all the time? >>>> >>> >>> Client (mainly fuse) inode table will be in memory all the time, until >>> kernel sends a FORGET. Brick side we have limited number of inodes in >>> memory. (There is an option called 'lru-limit'). >>> >>> >>>> >>>> >>>> When and how client's inode table update (how solve the inconsistency >>>> problem between clinet and brick inode table that is because of rebalance >>>> or other client fops) ? >>>> >>>> >>> All the translators are designed to handle the consistency check in >>> their 'lookup()' code, and they should send a response up with error saying >>> its a stale inode (ESTALE), upon receiving which, the client inode table >>> refreshes its inode, and does a fresh lookup again. This allows us to keep >>> the inode table in consistency. >>> >>> Hope that answers the question. >>> >>> -Amar >>> >>> >>> >>>> ___ >>>> Gluster-devel mailing list >>>> Gluster-devel@gluster.org >>>> http://lists.gluster.org/mailman/listinfo/gluster-devel >>>> >>> >>> >>> >>> -- >>> Amar Tumballi (amarts) >>> >> >> >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] commit hash update (a field like commit hash)
Hi I declare a new variable in dht_layout->list structure similar to commit_hash, but I can not update this field in global . This update is just local and in client that do this update, servers and othe client can not see this change. For update this field I do like commit_hash functions, dht_update_commit_hash_for_layout in dht_selfheal.c file, all things is ok, setxattr is done with no error in STACK_WIND , but the change does not achieve to server! How can I solve this problem, where should I search for the problem? ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] commit hash update (a field like commit hash)
Hi Thanks for your quick answer. My first problem was that I dont do the operation on directory, I correct it. After that correction I understand another mistake that is about dist_layout. After changing the field and storing that, in continue for another operation, disk layout is not like local layout and there is anomaly in layout that selfheal_dir_layout is called, and everything reset! so I think I should change the disk_layout and everything that is work with this structure. On Thu, Apr 6, 2017 at 7:19 PM, Mohammed Rafi K C <rkavu...@redhat.com> wrote: > Like commit hash, I hope you are doing this on directories only, > nevertheless it is good to look into the brick logs and client logs. If > logs are not helping, gdb will definitely help here. > > You can share your code with us if that is possible, more people can look > into your code to debug it. or give us more detail like how are you sending > the xattrs , things like that . > > Regards > Rafi KC > > > On 04/06/2017 07:37 PM, Tahereh Fattahi wrote: > > Hi > I declare a new variable in dht_layout->list structure similar to > commit_hash, but I can not update this field in global . > This update is just local and in client that do this update, servers and > othe client can not see this change. > For update this field I do like commit_hash functions, > dht_update_commit_hash_for_layout > in dht_selfheal.c file, all things is ok, setxattr is done with no error in > STACK_WIND , but the change does not achieve to server! > How can I solve this problem, where should I search for the problem? > > > ___ > Gluster-devel mailing > listGluster-devel@gluster.orghttp://lists.gluster.org/mailman/listinfo/gluster-devel > > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] About locking subvolume in dht translator
Thank you When I get the layout from the parent of a file inode (directory) I can see all of the subvolume, so I had thought maybe this work is correct! So is there any way to do a directory operation from a file? I had test creating parent frame and parent_loc, but there was no correct functionality On Fri, Apr 21, 2017 at 11:42 AM, Amar Tumballi <atumb...@redhat.com> wrote: > > > On Fri, Apr 21, 2017 at 12:21 PM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> Hi >> I want add some code in dht_create before doing create in hashed_subvol. >> I want write something in all subvols with setxattr operation. >> My problem is that when I want get lock (write lock) on subvols, I get >> error: >> >> pending frames: >> frame : type(1) op(CREATE) >> frame : type(1) op(CREATE) >> frame : type(0) op(0) >> patchset: git://git.gluster.com/glusterfs.git >> signal received: 11 >> time of crash: >> 2017-04-21 06:38:23 >> configuration details: >> argp 1 >> backtrace 1 >> dlfcn 1 >> . >> . >> . >> But if I just get lock on one subvol, the step of getting locks do >> correctly. >> I dont know the problem! Are subvols that get from layout from a file >> parent inode fake?? (from dht_get_layout function) >> If I get these locks from a directory for example in a setxattr >> operation, there is no problem. >> The function of lock is dht_blocking_inodelk in dht_commom.c and the >> operation >> >> STACK_WIND_COOKIE (frame, dht_blocking_inodelk_cbk, >>(void *) (long) i, >>local->lock.locks[i]->xl, >>local->lock.locks[i]->xl->fops->inodelk, >>local->lock.locks[i]->domain, >>>lock.locks[i]->loc, F_SETLKW, , >> NULL); >> >> (in dht_blocking_inodelk_rec function )raise error. >> > > Mostly because its a file and inode context is not present on all the > subvols? > > > >> >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > > > -- > Amar Tumballi (amarts) > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] About locking subvolume in dht translator
Hi I want add some code in dht_create before doing create in hashed_subvol. I want write something in all subvols with setxattr operation. My problem is that when I want get lock (write lock) on subvols, I get error: pending frames: frame : type(1) op(CREATE) frame : type(1) op(CREATE) frame : type(0) op(0) patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2017-04-21 06:38:23 configuration details: argp 1 backtrace 1 dlfcn 1 . . . But if I just get lock on one subvol, the step of getting locks do correctly. I dont know the problem! Are subvols that get from layout from a file parent inode fake?? (from dht_get_layout function) If I get these locks from a directory for example in a setxattr operation, there is no problem. The function of lock is dht_blocking_inodelk in dht_commom.c and the operation STACK_WIND_COOKIE (frame, dht_blocking_inodelk_cbk, (void *) (long) i, local->lock.locks[i]->xl, local->lock.locks[i]->xl->fops->inodelk, local->lock.locks[i]->domain, >lock.locks[i]->loc, F_SETLKW, , NULL); (in dht_blocking_inodelk_rec function )raise error. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] performance test and a spacial workload
Hi I want to create a performance test with a special workload: 1. create a file in a directory 2. setxattr on the directory of the previous file I could not merge this two in glister code and could not find a framework that generate this workload for me. I read the code of smallfile (a framework for performance testing the gluster document introduced), maybe there is a way to change the code of this software to do a setxattr on directory after create a file. Which one is better to spend time? change the gluster code for merge or the smallfile code? Can anyone help me? ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
Thank you I need this information divided to bricks. On Tue, May 2, 2017 at 11:07 PM, Vijay Bellur <vbel...@redhat.com> wrote: > > > On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> Hi >> >> I want to use a file as a counter when I create a file in dht xlator. >> I mean, after creating a new file, I want open a file in the same >> directory with a special name, read that, update the counter and write >> back. >> I think for this purpose I should open in dht_create_cbk, read in >> dht_open_cbk and write in dht_readv_cbk. >> I think I should use dht_open , dht_readv and dht_writev. Maybe I could >> create inputs for these function expect frame! is it correct to use the >> frame fro dht_create function? >> >> Is this scenario correct or there is better way? >> >> > Have you tried the object count feature [1] ? > > Regards, > Vijay > > [1] http://gluster-documentations.readthedocs.io/en/latest/ > Features/quota-object-count/ > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Detail of directory divided by briks in client side
Hi Is it possible to know the file count and directory count of one directory divided by bricks in client side? Is there any feature or idea to be implemented? ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
Hi I test getfattr -d -m. -e hex ' but i get different result: getfattr: Removing leading '/' from absolute path names # file: data/brick1/gv2/c trusted.gfid=0xae8c02978686440dac3754880dde5b93 trusted.glusterfs.dht=0x0001 maybe it is better to create a new question a bout detail of directory divided by brik! On Wed, May 3, 2017 at 4:15 PM, Tahereh Fattahi <t28.fatt...@gmail.com> wrote: > Yes, it is ok > Is it possible to see this attribute in client side? In which xlator? > > On Wed, May 3, 2017 at 2:58 PM, Sanoj Unnikrishnan <sunni...@redhat.com> > wrote: > >> I am not sure, if i got your requirement correctly. >> If you need to have the file count and directory count of each brick >> individually (without aggregation), you could get that by a getxattr of >> trusted.glusterfs.quota.size on the brick. >> The size xattr value can be interpreted as follows: >> getfattr -d -m. -e hex ' >> size: first 16 bytes >> filecount: Next 16 bytes >> dircount: Last 16 bytes >> >> Regards, >> Sanoj >> >> On Wed, May 3, 2017 at 12:20 PM, Tahereh Fattahi <t28.fatt...@gmail.com> >> wrote: >> >>> Thank you >>> I need this information divided to bricks. >>> >>> On Tue, May 2, 2017 at 11:07 PM, Vijay Bellur <vbel...@redhat.com> >>> wrote: >>> >>>> >>>> >>>> On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi <t28.fatt...@gmail.com> >>>> wrote: >>>> >>>>> Hi >>>>> >>>>> I want to use a file as a counter when I create a file in dht xlator. >>>>> I mean, after creating a new file, I want open a file in the same >>>>> directory with a special name, read that, update the counter and write >>>>> back. >>>>> I think for this purpose I should open in dht_create_cbk, read in >>>>> dht_open_cbk and write in dht_readv_cbk. >>>>> I think I should use dht_open , dht_readv and dht_writev. Maybe I >>>>> could create inputs for these function expect frame! is it correct to use >>>>> the frame fro dht_create function? >>>>> >>>>> Is this scenario correct or there is better way? >>>>> >>>>> >>>> Have you tried the object count feature [1] ? >>>> >>>> Regards, >>>> Vijay >>>> >>>> [1] http://gluster-documentations.readthedocs.io/en/latest/Featu >>>> res/quota-object-count/ >>>> >>> >>> >>> ___ >>> Gluster-devel mailing list >>> Gluster-devel@gluster.org >>> http://lists.gluster.org/mailman/listinfo/gluster-devel >>> >> >> > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] How rebalance volume without changing layout?
Hi Is it any way to rebalance a volume without any change in layout? (for example the layout changed before and now just need rebalance) I test rebalance and rebalace force, they change the layout. I could not do this rebalance for each file in a directory in client side from a setxattr because conf->defrag has null value. If I change the dht_init for initialization conf->defrag, I get error. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Adding one variable to inode structure
Thank you very much But if it was like trusted.glusterfs.dht or trusted.glusterfs.qouta it was more better. On Mon, Jun 5, 2017 at 2:58 PM, Pranith Kumar Karampuri <pkara...@redhat.com > wrote: > > > On Mon, Jun 5, 2017 at 12:40 PM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> yes you are correct. >> I thought that for creating a video file i first should do a getfattr, if >> it was not set then I should do a setxattr. >> So it would be have bad performance to do a getfattr after every create, >> for video files. Am I correct? >> > > How consistent should it be? May be you can do this operation on another > thread or something in the background so that your application may not see > the delay in doing I/O? > > >> >> On Mon, Jun 5, 2017 at 11:28 AM, Pranith Kumar Karampuri < >> pkara...@redhat.com> wrote: >> >>> Oh you want applications to do getfattr to find that it is there? >>> >>> I think I understood what you are looking for now. So basically >>> applications will keep doing creation of some files/directories, whenever a >>> video file is created you want the bricks to set an extended attribute on >>> the directory, so that the application can use it. I don't think this >>> doesn't look like the job of index xlator either. >>> >>> Why do you not want to set the extended attribute from client and want >>> the fs to do it on-behalf of the application? Setting it from client has >>> benefits like even when one brick is down it will automatically heal this >>> extended attribute etc. >>> >>> On Mon, Jun 5, 2017 at 12:10 PM, Tahereh Fattahi <t28.fatt...@gmail.com> >>> wrote: >>> >>>> I want in in mount directory in client side >>>> >>>> On Mon, Jun 5, 2017 at 11:07 AM, Pranith Kumar Karampuri < >>>> pkara...@redhat.com> wrote: >>>> >>>>> >>>>> >>>>> On Mon, Jun 5, 2017 at 11:24 AM, Tahereh Fattahi < >>>>> t28.fatt...@gmail.com> wrote: >>>>> >>>>>> Thank you >>>>>> I see that this xlator is loaded in server side. >>>>>> How can I access to variable in index xlator in client side? >>>>>> is it correct? getfattr trusted.index.myvar >>>>>> >>>>> >>>>> To which xlator do you want this information? Or do you want this on >>>>> mount? >>>>> >>>>> >>>>>> >>>>>> On Mon, Jun 5, 2017 at 10:00 AM, Pranith Kumar Karampuri < >>>>>> pkara...@redhat.com> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Jun 5, 2017 at 9:55 AM, Tahereh Fattahi < >>>>>>> t28.fatt...@gmail.com> wrote: >>>>>>> >>>>>>>> Assume that one client wants to search it's directories for video >>>>>>>> files. >>>>>>>> I want use something that helps this client to search just >>>>>>>> directories that have this kind of file. >>>>>>>> like a extended attribute, but I want this attribute be set by >>>>>>>> server not by client. >>>>>>>> I dont know which xlator is suitable for this work. >>>>>>>> >>>>>>> >>>>>>> The xlator that comes close to it is: index xlator >>>>>>> xlators/features/index/src >>>>>>> >>>>>>> You may have to track extra operations like >>>>>>> create/mknod/link/rename/unlink to make sure you get the >>>>>>> functionality you want. Basically this xlator helps you in indexing a >>>>>>> behavior. We use this for indexing directories/files that need heal etc. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> On Mon, Jun 5, 2017 at 7:13 AM, Pranith Kumar Karampuri < >>>>>>>> pkara...@redhat.com> wrote: >>>>>>>> >>>>>>>>> This sounds hacky. In general anything that is specific about an >>>>>>>>> inode, we try to store it in inode-ctx. Who uses this information >>>>>>>>> about >>>>>>>>> presence of video-file and how? May be with that knowledge there
[Gluster-devel] Rebalance source code
Hi
I want to follow the process when rebalance command is done.
Which files should I read?
I see something in cli directory that sends request to another abject. But
I need to find function that do core operation of rebalance.
I expect to see something like this in one function:
function rebalance
{
check all bricks
fix_layout for all directories
start files migration
}
Where can I find the core functions of rebalance in server side?
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Adding one variable to inode structure
Hi I want to add one boolean field to inode structure for a directory. when it is 1 it means that directory has one or more video file. when it is 0 it means that there is no video file in that directory. I can add this variable to inode structure and update it in server side, in posix_create function (when there is a create request for a video file). question is about client knowledge about this variable. is it possible that client can see the amount of this variable of different servers (bricks) and OR them to one variable in cient's inode of the directory? in which xlator I should try in client side? ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Adding one variable to inode structure
I want in in mount directory in client side On Mon, Jun 5, 2017 at 11:07 AM, Pranith Kumar Karampuri < pkara...@redhat.com> wrote: > > > On Mon, Jun 5, 2017 at 11:24 AM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> Thank you >> I see that this xlator is loaded in server side. >> How can I access to variable in index xlator in client side? >> is it correct? getfattr trusted.index.myvar >> > > To which xlator do you want this information? Or do you want this on mount? > > >> >> On Mon, Jun 5, 2017 at 10:00 AM, Pranith Kumar Karampuri < >> pkara...@redhat.com> wrote: >> >>> >>> >>> On Mon, Jun 5, 2017 at 9:55 AM, Tahereh Fattahi <t28.fatt...@gmail.com> >>> wrote: >>> >>>> Assume that one client wants to search it's directories for video files. >>>> I want use something that helps this client to search just directories >>>> that have this kind of file. >>>> like a extended attribute, but I want this attribute be set by server >>>> not by client. >>>> I dont know which xlator is suitable for this work. >>>> >>> >>> The xlator that comes close to it is: index xlator >>> xlators/features/index/src >>> >>> You may have to track extra operations like >>> create/mknod/link/rename/unlink to make sure you get the functionality >>> you want. Basically this xlator helps you in indexing a behavior. We use >>> this for indexing directories/files that need heal etc. >>> >>> >>>> >>>> On Mon, Jun 5, 2017 at 7:13 AM, Pranith Kumar Karampuri < >>>> pkara...@redhat.com> wrote: >>>> >>>>> This sounds hacky. In general anything that is specific about an >>>>> inode, we try to store it in inode-ctx. Who uses this information about >>>>> presence of video-file and how? May be with that knowledge there could be >>>>> a >>>>> possibility we can improve the solution. Do let us know the complete >>>>> problem you are trying to solve. >>>>> >>>>> On Mon, Jun 5, 2017 at 4:53 AM, Tahereh Fattahi <t28.fatt...@gmail.com >>>>> > wrote: >>>>> >>>>>> Hi >>>>>> I want to add one boolean field to inode structure for a directory. >>>>>> when it is 1 it means that directory has one or more video file. >>>>>> when it is 0 it means that there is no video file in that directory. >>>>>> I can add this variable to inode structure and update it in server >>>>>> side, in posix_create function (when there is a create request for a >>>>>> video >>>>>> file). >>>>>> question is about client knowledge about this variable. is it >>>>>> possible that client can see the amount of this variable of different >>>>>> servers (bricks) and OR them to one variable in cient's inode of the >>>>>> directory? in which xlator I should try in client side? >>>>>> >>>>>> ___ >>>>>> Gluster-devel mailing list >>>>>> Gluster-devel@gluster.org >>>>>> http://lists.gluster.org/mailman/listinfo/gluster-devel >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Pranith >>>>> >>>> >>>> >>> >>> >>> -- >>> Pranith >>> >> >> > > > -- > Pranith > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Adding one variable to inode structure
Assume that one client wants to search it's directories for video files. I want use something that helps this client to search just directories that have this kind of file. like a extended attribute, but I want this attribute be set by server not by client. I dont know which xlator is suitable for this work. On Mon, Jun 5, 2017 at 7:13 AM, Pranith Kumar Karampuri <pkara...@redhat.com > wrote: > This sounds hacky. In general anything that is specific about an inode, we > try to store it in inode-ctx. Who uses this information about presence of > video-file and how? May be with that knowledge there could be a possibility > we can improve the solution. Do let us know the complete problem you are > trying to solve. > > On Mon, Jun 5, 2017 at 4:53 AM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> Hi >> I want to add one boolean field to inode structure for a directory. >> when it is 1 it means that directory has one or more video file. when it >> is 0 it means that there is no video file in that directory. >> I can add this variable to inode structure and update it in server side, >> in posix_create function (when there is a create request for a video file). >> question is about client knowledge about this variable. is it possible >> that client can see the amount of this variable of different servers >> (bricks) and OR them to one variable in cient's inode of the directory? in >> which xlator I should try in client side? >> >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > > > -- > Pranith > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Adding one variable to inode structure
Thank you I see that this xlator is loaded in server side. How can I access to variable in index xlator in client side? is it correct? getfattr trusted.index.myvar On Mon, Jun 5, 2017 at 10:00 AM, Pranith Kumar Karampuri < pkara...@redhat.com> wrote: > > > On Mon, Jun 5, 2017 at 9:55 AM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> Assume that one client wants to search it's directories for video files. >> I want use something that helps this client to search just directories >> that have this kind of file. >> like a extended attribute, but I want this attribute be set by server not >> by client. >> I dont know which xlator is suitable for this work. >> > > The xlator that comes close to it is: index xlator > xlators/features/index/src > > You may have to track extra operations like create/mknod/link/rename/unlink > to make sure you get the functionality you want. Basically this xlator > helps you in indexing a behavior. We use this for indexing > directories/files that need heal etc. > > >> >> On Mon, Jun 5, 2017 at 7:13 AM, Pranith Kumar Karampuri < >> pkara...@redhat.com> wrote: >> >>> This sounds hacky. In general anything that is specific about an inode, >>> we try to store it in inode-ctx. Who uses this information about presence >>> of video-file and how? May be with that knowledge there could be a >>> possibility we can improve the solution. Do let us know the complete >>> problem you are trying to solve. >>> >>> On Mon, Jun 5, 2017 at 4:53 AM, Tahereh Fattahi <t28.fatt...@gmail.com> >>> wrote: >>> >>>> Hi >>>> I want to add one boolean field to inode structure for a directory. >>>> when it is 1 it means that directory has one or more video file. when >>>> it is 0 it means that there is no video file in that directory. >>>> I can add this variable to inode structure and update it in server >>>> side, in posix_create function (when there is a create request for a video >>>> file). >>>> question is about client knowledge about this variable. is it possible >>>> that client can see the amount of this variable of different servers >>>> (bricks) and OR them to one variable in cient's inode of the directory? in >>>> which xlator I should try in client side? >>>> >>>> ___ >>>> Gluster-devel mailing list >>>> Gluster-devel@gluster.org >>>> http://lists.gluster.org/mailman/listinfo/gluster-devel >>>> >>> >>> >>> >>> -- >>> Pranith >>> >> >> > > > -- > Pranith > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Rebalance source code
I had seen this function before but when dht xlator is not loaded in server
side, how this function will be called?
On Wed, Jun 7, 2017 at 4:58 PM, Nithya Balachandran <nbala...@redhat.com>
wrote:
>
>
> On 7 June 2017 at 17:31, Tahereh Fattahi <t28.fatt...@gmail.com> wrote:
>
>> Hi
>> I want to follow the process when rebalance command is done.
>> Which files should I read?
>> I see something in cli directory that sends request to another abject.
>> But I need to find function that do core operation of rebalance.
>> I expect to see something like this in one function:
>> function rebalance
>> {
>> check all bricks
>> fix_layout for all directories
>> start files migration
>> }
>> Where can I find the core functions of rebalance in server side?
>>
>
> You can start with gf_defrag_start_crawl in dht-rebalance.c
>
> Regards,
> Nithya
>
>
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] About spread count, lookup and layout
Hi I have some question about spread count: 1. When I set spread count less than subvolume count, I see the hash is assigned to one directory layout for some subvolumes are zero (that is ok) , but I expect these subvolumes should not bee looked up in case lookup_everywhere for creating a file. Unfortunately I see these lookups in server profile. Am I correct? is there any way for this case to reduce the amount of lookups that is send to the servers? (lookup optimize is off and I want this) 2. I see the option for changing the spread count and fixing the layout in dht xlaor (dht_setxattr), but when I tried to set this attribute and change the layout I dont see any change!! while there is no error and crash (I call setfattr in client side because I dont see dht xlator in server graph) ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] About spread count, lookup and layout
Thanks a lot I read in document that when we want to create a file, first we should be sure that this file does not exist in the directory. So first look at the hashed subvolume and if it does not exist then broad cast this lookup to other subvolume (when lookup optimize is off) to ensure that this file was not created before. I had thought that lookup everywhere is for searching file not for layout. I want to reduce the lookup for file (not layout) when lookup optimize is off and compare it when lookup optimize is on. (of course lookup optimize on is better , I want do this) On Sun, Apr 30, 2017 at 8:12 AM, Raghavendra Gowdappa <rgowd...@redhat.com> wrote: > > > - Original Message ----- > > From: "Tahereh Fattahi" <t28.fatt...@gmail.com> > > To: "Gluster Devel" <gluster-devel@gluster.org> > > Sent: Sunday, April 30, 2017 8:03:15 AM > > Subject: [Gluster-devel] About spread count, lookup and layout > > > > Hi > > I have some question about spread count: > > > > 1. When I set spread count less than subvolume count, I see the hash is > > assigned to one directory layout for some subvolumes are zero (that is > ok) , > > but I expect these subvolumes should not bee looked up in case > > lookup_everywhere for creating a file. Unfortunately I see these lookups > in > > server profile. > > Am I correct? > > Yes. lookup-everywhere is a fall back mechanism for the scenarios where > layout can't be trusted (like in the window b/w fix layout after add/remove > brick etc and healing of the immediate children of directory as per the new > layout). Since layout itself can't be trusted, it doesn't matter whether it > has zero ranges or not. A better way of solving this problem (than > identifying 0 ranges etc) is lookup-optimize, which identifies scenarios > (predominantly add/remove brick) and decides whether to use > lookup-everywhere before saying that a file/directory is not present. > > > is there any way for this case to reduce the amount of lookups that is > send > > to the servers? (lookup optimize is off and I want this) > > Is there any reason why you want lookup-optimize to be off? > > > > > 2. I see the option for changing the spread count and fixing the layout > in > > dht xlaor (dht_setxattr), but when I tried to set this attribute and > change > > the layout I dont see any change!! while there is no error and crash (I > call > > setfattr in client side because I dont see dht xlator in server graph) > > Probably there might be bugs in that codepath. Can you please file a bug? > > regards, > Raghavendra > > > > > ___ > > Gluster-devel mailing list > > Gluster-devel@gluster.org > > http://lists.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] DHT xlator, read and write a file during creating a new file
Hi I want to use a file as a counter when I create a file in dht xlator. I mean, after creating a new file, I want open a file in the same directory with a special name, read that, update the counter and write back. I think for this purpose I should open in dht_create_cbk, read in dht_open_cbk and write in dht_readv_cbk. I think I should use dht_open , dht_readv and dht_writev. Maybe I could create inputs for these function expect frame! is it correct to use the frame fro dht_create function? Is this scenario correct or there is better way? ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
Thank you very much I had test this before. I want all the client can see this counter and can update that, so I add an attribute for directory. setxattr for directory was very in-efficient (I saw a lot of lookup request to server because of setxattr) for every create file when our workload generate a lot of file. So which one is better in your opinion? On Tue, May 2, 2017 at 5:06 PM, Amar Tumballi <atumb...@redhat.com> wrote: > > > On Tue, May 2, 2017 at 5:30 PM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> Hi >> >> I want to use a file as a counter when I create a file in dht xlator. >> I mean, after creating a new file, I want open a file in the same >> directory with a special name, read that, update the counter and write >> back. >> I think for this purpose I should open in dht_create_cbk, read in >> dht_open_cbk and write in dht_readv_cbk. >> I think I should use dht_open , dht_readv and dht_writev. Maybe I could >> create inputs for these function expect frame! is it correct to use the >> frame fro dht_create function? >> >> Is this scenario correct or there is better way? >> >> This is correct. But very in-efficient (for so many different fops for > one fop from user). See if you can keep another extended attribute itself > which you can update. That way, you can just handle the counter management > using 'xattrop' or `setxattr()' fops > > Regards, > Amar > >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > > > -- > Amar Tumballi (amarts) > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file
Yes, it is ok Is it possible to see this attribute in client side? In which xlator? On Wed, May 3, 2017 at 2:58 PM, Sanoj Unnikrishnan <sunni...@redhat.com> wrote: > I am not sure, if i got your requirement correctly. > If you need to have the file count and directory count of each brick > individually (without aggregation), you could get that by a getxattr of > trusted.glusterfs.quota.size on the brick. > The size xattr value can be interpreted as follows: > getfattr -d -m. -e hex ' > size: first 16 bytes > filecount: Next 16 bytes > dircount: Last 16 bytes > > Regards, > Sanoj > > On Wed, May 3, 2017 at 12:20 PM, Tahereh Fattahi <t28.fatt...@gmail.com> > wrote: > >> Thank you >> I need this information divided to bricks. >> >> On Tue, May 2, 2017 at 11:07 PM, Vijay Bellur <vbel...@redhat.com> wrote: >> >>> >>> >>> On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi <t28.fatt...@gmail.com> >>> wrote: >>> >>>> Hi >>>> >>>> I want to use a file as a counter when I create a file in dht xlator. >>>> I mean, after creating a new file, I want open a file in the same >>>> directory with a special name, read that, update the counter and write >>>> back. >>>> I think for this purpose I should open in dht_create_cbk, read in >>>> dht_open_cbk and write in dht_readv_cbk. >>>> I think I should use dht_open , dht_readv and dht_writev. Maybe I could >>>> create inputs for these function expect frame! is it correct to use the >>>> frame fro dht_create function? >>>> >>>> Is this scenario correct or there is better way? >>>> >>>> >>> Have you tried the object count feature [1] ? >>> >>> Regards, >>> Vijay >>> >>> [1] http://gluster-documentations.readthedocs.io/en/latest/Featu >>> res/quota-object-count/ >>> >> >> >> ___ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
