Re: [Gluster-devel] [Gluster-Maintainers] Proposed Protocol changes for 4.0: Need feedback.
Not sure how many of you have gone through the email! As we are done with 3.12 release, gluster project is entering into a territory where goal is 4.0 but branch cut dates are not yet decided. As protocol changes are one of the required feature for Gluster 4.0, I propose doing a day long hackathon for finishing most of the pointers mentioned below (and more if you have more proposed). We can co-ordinate on #gluster-dev, and try to answer all the questions. I am happy to jump on a BJ session every 6hrs to track the progress and have more discussions if required. Plan is, after the hackathon, we have all the dependent patches for this feature out for review, and also have 'agreement' from maintainers on the approach, so they can get in together, without breaking a lot of things. I propose to do it around September 3rd week. If more than 5 people are ready for it, I will plan out execution path for it further. Regards, Amar On Wed, Aug 16, 2017 at 9:20 AM, Nithya Balachandran wrote: > > > On 11 August 2017 at 18:04, Amar Tumballi wrote: > >> Hi All, >> >> Below are the proposed protocol changes (ie, XDR changes on the wire) we >> are thinking for Gluster 4.0. >> >> >>- rchecksum/fsetattr: Add 'gfid' field on wire >> >> Basic work already done at https://review.gluster.org/#/c/3956/ . >> Considering its 5yrs old patch, I refreshed it at >> https://review.gluster.org/17656 for experimental branch, and it is all >> working fine. >> >> This patch also helps in creating new RPC program etc, so for all other >> XDR changes, we only need to handle the specific change in the patch. >> >> STATUS: GREEN >> >>- statx() support >> >> https://github.com/gluster/glusterfs/issues/273 talks more on it. We can >> consider to make XDR changes for sure even if we don't implement the fops >> in all other xlator IMO, so there won't be clients' compatibility issue. >> >> STATUS: RED (As no work has been started yet) >> >>- Changes to 'gf_flock' structure on wire >> >> More on it @ https://review.gluster.org/#/c/15698. Can be handled >> without this change by using xdata, but adding this field in the XDR will >> make it faster, and less error prone. >> >> STATUS: YELLOW >> >>- Changes in few of the 'fops' to get struct iatt in _cbk >> >> This is needed for mainly handling the cases of fail over during >> rebalance, self-heal etc. Today, because we don't have a protocol support, >> our cross architecture compatibility is broken, mainly because we send iatt >> as binary in xdata dict, which is not desirable. >> > > +1 > > >> STATUS: RED >> >> (Red as we need to hear from team on what are the changes needed, and it >> would be significant change as we may have to change the fops signature >> itself). >> >>- fadvise() >> >> As per the email thread http://lists.gluster.org/piper >> mail/gluster-devel/2017-August/053457.html, if we implement the fop, we >> would need a new XDR for it. >> >> STATUS: RED >> >> (It is red as it is still in discovery phase) >> >>- Misc >> >> We don't have any other proposal for protocol change for now, other than >> a suggestion from Jeff Darcy about taking out the common flags we use >> across the board, inside xdata, and make them as 'flags' itself in XDR, for >> better perf, and manageability. >> >> STATUS: RED >> >> (Mainly because the work is about discovery and changing the fops >> signature itself). >> >> >> This is the good time to highlight if you need any further changes in >> protocol itself, and start towards getting it implemented and tested. I >> volunteer to review all such patches, and happy to co-ordinate it on >> experimental branch before sending them as a single patch (or multiple >> dependent, granular patches) on master when we feel it is ready. >> >> >> Regards, >> >> Amar >> >> >> -- >> Amar Tumballi (amarts) >> >> ___ >> maintainers mailing list >> maintain...@gluster.org >> http://lists.gluster.org/mailman/listinfo/maintainers >> >> > -- Amar Tumballi (amarts) ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [Gluster-Maintainers] Proposed Protocol changes for 4.0: Need feedback.
On 11 August 2017 at 18:04, Amar Tumballi wrote: > Hi All, > > Below are the proposed protocol changes (ie, XDR changes on the wire) we > are thinking for Gluster 4.0. > > >- rchecksum/fsetattr: Add 'gfid' field on wire > > Basic work already done at https://review.gluster.org/#/c/3956/ . > Considering its 5yrs old patch, I refreshed it at > https://review.gluster.org/17656 for experimental branch, and it is all > working fine. > > This patch also helps in creating new RPC program etc, so for all other > XDR changes, we only need to handle the specific change in the patch. > > STATUS: GREEN > >- statx() support > > https://github.com/gluster/glusterfs/issues/273 talks more on it. We can > consider to make XDR changes for sure even if we don't implement the fops > in all other xlator IMO, so there won't be clients' compatibility issue. > > STATUS: RED (As no work has been started yet) > >- Changes to 'gf_flock' structure on wire > > More on it @ https://review.gluster.org/#/c/15698. Can be handled without > this change by using xdata, but adding this field in the XDR will make it > faster, and less error prone. > > STATUS: YELLOW > >- Changes in few of the 'fops' to get struct iatt in _cbk > > This is needed for mainly handling the cases of fail over during > rebalance, self-heal etc. Today, because we don't have a protocol support, > our cross architecture compatibility is broken, mainly because we send iatt > as binary in xdata dict, which is not desirable. > +1 > STATUS: RED > > (Red as we need to hear from team on what are the changes needed, and it > would be significant change as we may have to change the fops signature > itself). > >- fadvise() > > As per the email thread http://lists.gluster.org/ > pipermail/gluster-devel/2017-August/053457.html, if we implement the fop, > we would need a new XDR for it. > > STATUS: RED > > (It is red as it is still in discovery phase) > >- Misc > > We don't have any other proposal for protocol change for now, other than a > suggestion from Jeff Darcy about taking out the common flags we use across > the board, inside xdata, and make them as 'flags' itself in XDR, for better > perf, and manageability. > > STATUS: RED > > (Mainly because the work is about discovery and changing the fops > signature itself). > > > This is the good time to highlight if you need any further changes in > protocol itself, and start towards getting it implemented and tested. I > volunteer to review all such patches, and happy to co-ordinate it on > experimental branch before sending them as a single patch (or multiple > dependent, granular patches) on master when we feel it is ready. > > > Regards, > > Amar > > > -- > Amar Tumballi (amarts) > > ___ > maintainers mailing list > maintain...@gluster.org > http://lists.gluster.org/mailman/listinfo/maintainers > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [Gluster-Maintainers] Proposed Protocol changes for 4.0: Need feedback.
Hi Niels, I guess these are the related issues you were looking for: https://github.com/gluster/glusterfs/issues/203 https://github.com/gluster/glusterfs/issues/67 On Fri, Aug 11, 2017 at 6:41 PM, Niels de Vos wrote: > On Fri, Aug 11, 2017 at 06:04:36PM +0530, Amar Tumballi wrote: > > Hi All, > > > > Below are the proposed protocol changes (ie, XDR changes on the wire) we > > are thinking for Gluster 4.0. > > One of the things that is missing in this list, is to have dictionary > entries have a (key, type, value) format. This makes it possible to have > all the values XDR encoded/decoded correctly. When new values get > transported over the network, we can force the need for XDR encoding > functions for these types. > > I thought there was a GitHub issue for this already (or maybe a bug?), > but I fail to find it. > > Niels > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [Gluster-Maintainers] Proposed Protocol changes for 4.0: Need feedback.
On Fri, Aug 11, 2017 at 06:04:36PM +0530, Amar Tumballi wrote: > Hi All, > > Below are the proposed protocol changes (ie, XDR changes on the wire) we > are thinking for Gluster 4.0. One of the things that is missing in this list, is to have dictionary entries have a (key, type, value) format. This makes it possible to have all the values XDR encoded/decoded correctly. When new values get transported over the network, we can force the need for XDR encoding functions for these types. I thought there was a GitHub issue for this already (or maybe a bug?), but I fail to find it. Niels ___ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel
