Re: RFE: use patchwork to submit a patch
Hi Ted, On Fri, Nov 08, 2019 at 09:02:49AM -0500, Theodore Y. Ts'o wrote: > On Fri, Nov 08, 2019 at 10:44:25AM +0100, Dmitry Vyukov wrote: > > syzbot asks to provide fixing commit in a particular format (and on 1 > > line, otherwise not possible to parse back). > > > > It sent this yesterday: > > https://groups.google.com/forum/message/raw?msg=syzkaller-bugs/IZZUu-BobtI/xAkxm6H4EQAJ > > This is Shuah's reply, not the syzbot original message. Looks like > you cut and pasted the same URL twice? > > The correct URL is: > > https://groups.google.com/forum/message/raw?msg=syzkaller-bugs/IZZUu-BobtI/wQdqUSHuEQAJ > > .. and it looks like what Syzbot sent was already line-wrapped: > > > If the result looks correct, please mark the bug fixed by replying with: > > > >#syz fix: usb: usbip: Fix BUG: KASAN: slab-out-of-bounds in > >vhci_hub_control() > > So I don't think you can blame this on how Shuah replied, or how > Mozilla handled things. It looks like either Syzbot or more likely, > GMail on its outgoing path "added value" by line-wrapping the message, > probably because of the following MIME setting: > > > Content-Type: text/plain; charset="UTF-8"; format=flowed; delsp=yes > > Flowing text lines and deleting spaces is always, ALWAYS, a bad idea > when sending plain text. In this case, it wasn't just a patch that > got corrupted, but the instructions from Syzbot, which it appears > Shuah followed faithfully. :-) > > IBM handled this problem by standing up an open source e-mail system > because Lotus Notes couldn't be fixed. If we can't fix GMail, then > David Miller may be right, and e-mail may be doomed. (I don't like > that answer myself, since so far all of the alternatives seem to be > categorically worse for our use case, but...) Since when did we decide to let Google dictate what tools were are or are not allowed to use ? :-) If gmail isn't suitable as an e-mail provider, we should be vocal about it. -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: RFE: use patchwork to submit a patch
Hi Steven, On Tue, Oct 15, 2019 at 12:47:49PM -0400, Steven Rostedt wrote: > On Tue, 15 Oct 2019 19:37:04 +0300 Laurent Pinchart wrote: > > > But how far could we push this reasoning ? I've worked for a company > > whose corporate policy was that all source code had to be stored in SVN, > > not exception. I didn't reach the community to move kernel development > > away from git. I've also worked with people whose corporate policy was > > that they had to do Linux kernel development on Windows, without using > > any virtual machine. There are all kind of crazy corporate policies, and > > if we don't push the pain it inflicts on all of us back to the people > > who enact those crazy policies, we'll always lose. > > I understand your sentiment. But most of your examples are one-off > "crazy corporate policies". The "sucky email server" and "you must use > this sucky email server" is rather standard. Not saying that we don't > want to pressure them to change, but I'm hearing from people (like Dave > Miller), that new developers are moving away from email for > development. I think that's two different issues though, which certainly overlap (but I'm not sure to what extent). I'm also not sure if we can say that new developers are moving away from e-mail, or simply start development with a different, non e-mail-based process. It makes a bit of a difference, because some new developers may not have any issue with e-mail, they may just not have considered it. This of course doesn't mean that nobody has issues with e-mail, that nobody is moving away from e-mail, or that nobody would have trouble moving to e-mail. > I thought part of this thread was talking about having other ways > besides email to submit a patch. Something that can help people > correctly send a patch (at least their formatting) by making sure they > fill out the proper fields and have a tool that helps them do so. I > brought up the corporate email servers just as an example of having > this help out there too. I'm very biased here, but for me a CLI tool that asks me questions and generates the patch series (or directly sends it) would be easier to use than a web UI. That's because I would stay on the CLI that I'm already using for git. This of course assumes a working git-send-email configuration, which we all no is not a given. I thus wonder if it wouldn't be better to have a CLI helper to create the patches, and provide some sort of https-to-smtp service (possibly in the form of pushing a tag with a message for instance). There are many options possible to work around broken e-mail workflows, so instead of focussing on a web UI because it was the first idea that was proposed, let's make sure we consider other ones. And maybe we'll decide that a web UI is the best option. > I plan on continuing to develop mostly in email (I still send my patch > series via quilt!). But I'm not going to enforce everyone to continue > to use email if we can come up with a better way. I also want to make > sure that whatever we do come up with will still support email. Hypothetically speaking, if there was a service that allowed sending a patch series through a git push with a tag containing a cover letter in its message, and that service would send out e-mails to mailing lists (with the option to self-host it if you want), would you consider using it ? -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: RFE: use patchwork to submit a patch
Hi Drew,
On Tue, Oct 15, 2019 at 12:34:57PM -0400, Drew DeVault wrote:
> On Tue Oct 15, 2019 at 7:32 PM Laurent Pinchart wrote:
> > I certainly wouldn't recommend a solution based on a proprietary
> > closed-source stack :-) But as we're talking about performing new
> > development for patchwork, I wanted to point out that we could also
> > consider a different technical approach that would involve new
> > development for a different open-source project. For instance, is the
> > above idea something that could be developed on top of gitolite ? Or
> > possibly even as a tiny standalone git server ?
>
> Some of the work I'm doing with code review on sourcehut can be
> generalized, to support writing up arbitrary mailing lists with
> arbitrary review tools with a bidirectional workflow. I intend to make a
> reality of this for at least lists.sr.ht <-> gitlab and lists.sr.ht <->
> github, so that patchsets and the resulting discussion gets turned into
> git{hub,lab} code reviews, and vise versa. I think it's possible to make
> a system which presents both sides with an experience that is idiomatic
> for each context, so that you can't really tell whose using which tool
> because they're both speaking each other's conventional language.
First of all, please let me thank you for all the effort you're putting
into this.
I'm not very confident that perfect transparent e-mail bridges could be
developed, and the culprit here it e-mail. From forge to e-mail there's
no real issue, we have structure data, and we write it out in text form.
The other way around, though, involves recovering structure from text.
If the MTAs, MDAs, MUAs and, quite importanttly, users, behave
correctly, that's doable. We can handle some of the "features" of common
M*A, but if someone decides to throw the formatting through the window
completely, we'll be screwed.
An idea I was toying with in the past was to create a structured format
for review, that would match what we use now in e-mail very closely, but
with a clearly defined syntax and grammar. The format would appear as
just plain e-mails to users, and a MUA that behaves reasonably would
likely produce valid structured data as e-mail replies. Over time we
could develop clients or teach some MUAs about the structured format,
removing the risk that they, or the end users, would mess up a reply.
The migration would be mostly transparent as it wouldn't really impact
existing users of e-mail, and could thus be rolled out over a long
period of time.
--
Regards,
Laurent Pinchart
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork
Re: RFE: use patchwork to submit a patch
Hi Steven, On Tue, Oct 15, 2019 at 08:37:41AM -0400, Steven Rostedt wrote: > On Mon, 14 Oct 2019 16:56:58 -0400 "Theodore Y. Ts'o" wrote: > > > So it's an extreme case, and is an example of corp e-mail insanity. > > The good news is that for companies that are really serious about > > working with upstream, they will roll out solutions for their own > > customers. For example, with IBM, since Lotus notes was maximally > > unfriendly to OSS development practices, they created a separate mail > > system with l...@linux.ibm.com addresses be cause it was simply > > impossible to engage with upstream development using ibm.com > > addresses. So there are work arounds, and I encourage you to reach > > out to other Google kernel developers --- and if you are unhappy, > > please file internal bugs against the corp infrastructure. > > Note, one issue is that some corporations do not want to support two > email services. It's hard enough securing one, opening up another one > is just opening another door to be cracked. > > We do file bugs when there's issues, but that doesn't mean they will be > resolved. I have special permission to use my personal account, but > most people at my company do not have that permission, and this is a > heart ache for those that need to send email via the corporate email > servers. But how far could we push this reasoning ? I've worked for a company whose corporate policy was that all source code had to be stored in SVN, not exception. I didn't reach the community to move kernel development away from git. I've also worked with people whose corporate policy was that they had to do Linux kernel development on Windows, without using any virtual machine. There are all kind of crazy corporate policies, and if we don't push the pain it inflicts on all of us back to the people who enact those crazy policies, we'll always lose. > Now, if we had a way to send and receive upstream patches via a web > site, that would actually make things easier. -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: RFE: use patchwork to submit a patch
Hi Konstantin, On Tue, Oct 15, 2019 at 11:40:10AM -0400, Konstantin Ryabitsev wrote: > On Mon, Oct 14, 2019 at 06:33:33PM +0300, Laurent Pinchart wrote: > > If the goal is to work around SMTP-related technical issues, is a web UI > > really the best way to go ? Wouldn't it be better to do the same through > > a git push ? We could setup a git server that requires authentication, > > and implement a push-to-email bridge. The information that would need to > > be entered in a web UI could be put in a tag message, and we could have > > a CLI to create the tag from a list of questions. > > Well, this is largely what GitGitGadget does > (https://gitgitgadget.github.io), and we could go that route, sure. I'm > reluctant only because, quoth: > > GitGitGadget itself is a GitHub App that is backed by an Azure > Function written in pure Javascript which in turn triggers an Azure > Pipeline written in Typescript (which is really easy to understand and > write for everybody who knows even just a little Javascript), > maintained at https://github.com/gitgitgadget/gitgitgadget. > > I have zero familiarity with any of the above. That said, we do have a > bunch of CI engineers working at the LF, and I can probably avail myself > of their expertise if we decide to set this up. I certainly wouldn't recommend a solution based on a proprietary closed-source stack :-) But as we're talking about performing new development for patchwork, I wanted to point out that we could also consider a different technical approach that would involve new development for a different open-source project. For instance, is the above idea something that could be developed on top of gitolite ? Or possibly even as a tiny standalone git server ? -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: RFE: use patchwork to submit a patch
Hi Dmitry, On Tue, Oct 15, 2019 at 06:49:27AM +0200, Dmitry Vyukov wrote: > On Mon, Oct 14, 2019 at 5:12 PM Laurent Pinchart wrote: > > On Mon, Oct 14, 2019 at 04:58:17PM +0200, Dmitry Vyukov wrote: > >> On Fri, Oct 11, 2019 at 7:20 PM Shuah Khan wrote: > >>> On 10/11/19 2:57 AM, Greg KH wrote: > >>>> On Thu, Oct 10, 2019 at 10:41:50AM -0400, Konstantin Ryabitsev wrote: > >>>>> Hi, all: > >>>>> > >>>>> I would like to propose a new (large) feature to patchwork with the > >>>>> goal to > >>>>> make the process of submitting a patch easier for newbies and people > >>>>> generally less familiar with patch-based development. This was discussed > >>>>> previously on the workflows list: > >>>>> https://lore.kernel.org/workflows/20190930202451.GA14403@pure.paranoia.local/ > >>>>> > >>>>> How I envision this would work: > >>>>> > >>>>> - user creates an account (which requires a mail confirmation) >> - > >>>>> they choose a "submit patch" option from the menu > >>>>> - the patch submission screen has a succession of screens: > >>>>> > >>>>> 1. a screen with a single field allowing a user to paste a URL to > >>>>> their > >>>>> fork of the git repository. Once submitted, patchwork does a "git > >>>>> ls-remote" to attempt to get a list of refs and to verify that this > >>>>> is > >>>>> indeed a valid git repository > >>>> > >>>> s/valid git repository/valid git repository based on the kernel git tree/ > >>>> > >>>> Otherwise you might be sending out lots of emails for other projects :) > >>>> > >>>>> 2. next screen asks the user to select the ref to work from using the > >>>>> list obtained from the remote. Once submitted, patchwork performs a > >>>>> `git > >>>>> clone --reference` to clone the repository locally using a local > >>>>> fork of > >>>>> the same repo to minimize object transfer. This part requires that: > >>>>>a. patchwork project is configured with a path to a local fork, > >>>>> if this feature is enabled for a project > >>>>>b. that fork is kept current via some mechanism outside of > >>>>> patchwork (e.g. with grokmirror) > >>>>>c. there is some sanity-checking during the clone process to > >>>>> avoid abuse (e.g. a sane timeout, a tmpdir with limited size, > >>>>> etc > >>>>> -- other suggestions welcome) > >>>>> > >>>>> 3. next screen asks the user to pick a starting commit from the log. > >>>>> Once submitted, patchwork generates the patch from the commit > >>>>> provided > >>>>> to the tip of the branch selected by the user earlier, > >>>>> using git format-patch. > >>>>> > >>>>> 4. next screen asks the user to review the patch to make sure this is > >>>>> what they want to submit. Once confirmed, patchwork performs two > >>>>> admin-defined optional hooks: > >>>>> > >>>>>a. a hook to generate a list of cc's (e.g. get_maintainer.pl) > >>>>>b. a sanity check hook (e.g. checkpatch.pl) > >>>> > >>>> I will note that many "first patch" submissions are checkpatch.pl > >>>> cleanups for staging. When doing that, I require that they do "one > >>>> logical change per patch", which means that many of the individual > >>>> patches themselves will not be checkpatch.pl clean, because many lines > >>>> have multiple issues with them (tabs, spaces, format, length, etc.) > >>>> > >>>> So other than that minor thing, sounds interesting. It's hard to > >>>> determine just how difficult the whole "set up git and send a patch out" > >>>> process is for people these days given the _huge_ numbers of new > >>>> contributions we keep getting, and the numerous good tutorials we have > >>>> created that spell out exactly how to do this. > >>>> > >>>> So you might be "solving" a p
Re: RFE: use patchwork to submit a patch
On Tue, Oct 15, 2019 at 11:11:57AM +0200, Dmitry Vyukov wrote: > On Tue, Oct 15, 2019 at 10:57 AM Eric Wong wrote: > > Dmitry Vyukov wrote: > >> As one data point, I cannot send emails with git send-email anymore. > >> It used to work, then broke and I don't know how to fix it. Now it says: > >> > >> 5.7.8 Username and Password not accepted. Learn more at > >> 5.7.8 https://support.google.com/mail/?p=BadCredentials > >> s10sm8376885wrr.5 - gsmtp > >> > >> I suspect it has something to do with two factor auth. > >> So that's it: it cannot contribute to kernel right now. > >> I will not consider time spent fixing it as useful time investment. > > > > I'm sorry you feel that way about time investments... > > But I've always assumed that's also the sentiment for time spent > > learning ANY new tools or workflow changes that come along. > > This is true. But the fact that there is a learning curve to anything > does not justify any learning curve for everything. Some parts of > technology may be isolated completely and one does not need to learn > anything about that part. For example, today to compile a high-level > language one generally does not need to learn anything about machine > instructions. That's only true to some extent. Yes, you can develop in java without knowing what a CPU is, but to develop efficient and safe software, a knowledge of the whole stack is very useful, when not required. > So the question is: is SMTP/IMAP is something that > inherently needs to be learned for contribution to kernel or it can be > hidden/not required/made simpler? And looking at github/facebook I > would assume that contributors do not have to be exposed to that. Could we develop a patch submission and review based on facebook if we wanted to ? Most likely yes. Would we consider that as a good idea ? Most likely no. There are always pros and cons in any workflow, and while I could personally move away from e-mail, I would want a solution that brings me most of the benefits of e-mail, in particular decentralisation. git**b creates both a central point of failure and a central point of trust, so it's a big no-go as far as I'm concerned. On the other hand, creating a solution that enables optional use of forges for contributors would prefer using them doesn't bother me at all, quite the contrary. > >> Any kernel documentation that I can find for gmail, mentions config > >> that I am already using and that is not working: > >> https://www.kernel.org/doc/html/latest/search.html?q=gmail_keywords=yes=default# > >> https://www.kernel.org/doc/html/latest/process/email-clients.html?highlight=gmail > > > > Fwiw, git-send-email(1) manpage also has a special section for gmail: > > > > https://kernel.org/pub/software/scm/git/docs/git-send-email.html > > > > and a link for app-specific passwords: > > > > https://security.google.com/settings/security/apppasswords > > > > Perhaps that helps? > > For me that page says "The setting you are looking for is not > available for your account". > I suspect app passwords work if 2-factor auth is enabled, but what > enabled on my account is "Use your phone to sign in", which is > different from 2-factor auth setting. Did git-send-email start failing when enabling phone as a mean to sign in, or was it unrelated ? -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: RFE: use patchwork to submit a patch
Hi Konstantin, On Fri, Oct 11, 2019 at 05:35:53PM -0400, Konstantin Ryabitsev wrote: > On Fri, Oct 11, 2019 at 09:23:08PM +, Eric Wong wrote: > >> (This is the same reason I generally disagree with Eric Wong about > >> preserving SMTP as the primary transmission protocol -- I've heard lots of > >> complaints both from kernel developers and especially from people trying to > >> contribute to CAF about corporate policies actually making it impossible to > >> submit patches -- and no, using a different mail server is not a > >> possibility > >> for them because it can be a firing offense under their IT AUP rules.) > > > > I'm not opposed to a webmail interface tailored to kernel hacking > > which does stuff like checkpatch.pl and get_maintainer.pl before > > sending (similar to your patchwork proposal and > > gitgadgetgadget). That would get around security appliances > > but SMTP would still be used in the background. > > > > Or offer full-blown HTTPS webmail + IMAP + SMTP access like any > > other webmail provider + checkpatch + get_maintainer helpers. > > Well, this is the bit where I say that it may not be allowed by > corporate rules. I see this all the time in CAF/Android world where > companies *require* that all email goes through their SMTP server so > that it can be properly logged (often for legal reasons). And it is > often equally required that any code submissions come from > per...@corporate.com and not per...@free-email-provider.com for > License/CLA reasons, so setting up a webmail server is not a solution > either. > > This is basically why SMTP sucks in my view -- and it's worthless trying > to pick fights with IT departments, because they are told to do so by > lawyers. So, I want to take SMTP out of the equation: > > 1. provide a way for someone to submit a patch using a web interface >(but still in a way that From: is their corporate ID) > 2. use individual git feeds as a way to send out patches instead of >always being secondary to SMTP If the goal is to work around SMTP-related technical issues, is a web UI really the best way to go ? Wouldn't it be better to do the same through a git push ? We could setup a git server that requires authentication, and implement a push-to-email bridge. The information that would need to be entered in a web UI could be put in a tag message, and we could have a CLI to create the tag from a list of questions. -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: RFE: use patchwork to submit a patch
I don't think this a problem that is worth solving. > > > When a new developer wants to send a patch, they don't need to create > > > any accounts. They setup their email client and send patch. > > > > > > We have several resources that walk them through setting up email > > > clients and sending patches. checkpatch.pl can be automated with > > > git hooks. > > > > > > >> I know this is a pretty big RFE, and I would like to hear your thoughts > > > >> about this. If there is general agreement that this is doable/good > > > >> idea, I > > > >> may be able to come up with funding for this development as part of the > > > >> overall tooling improvement proposal. > > > > > > > > The workflow seems sane, and matches what most people do today, with the > > > > exception that it "solves" the git send-email issue, right? Is that our > > > > biggest barrier? > > > > > > > > I would recommend interviewing some of the recent kernel mentor project > > > > and outreachy applicants first, to try to determine exactly what their > > > > problems, if any, were with our development process. If they say that > > > > this type of tool/workflow would have saved them hours of time and > > > > energy, then that's a great indication that we should try to do this. > > > > > > > > > > I would say considering the number of applicants to mentorship program > > > and new developers it will be lot overhead to require them to create > > > patchwork accounts, and it might even be hard overtime. A lot of them > > > start out and drop out in the middle. With the current setup, nothing > > > to cleanup. > > > > > > Setting up email clients and git hooks is one time task. It is the > > > easiest of the learning curve for many new developers. New developers > > > struggle with getting the change logs right, coding styles right, and > > > responding to review comments and acting on them. > > > > > > These aren't something that can be automated and they just have to > > > learn through experience of sending patches. > > > > > > My opinion based on contact with new developers as well running the > > > mentorship program, I would sat this isn't something that needs > > > solving. > > > > As one data point, I cannot send emails with git send-email anymore. > > It used to work, then broke and I don't know how to fix it. Now it says: > > > > 5.7.8 Username and Password not accepted. Learn more at > > 5.7.8 https://support.google.com/mail/?p=BadCredentials > > s10sm8376885wrr.5 - gsmtp > > > > I suspect it has something to do with two factor auth. > > So that's it: it cannot contribute to kernel right now. > > That is because your employer changed how it manages imap. So yes, this > configuration is now broken, you can not contribute to the kernel this > way. They know about it, and there's an "opt-out" list you can sign up > for if you want to fix it. Nothing the community can do about something > crazy like this. > > > As another data point, I spoke to KP Singh at the Plumbers. He is a > > "returning" kernel developer (so already did this before), he said it > > took him 3 days and 52 configurations changes (all were committed to > > git, so was possible to count exactly) to setup mail client properly. > > And he is "staffed" to do kernel work, I would expect that most people > > who don't _have_ to do kernel contributions will turn away half-way. > > > > As another data point, several people told me that they are afraid of > > sending kernel patches b/c there is so much "on you" to do right. > > > > I would say that we need to aim at a process that does not require a > > friendly experienced person to answer any of your questions in the > > common case. Lots of people will simply not ask any questions. > > Again, interview the outreachy and mentorship applicants and see what > they say about this. > > All corporate email systems do crazy things with email to help prevent > them from participating in Linux kernel development. We have known this > for decades. Is it the community's job to fix that, or is it the > individual company's job to do that for when they want to have people > participate? As I have replied separately, I don't think it's our duty to fix corporate e-mail servers, but when we know about common issues with large e-mail providers, and especially when we know of possible workarounds, I think it would be useful to centralise that information in a place that newcomers can easily find. > There's a good reason almost all Linux groups at companies have a Linux > email server in the corner from which to send out patches from. This > started 2 decades ago with IBM, and continues to this day with many many > many other companies. That's proof that if a company does want to > participate, it will do the needed work to do so. > > But that's not the group of people we are trying to help here, are we? > I can't tell, there seems to be complaints from both sides (newbies and > companies...) -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: RFE: use patchwork to submit a patch
r this development as part of the > >>> overall tooling improvement proposal. > >> > >> The workflow seems sane, and matches what most people do today, with the > >> exception that it "solves" the git send-email issue, right? Is that our > >> biggest barrier? > >> > >> I would recommend interviewing some of the recent kernel mentor project > >> and outreachy applicants first, to try to determine exactly what their > >> problems, if any, were with our development process. If they say that > >> this type of tool/workflow would have saved them hours of time and > >> energy, then that's a great indication that we should try to do this. > > > > I would say considering the number of applicants to mentorship program > > and new developers it will be lot overhead to require them to create > > patchwork accounts, and it might even be hard overtime. A lot of them > > start out and drop out in the middle. With the current setup, nothing > > to cleanup. > > > > Setting up email clients and git hooks is one time task. It is the > > easiest of the learning curve for many new developers. New developers > > struggle with getting the change logs right, coding styles right, and > > responding to review comments and acting on them. > > > > These aren't something that can be automated and they just have to > > learn through experience of sending patches. > > > > My opinion based on contact with new developers as well running the > > mentorship program, I would sat this isn't something that needs > > solving. > > As one data point, I cannot send emails with git send-email anymore. > It used to work, then broke and I don't know how to fix it. Now it says: > > 5.7.8 Username and Password not accepted. Learn more at > 5.7.8 https://support.google.com/mail/?p=BadCredentials > s10sm8376885wrr.5 - gsmtp > > I suspect it has something to do with two factor auth. > So that's it: it cannot contribute to kernel right now. > I will not consider time spent fixing it as useful time investment. Starting from an estalished working process, a change on your e-mail provider side broke your workflow. The exact same problem could happen regardless of how changes get submitted, a corporate HTTP proxy or firewall could also break HTTP-based submissions. Now, gmail being one of the largest e-mail providers, I think it's fair to consider that the kernel community should provide clear and easy to follow instructions on how to use git-send-email with gmail. In particular, with two-factor authentication being widespread, how to set it up with git-send-email should likely be described in https://www.kernel.org/doc/html/latest/process/email-clients.html. Blaming it solely on the SMTP protocol is a bit of a shortcut. > Any kernel documentation that I can find for gmail, mentions config > that I am already using and that is not working: > https://www.kernel.org/doc/html/latest/search.html?q=gmail_keywords=yes=default# > https://www.kernel.org/doc/html/latest/process/email-clients.html?highlight=gmail > > As another data point, I spoke to KP Singh at the Plumbers. He is a > "returning" kernel developer (so already did this before), he said it > took him 3 days and 52 configurations changes (all were committed to > git, so was possible to count exactly) to setup mail client properly. > And he is "staffed" to do kernel work, I would expect that most people > who don't _have_ to do kernel contributions will turn away half-way. That's very interesting information, is there any way that more details about the 52 steps could be shared ? > As another data point, several people told me that they are afraid of > sending kernel patches b/c there is so much "on you" to do right. Is that related to the submission mechanism, or to all the other things you need to get right ? > I would say that we need to aim at a process that does not require a > friendly experienced person to answer any of your questions in the > common case. Lots of people will simply not ask any questions. I fully agree with that. -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: [PATCH 06/10] parser: Use full regexps for delegation rules paths
Hi Stephen,
On Thursday 24 December 2015 14:06:58 Finucane, Stephen wrote:
> On 28 Nov 10:14, Mauro Carvalho Chehab wrote:
> > From: Laurent Pinchart <laurent.pinch...@ideasonboard.com>
> >
> > Paths are validated by trying to compile it as a regexp using a custom
> > validator.
> >
> > Signed-off-by: Laurent Pinchart <laurent.pinch...@ideasonboard.com>
> > Signed-off-by: Mauro Carvalho Chehab <mche...@osg.samsung.com>
>
> Some small nits that I can fix myself. Other than that,
>
> Acked-by: Stephen Finucane <stephen.finuc...@intel.com>
>
> > ---
> >
> > patchwork/bin/parsemail.py | 19 +--
> > patchwork/models.py| 9 -
> > 2 files changed, 25 insertions(+), 3 deletions(-)
> >
> > diff --git a/patchwork/bin/parsemail.py b/patchwork/bin/parsemail.py
> > index 4f22c7f2d6a0..5d6ddf45d264 100755
> > --- a/patchwork/bin/parsemail.py
> > +++ b/patchwork/bin/parsemail.py
> > @@ -339,18 +339,33 @@ def get_state(state_name):
> > pass
> > return get_default_initial_patch_state()
> >
> > +class Rule(object):
> > +pass
> > +
>
> nit: a defaultdict ({user: [path, path...]}) would be more semantic
> here. I can change this, unless you've any reason I shouldn't.
No reason, please go ahead. I didn't know about defaultdict, thanks for the
tip.
> > def auto_delegate(project, filenames):
> > if not filenames:
> > return None
> >
> > -rules = list(DelegationRule.objects.filter(project = project))
> > +# Precompile the path regexps
> > +rules = []
> > +for rule in DelegationRule.objects.filter(project = project):
> > +r = Rule()
> > +r.user = rule.user
> > +
> > +try:
> > +r.path = re.compile(rule.path)
> > +except:
>
> I'm going to make this capture 're.error' only, if that's OK.
No issue with that.
> > +print '%s is not a valid regular expression' % rule.path
> > +continue
> > +
> > +rules.append(r)
> >
> > patch_delegate = None
> >
> > for filename in filenames:
> > file_delegate = None
> > for rule in rules:
> > -if fnmatch(filename, rule.path):
> > +if rule.path.match(filename):
> > file_delegate = rule.user
> > break;
--
Regards,
Laurent Pinchart
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork
Re: [PATCH 00/10] Patchwork Autodelegate patches
Hi Stephen, On Thursday 24 December 2015 12:45:47 Finucane, Stephen wrote: > On 28 Nov 10:14, Mauro Carvalho Chehab wrote: > > This patch series contain the non-LinuxTV specific patches, > > mainly the auto-delegate patches written by Laurent Pinchart. > > > > Those got ported to patchwork upstream. > > > > Please notice that I'm still fighting to finish the migration > > from Django 1.4 to Django 1.7. So, this patch series is not > > fully tested. > > > > The auto-delegation patches are authored by Laurent, and were > > rebased by me to apply under the upstream branch. They're > > working for a long time at LinuxTV.org, but the rebase might > > cause some troubles on them. > > > > So, please test. > > [snip] > > OK, so I've tested this. The rebase wasn't as bad as we feared and > while there are a few issues they're nothing serious (I'll reply to > these separately). However, I do have a design question I'd like to > pose to you, Laurent and anyone else interested, before I start > merging this. > > My question is this: is there any advantage to providing a larger > "hook" system in patchwork? I ask this because this is the second > series that attempts a hook-style feature (the other being Damien's > git-send-email handler [1]). Both of these features do special actions > based on a patch's content: delegating a patch in this case and > ignoring it in the case of the git-send-email handler. I wonder if > there would be any advantage to making this more generic. For example: > > if patch.subject contains "models: " then set delegate to "Bob" >|___| |-| || > element match criteria action > > Some other examples: > > if patch.content contains "+++ hello_world.py" then set state to > "rejected" > if patch.author equals "exam...@example.org" then ... > > This would provide an immense amount of flexibility, assuming this > flexibility would be beneficial. It would also allow us to avoid > creating Django migrations for each new feature of this sort, which > will keep the code simpler and reduce the load on the sysadmins > maintaining patchwork instances (we'd only need migrations if we > didn't scope out the actions sufficiently initially and ended up > needing new actions. > > However, all this is me thinking out loud for now, and I'm busy enough > reviewing patches and trying to get something approaching series > support into upstream (plus, you know, my day job) to volunteer > implementing this. As such, if you agree with my idea but don't think > it worth the effort (or just flat out disagree) then I'm happy to > give my review comments on this with an eye to merging it shortly. If > you do agree though, maybe we should hold off merging this until we > explore these options? > > Thoughts? I generally like implementing features in a generic way, but as an occasional patchwork user I have a hard time judging how useful it would be. The trust issue that Johannes raised should also be considered here, to make sure that a malicious user wont be able to take advantage of either the criteria or the action to attack the patchwork host. -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: [PATCH 05/10] models: Add priority field to DelegationRule
Hi Stephen, On Thursday 24 December 2015 13:59:17 Finucane, Stephen wrote: > On 28 Nov 10:14, Mauro Carvalho Chehab wrote: > > From: Laurent Pinchart <laurent.pinch...@ideasonboard.com> > > > > The priority allows sorting delegation rules according to their > > priorities. Higher priority rules will be evaluated first. > > > > Signed-off-by: Laurent Pinchart <laurent.pinch...@ideasonboard.com> > > Signed-off-by: Mauro Carvalho Chehab <mche...@osg.samsung.com> > > Good idea. Are you OK for me to combine this with patch three (? the > one where this model is created)? Sure, no problem. -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
Re: [PATCH 06/10] parser: Use full regexps for delegation rules paths
Hi Johannes, On Friday 25 December 2015 16:59:50 Johannes Berg wrote: > On Thu, 2015-12-24 at 14:06 +, Finucane, Stephen wrote: > > > > > Paths are validated by trying to compile it as a regexp using a > > > custom validator. > > > > > > Signed-off-by: Laurent Pinchart <laurent.pinch...@ideasonboard.com> > > > Signed-off-by: Mauro Carvalho Chehab <mche...@osg.samsung.com> > > > > Some small nits that I can fix myself. Other than that, > > A small comment that may or may not be relevant - but there's a bunch > of things one can do with regexes, from taking a lot of CPU to taking a > lot of memory. > > What's the trust model for running regexes? I haven't found it in the > patches easily right now. If it's configured only in the config file it > should be OK, but if any kind of remote user can configure it then it > may need safeguards of some kind? > > I'm just thinking of a use case like kernel.org where you don't really > even trust the people who are the typical delegates/admins in patchwork > for a given project, since they are pretty much just random people the > admin doesn't necessarily trust much. > > (Or put another way - I'd hate for them to patch out/disable this > feature because of security concerns, since I'd want to use it on > kernel.org, but I'm not sure the admins would want me configuring > arbitrary regexes there) I agree with your concerns but haven't given them a thought to be honest. Right now only patchwork admins can changes the rules, but as you mention we might not trust them. I've used regexps for convenience, we could possibly replace them with a less dangerous type of pattern. One option I was toying with was to create rules automatically from MAINTAINERS, but I don't think that would be flexible enough. -- Regards, Laurent Pinchart ___ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
