Re: Reassigning multiple bugs for shell script analysis from Lintian
On Wed, 2020-07-08 at 21:41 +0100, Samuel Henrique wrote: > > Paul Wise > > It also seems unlikely shellcheck would add a bridge between Haskell > > and Perl of the kind needed to implement custom checks. > > I don't think such a thing is needed, shellcheck already provides > multiple machine-readable output formats, which is the way IDEs > integrate with it. Would you happen to be thinking about some usecase > that is not covered by this? As mentioned in your self reply this wouldn't enable custom checks, but I noticed that morbig's machine-readable output could enable them. > I couldn't find this lintshell project, would you mind to give some > references? It's the first time I'm hearing about it. It is a subset of the CoLiS work Ralf has been working on since 2015: https://www.irif.fr/~treinen/colis/ https://github.com/colis-anr/ https://github.com/colis-anr/lintshell https://debconf19.debconf.org/talks/105-symbolic-execution-of-maintainer-scripts/ https://debconf18.debconf.org/talks/90-mining-debian-maintainer-scripts/ https://debconf16.debconf.org/talks/63/ Ralf: could you link to all the CoLiS talks/presentations you and your team have made from the CoLiS website? > To add to the general discussion, the way I envision this moving > forward is that lintian integrates with linters (by their > machine-readable outputs, just like IDEs) and calls them against the > target files, with the possibility of ignoring checks that we might > agree we don't want. I'd suggest for shellcheck to at least disable the style checks, those are just going to be a lot of noise for many maintainers. > Adding Debian specific checks would depend on a bunch of factors like: > someone contributing directly to the linter tool, upstream accepting > it, and the check per-se making sense to be upstreamed, but most > importantly; providing Debian-specific checks would be a bonus, just > by having plain shellcheck run by default on things like maintscripts > would be a win. It seems unlikely that shellcheck upstream would accept checks that are truly Debian-specific, so I would think a better design would be to add either a plugin system or a machine-readable parse tree output mode to shellcheck. Or just use morbig's existing output. PS: there are a couple of other shell linting tools listed here: https://github.com/collab-qa/check-all-the-things/blob/master/data/sh.ini And also some more are on other lists of linting tools: https://github.com/collab-qa/check-all-the-things/raw/master/doc/TODO PPS: personally I'm not sure lintian is the right place to do generalised application of static/dynamic analysis tools to packages available in Debian. For the lone developer case I mostly like the way the check-all-the-things tool works. I think a centralised service could be based on DACA or Debile, check-all-the-things, maybe other code for more complicated checks, the SARIF interchange format, donated credits on the various cloud services and donated time on hardware owned by individuals and orgs for arch-specific checks. https://wiki.debian.org/qa.debian.org https://github.com/collab-qa/check-all-the-things/issues/4 https://docs.oasis-open.org/sarif/sarif/v2.0/csprd01/sarif-v2.0-csprd01.html -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: Reassigning multiple bugs for shell script analysis from Lintian
> > Paul Wise > > It also seems unlikely shellcheck would add a bridge between Haskell > > and Perl of the kind needed to implement custom checks. > > I don't think such a thing is needed, shellcheck already provides > multiple machine-readable output formats, which is the way IDEs > integrate with it. Would you happen to be thinking about some usecase > that is not covered by this? Just noticed you were talking about custom checks, as I mentioned in the last part of my previous email, I think custom checks are an extra, so this requirement could be dropped (though I'm not the person working on this) and the default checks could be implemented. Regards, -- Samuel Henrique
Re: Reassigning multiple bugs for shell script analysis from Lintian
Hello all, sorry for the late reply, > Felix Lechner > Over the years, Lintian accumulated many requests for features better > addressed by a shell script analyzer. If there are no objections, I > plan to assign them a copy each to morbig and shellcheck. If I understood correctly, your intent is to create a wishlist bug to shellcheck, but I don't understand what would be the (active) role of shellcheck in this, could you clarify? > Paul Wise > It also seems unlikely shellcheck would add a bridge between Haskell > and Perl of the kind needed to implement custom checks. I don't think such a thing is needed, shellcheck already provides multiple machine-readable output formats, which is the way IDEs integrate with it. Would you happen to be thinking about some usecase that is not covered by this? > Ralf Treinen > what I can do is look through the bug reports blocked by #629247, > create whishlist bug reports for them on the lintshell project on gitlab I couldn't find this lintshell project, would you mind to give some references? It's the first time I'm hearing about it. To add to the general discussion, the way I envision this moving forward is that lintian integrates with linters (by their machine-readable outputs, just like IDEs) and calls them against the target files, with the possibility of ignoring checks that we might agree we don't want. Adding Debian specific checks would depend on a bunch of factors like: someone contributing directly to the linter tool, upstream accepting it, and the check per-se making sense to be upstreamed, but most importantly; providing Debian-specific checks would be a bonus, just by having plain shellcheck run by default on things like maintscripts would be a win. Regards, -- Samuel Henrique
Re: Reassigning multiple bugs for shell script analysis from Lintian
Hello again, On Mon, Jun 15, 2020 at 12:30:51PM -0700, Felix Lechner wrote: > Over the years, Lintian accumulated many requests for features better > addressed by a shell script analyzer. If there are no objections, I > plan to assign them a copy each to morbig and shellcheck. what I can do is look through the bug reports blocked by #629247, create whishlist bug reports for them on the lintshell project on gitlab if they are relevant, and mark them as forwarded on the BTS. That seems to me a reasonable compromise, even if that is strictly speaking not the meaning of "forwarded". What do you think? BTW, the lintian bug overview is a bit scary. have you considered using usertags to structure that page, for instance according to lintian categorires, as it is done for the bug overview of qa.d.o ? -Ralf.
Re: Reassigning multiple bugs for shell script analysis from Lintian
On Mon, 2020-06-15 at 12:30 -0700, Felix Lechner wrote: > Over the years, Lintian accumulated many requests for features better > addressed by a shell script analyzer. If there are no objections, I > plan to assign them a copy each to morbig and shellcheck. Some caveats that make this not as feasible as you might think: morbig is in OCaml and shellcheck is in Haskell, which means that there are fewer people available to work on these tools. It seems likely that some of the features requested are Debian-specific so shellcheck is unlikely to implement them. It also seems unlikely shellcheck would add a bridge between Haskell and Perl of the kind needed to implement custom checks. I'm not sure of the development status of morbig, does it still have funding Ralf? It seems development has stopped since last year. lintshell is just a prototype, it has very few checks. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Reassigning multiple bugs for shell script analysis from Lintian
Hi, Over the years, Lintian accumulated many requests for features better addressed by a shell script analyzer. If there are no objections, I plan to assign them a copy each to morbig and shellcheck. Many of the bugs are blocked by Bug#629247, so that's a good place to start. Lintian will only keep the master bug. It is entitled: "Please use a decent shell script parser." We look forward to enhancing our user experience with your programs. Please let us know your thoughts and make sure to copy Paul Wise. Thanks! Kind regards Felix Lechner
