Re: Editor extensions to help editing debian/* files?
On Sunday, 21 January 2024 08:43:05 CET Otto Kekäläinen wrote: > What editors and extensions are you using to augment your productivity > and minimize mistakes when editing debian/* files? That's not exactly an editor extension, but I use 'cme edit dpkg' to modify debian files. This tool provide a GUI with integrated help to edit most debian/* files. See https://github.com/dod38fr/config-model/wiki/Managing-Debian-packages-with-cme HTH
Re: Editor extensions to help editing debian/* files?
Hi Otto, Quoting Otto Kekäläinen (2024-01-29 07:06:51) > > > It would be nice though if the version in Debian was newer :) > > > > I probably won't update it until it is suitable for inclusion in a > > stable release. This means at minimum being relatively safe to run in > > an untrusted source tree, probably by using debvm to generate a VM > > containing the relevant tools and running the tools in the VM. > > As it happens, I have been writing this tool[1] that runs all builds > and tests inside a container, and check-all-the-things would be a > perfect match to be wrapped by the command 'debcraft validate' that > mounts sources inside a throwaway container.. ;) > > https://salsa.debian.org/otto/debcraft could you add an entry for your tool as docker solution number twelve to this table: https://wiki.debian.org/SystemBuildTools#Package_build_tools Thanks! cheers, josch signature.asc Description: signature
Re: Editor extensions to help editing debian/* files?
Hi! > > It would be nice though if the version in Debian was newer :) > > I probably won't update it until it is suitable for inclusion in a > stable release. This means at minimum being relatively safe to run in > an untrusted source tree, probably by using debvm to generate a VM > containing the relevant tools and running the tools in the VM. As it happens, I have been writing this tool[1] that runs all builds and tests inside a container, and check-all-the-things would be a perfect match to be wrapped by the command 'debcraft validate' that mounts sources inside a throwaway container.. ;) https://salsa.debian.org/otto/debcraft
Re: Editor extensions to help editing debian/* files?
On Thu, 2024-01-25 at 22:22 -0800, Otto Kekäläinen wrote: > Yeah, I remember looking into cats some years back as a place to learn > what commands exist. Similarly I also occasionally browse > https://pre-commit.com/hooks.html. Yeah, there are lots of other tools similar to cats, many of them are linked from the doc/TODO file but there are more I need to add there and probably many more that I don't know about. > The challenge with having all possible checkers is that they create a > lot of noise and managing the false positives end up being a lot of > work. Not suitable for daily use, but I will probably use cats for > "annual checkups" on stuff I maintain. One of the newer features added to cats in 2019 helps there, for some commands it only enables the highest severity level, then if there is no output it moves on to the next level, and so on. You can also disable individual checks or groups of checks as desired. Probably there are some other things we could add to improve the situation. > It would be nice though if the version in Debian was newer :) I probably won't update it until it is suitable for inclusion in a stable release. This means at minimum being relatively safe to run in an untrusted source tree, probably by using debvm to generate a VM containing the relevant tools and running the tools in the VM. > By the way, I submitted some small fixes at > https://github.com/collab-qa/check-all-the-things/pulls. Thanks, I will review them when I get time. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: Editor extensions to help editing debian/* files?
Hi! > Thats beginning to look like the history of check-all-the-things. Yeah, I remember looking into cats some years back as a place to learn what commands exist. Similarly I also occasionally browse https://pre-commit.com/hooks.html. The challenge with having all possible checkers is that they create a lot of noise and managing the false positives end up being a lot of work. Not suitable for daily use, but I will probably use cats for "annual checkups" on stuff I maintain. It would be nice though if the version in Debian was newer :) Since https://tracker.debian.org/pkg/check-all-the-things lacks a working git and homepage link, you guys should probably do a new upload to Debian. By the way, I submitted some small fixes at https://github.com/collab-qa/check-all-the-things/pulls.
Re: Editor extensions to help editing debian/* files?
On Sat, 2024-01-20 at 23:43 -0800, Otto Kekäläinen wrote: > PS. Related, these are commands I frequently run manually but don't > have any editor integration for: Thats beginning to look like the history of check-all-the-things. Initially I maintained such a list of commands on the wiki: https://wiki.debian.org/HowToPackageForDebian#Check_points_for_any_package Then later I worked on a tool to replace that to ease maintenance. Jakub Wilk was in parallel working on a second implementation. Eventually I found out and since his tool had a better design but mine had a better name ;) then we joined forces to work on his together: https://github.com/collab-qa/check-all-the-things I haven't had the motivation to work on it for some years though, but I keep adding TODO items/ideas to a local tree just in case that changes. There are of course a ton of other tools with the same aim out there, but almost all of them are aimed at web output or editor output etc, while check-all-the-things is currently aimed at command-line users, and most other tools have a more complex process to add new checkers. I cannot recommend running it on a source tree you don't trust, because it currently has no sandboxing mechanisms so the source tree could contain exploits for the QA tools that it may run. Now that debvm and bwrap etc exist, there is the potential for this to be fixed nicely. If SARIF support were ever to be implemented, there is the potential for it to produce machine-readable output and thus be useful for editors and to become the basis of Debian-wide static/dynamic analysis, perhaps as an integral part of Debusine when that happens. https://github.com/collab-qa/check-all-the-things/issues/4 -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: Editor extensions to help editing debian/* files?
> I looked into better tooling/editor support in general for Debian > languages in general. I think the industry answer is that "someone" > ought to build a "language (LSP) server" for the Debian languages, which > would enable editors with LSP support[1] to get the same basic features. True - LSPs is the common API into linters (and more) nowadays, but I was ready to settle with something simpler, as building a LSP for Debian I imagine is a lot of work.. .. > I might have a stab at writing a prototype for a LSP for Debian formats. ..however, if you start doing a minimal prototype it would be a great! Who knows if the LSP some day in the future would forward snippets to a Debian-specific LLM that would give hints on what needs to be changed to conform to the Debian policy :)
Re: Editor extensions to help editing debian/* files?
Otto Kekäläinen:
Hi!
Thanks for the tip, Niels!
It would be cool if dh_assistant had some kind of generic command like
"dh_assistant validate" which would attempt to introspect all
information silently and emit output only if it fails to parse
something.
That might be an option. The question is what more would you expect here
that is actually feasible to implement. :)
Feel free to propose concrete ideas and I will have a look at them.
Sadly, debhelper was never designed to be introspect-able, but depending
on the ask it might be doable or doable with some limitations.
Additionally it could emit a non-zero exit code on errors.
I tested your latest command and it works as you expected, though it
does not use exit code.
> # dh_assistant detect-unknown-hook-targets
The hook target override_dh_car_configure in debian/rules does not
seem to match any known commands.
# echo $?
0
Adding an exit 2 as a "linter" exit code in the next version with a
"--no-linter-exit-code" (open for better names) to have it always return
0 (which I suspect will be easier for some users).
Also if the JSON included the filename and line number of the finding
it would be handy:
{
"unknown-hook-targets": [
{
"candidates": [],
"filename": "debian/rules",
"target-name": "override_dh_car_configure"
}
]
}
I would be happy to include that. Unfortunately, I only have the
information available from:
LC_ALL=C make -Rrnpsf debian/rules debhelper-fail-me 2>/dev/null
Which seems to have the following limitations:
* filename/line is *not* included for "empty" targets
* When a line number is present, it is for the first line of the recipe
and not the target definition itself. One might be tempted to do
(lineno - 1) but it is not accurate[0].
I am open to suggestions for this that does not involve parsing the
makefile itself. I will leave ad-hoc parsing of Turing complete
languages to other programs.
I don't know how much ambition you have for expanding the scope of
dh_assistant.
The dh_assistant tool is growing organically based on requests, or
things I need, or when I notice people needing to introspect something
and I can easily provide it via dh_assistant. :)
I did not aim for it to be a full-fledged linter, though I do not mind
adding debhelper related linting where possible.
We already have Lintian which has a massive amount of
checks, including ones related to debian/rules. It is just a pity
Lintian does not support checking a single file or the debian/
directory contents without building the package.. [1]
[1] https://bugs.debian.org/262783
The infrastructure and design choices of lintian is working against
lintian here. I doubt #262783 will ever be solved as there is a major
difference between a 'clean assembled packages' and the '"mess" you call
a git checkout".
Even if we get #262783, a lot of lintian checks are written to check
.deb packages because that is the final result and you avoid a lot of
"what if d/rules decided to do X - what if it does *not* do X"
ambiguity. Which the person who does #262783 would probably also spend a
lot of effort porting binary checks in to the ("dirty") source context
where you have less context.
I looked into better tooling/editor support in general for Debian
languages in general. I think the industry answer is that "someone"
ought to build a "language (LSP) server" for the Debian languages, which
would enable editors with LSP support[1] to get the same basic features.
Among features that LSP servers can provide are:
* Configurable on save actions (such as wrap-and-sort)
* Format file on user request (wrap-and-sort, again)
* Diagnostics (linting)
* Text highlighting / inline documentation / inlay hints
* Auto-completion
The first half is basically what you are looking for and the latter half
is what I have been focusing on in my plugin, so it does seem to cover
our interests.
I might have a stab at writing a prototype for a LSP for Debian formats.
But it will not be in `dh_assistant`.
Best regards,
Niels
--
[0]: Sample makefile
"""
my-target:
# Some makefile level comment (not indented)
first line of the target definition
"""
Here the line number reported by make would be 3, but the target
definition is actually line 1 (so doing a -1 would be "off by one per
line of comment")
[1]: Which allegedly includes:
* emacs via eglot (?), which comes built-in these days
* neovim
* atom/pulsar via plugin (?)
* IDEA via open source plugin or builtin via their paid subscription
Re: Editor extensions to help editing debian/* files?
Hi!
Thanks for the tip, Niels!
It would be cool if dh_assistant had some kind of generic command like
"dh_assistant validate" which would attempt to introspect all
information silently and emit output only if it fails to parse
something. Additionally it could emit a non-zero exit code on errors.
I tested your latest command and it works as you expected, though it
does not use exit code.
# dh_assistant detect-unknown-hook-targets
The hook target override_dh_car_configure in debian/rules does not
seem to match any known commands.
# echo $?
0
Also if the JSON included the filename and line number of the finding
it would be handy:
{
"unknown-hook-targets": [
{
"candidates": [],
"filename": "debian/rules",
"target-name": "override_dh_car_configure"
}
]
}
I don't know how much ambition you have for expanding the scope of
dh_assistant. We already have Lintian which has a massive amount of
checks, including ones related to debian/rules. It is just a pity
Lintian does not support checking a single file or the debian/
directory contents without building the package.. [1]
[1] https://bugs.debian.org/262783
Re: Editor extensions to help editing debian/* files?
Niels Thykier: [...] Btw, `debhelper` has a `dh_assistant` command that can do some very basic analysis as well. Not sure any of it is useful for editor integration (especially because some of the features requires that it receives the same arguments as `dh` or/and `dh_auto_configure`). Personally, I have used `dh_assistant detect-hook-targets` to detect which overrides that `dh` would pick up (relies heavily on "Build-Depends: dh-sequence-foo" style add ons though). Admittedly, not in an editor context but if you can combine it with something that reads known make file targets, you could get a "override_typo looks like an override target but `dh` does not recognize it"-style warning out of it. Maybe I should just add that feature directly to `dh_assistant`. Then you can have one more command for your checklist! :D Best regards, Niels Ok, added this to debhelper/13.11.10: $ apt satisfy 'debhelper (>= 13.11.10), libtext-levenshtein-perl' $ dh_assistant detect-unknown-hook-targets [--output-format=json] The code can have false positives - especially if you use `dh ... --with ...`. In that case, you have to pass the same `--with ...` to `dh_assistant` to avoid the false positive. All the more reason to use declarative `Build-Depends: dh-sequence-foo`. For machine output use the JSON format (lintian/lintian-brush, etc.). The text output is subject to change. MRs for improvements welcome at https://salsa.debian.org/debian/debhelper/ End sidebar/thread hijack. Best regards, Niels
Re: Editor extensions to help editing debian/* files?
Otto Kekäläinen: Hi! What editors and extensions are you using to augment your productivity and minimize mistakes when editing debian/* files? I am aware of dpkg-dev-el for Emacs mentioned in the DD reference[1]. I am a big fan of Pulsar[2] and recently found a 'language-debian' plugin for Pulsar[3], but didn't get it to emit any errors/messages. I would be interested to learn what editors and integrations others use specifically for debian/* files. I've witnessed several old DDs stop their Debian work, and many aspiring ones give up on becoming a DD, because the Debian packaging work is so laboursome. One small thing that could ease the burden could be better editor integrations that help people write and maintain the debian/* files with less effort. - Otto [...] Hi Otto Personally, I use PyCharm/IDEA with the IDEA-debpkg plugin (the latter I wrote because there was no existing plugin) or emacs with dpkg-dev-el depending on the context. I think my use of PyCharm/IDEA started for similar reasons that you are praising Pulsar - if I need to work on another file, it would have an integration for that (like the preview pane with markdown, support for shellcheck, etc.). In most cases, PyCharm (and I presume Pulsar as well) has basic support out of the box or can hint you to a plugin based on the filename (extension based) - I do not have to hunt it down. Sadly, except for my own Debian plugin because d/control and d/changelog does not have extensions... oh well. Thanks for the command list. I have added a few of them to my todo list for my plugin. Like `wrap-and-sort` is due now that it supports comments and has better defaults out of the box. Btw, `debhelper` has a `dh_assistant` command that can do some very basic analysis as well. Not sure any of it is useful for editor integration (especially because some of the features requires that it receives the same arguments as `dh` or/and `dh_auto_configure`). Personally, I have used `dh_assistant detect-hook-targets` to detect which overrides that `dh` would pick up (relies heavily on "Build-Depends: dh-sequence-foo" style add ons though). Admittedly, not in an editor context but if you can combine it with something that reads known make file targets, you could get a "override_typo looks like an override target but `dh` does not recognize it"-style warning out of it. Maybe I should just add that feature directly to `dh_assistant`. Then you can have one more command for your checklist! :D Best regards, Niels
