Re: Tooling Integration and Developer Experience
On 1/30/23 13:53, Warner Losh wrote: On Mon, Jan 30, 2023 at 3:40 AM Kurt Jaeger wrote: Hi, On 1/30/23 02:54, Julian H. Stacey wrote: The main idea: to prevent information fragmentation andimprove discoverability, cross-referencing abilities, search, etc. With regards to improving discoverability, Phabricator's Owner tool could be a good tactical move: it allow to bind code area to peoples in order to automatically add them to reviews. If you know phabricator in more detail, is there any kind of tool to understand the activity going on ? In bugs.freebsd.org, there is the dashboard: https://bugs.freebsd.org/bugzilla/page.cgi?id=dashboard.html I think we might need something similar to help us understand the current state of the phabricator instance and the work being done. Phab allows Dashboards, but no-one had the time to configure some queries to provide relevant stats. Phab is a terrible tool for discovery. For example, how do I query all the reviews I've ticked 'OK' that are still open, by non-committers? How do I flag things as 'interesting to me'? I can tick a flag, but I can't query flags. Also, I can't get an email address for submitter either. That makes it more of a pain to land the commit. You can search flags here [1]. You can filter them by color and the object (i.e. differential revision or any other Phab thing). Flags are personal and not visible to anybody else For common use I think tags are better and are queryable in here [2]. Tags require projects, projects can be created by administrators, this is a bit counter-intuitive, but it works But there's two other issues: The FreeBSD project has had a long history of being behind, regardless of the tools we use. There's a labor shortage to process these things as well. Second, lots of people want to talk, but few want to do the work. I tried leading an effort in this area,but grew weary of the passive-aggressive comments about how I basically sucked for not having it done already (from the same people that did 0 actual work on it). I'd love to help and do the grunt work. What is important is some form of consensus that project actually needs this. I don't know how this works, the is very little visibility from the Core on these matters. [1]https://reviews.freebsd.org/flag/ [2] https://reviews.freebsd.org/differential/query/advanced/ -- Ihor Antonov
Re: Tooling Integration and Developer Experience
On 1/30/23 10:08, Kurt Jaeger wrote: Hi! ihor@antonovs.family wrote: This can be as easy as moving everything into Phabricator. There's the issue that Phabricator itself is no longer supported upstream: https://admin.phacility.com/phame/post/view/11/phacility_is_winding_down_operations/ https://github.com/phacility/phabricator I have notes from Oct. 2021 which point to a fork: https://we.phorge.it/ That fork seems to be somewhat alive. Moving our phab instance to phorge is probably no trivial task. Should be no harder than regular update. They even have a HOWTO https://we.phorge.it/w/installation_and_setup/update_from_phabricator/ From operational point of view it requires the least amount of effort, comparing to full-scale migration to any other platform. The Phorge seems to be active, so we can ask them to improve query functionality..(I am an optimist) Overall short-term plan could look like this: 1. Upgrade to Phorge 2. Setup Maniphest for bugs and tasks 3. Migrate bugs into Maniphest 4. Enable Harbormaster (Build/CI) - this requires coordination with whoever is working on pre-commit CI. This should improve things quite a bit already. Long term we can add Ponder and Phriction (Wiki and Q) and move current wiki into it too. Infra operations are hard, and I have experience with it. I'd be happy to help. -- Ihor Antonov
Tooling Integration and Developer Experience
On 1/30/23 02:54, Julian H. Stacey wrote: > Jamie Landeg-Jones wrote: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=261657 is a trivial fix >> to an admittedly trivial issue, but it's soon going to hit one year old, >> and has not had any feedback. Not even "this is rubbish. close ticket" >> >> | jamie@catwalk:~ % stat 'so good they named it twice' >> | stat: so good they named it twice: stat: No such file or directory >> >> As such, it's the oldest of my patches to be completely ignored, but then, >> most of my fixes I haven't even submitted, because, what's the point? >> I've instead spent time writing something so the patches are automatically >> aplied to my src tree, and distributed to all my servers. Forked from: 1 year src-patch anniversary. I feel Jamie's pain, this kind of experience can be very discouraging to any contributor without commit bit. All developers like quick feedback loops. Nobody wants to wait a year. I think FreeBSD project looses a lot of potential contributors due to issues of this kind. I don't believe there is any ill intent, there is no elite cast of grumpy commit-bit holders who only work on what they are interested in, ignoring the project as a whole. Far from it. But I do hope that the situation can be improved and I want to offer my view and opinion. The Problem --- I do believe that the source of all problems is lack of integration in tooling and communication. Let me elaborate: FreeBSD project has a lot of tools, but the tools are not well integrated together: - There are too many places where a patch can be posted: phabricator, github, bugzilla, mailing list. - There are too many places to have a conversation: mailing lists, phabricator reviews, bugzilla comments, github issues and PRs, forum, multiple IRC channels spanning multiple IRC servers, etc. - A posted patch is cat in the bag, there is no pre-commit CI to do some basic sanity-checking, commit-bit holders need to do a lot of work to verify the commit (run CI on it) - Tools are not integrated. There is no information flow between them, no effective cross-referencing, lookup or discover, etc. - Bugs in Bugzilla are not visible in Phabricator. - Commits in Phabricator do not resolve bugs in Bugzilla - Jenkins CI/CD and Phabricator don't know about each other. ... there are probably more examples, but this is enough to draw a few conclusions: 1. Information is fragmented and is easily lost or forgotten. 2. It takes manual human effort to update information in multiple systems. 3. Human attention (developers, contributors, etc.) to different systems is spread unequally. This leads to poor developer experience, regardless of commit-bit status. A patch posted in bugzilla went unnoticed for a year until frustrated and desperate contributor started complaining about it in the mailing list, and was committed hours later. The is also a lack of designated maintainers (I am drawing the analogy from Linux kernel) A role who's job is to integrate: collect all patches, feedback, reports about a specific area (kernel subsystem, userland tool or whatnot), and update/curate the knowledge and communication around this area. In my 15+ year career in IT I've seen multiple projects fail due to communication and integration issues. Without concentrated effort and strong leadership these problems rarely go away on their own. Proposed Solutions -- In the order of implementation: 1. Tooling integration: This can be as easy as moving everything into Phabricator. Phabricator, apart from features that we already use, has support for CI/CD, bug reports, wiki, project planning and milestones, chat, etc. Alternative platforms can be used as well: GitLab, SourceHut The main idea: to prevent information fragmentation and improve discoverability, cross-referencing abilities, search, etc. The challenge: is inertia and migration of existing information out of currently used tools. The sentiment: we don't need more tools, we need fewer tools that work better together. 2. Growing the community: Integrated tooling improves productivity and allows focusing on quickening the feedback loop: accepting/rejecting/commenting one-off contributions faster. Regular contributors will be more visible and will get commit-bit faster. With enough commit-bit holders focused maintainership practice can be started. In the end this is just my opinion, I hope it will spark some conversation. Thanks for reading this far :) -- Ihor Antonov
Re: Version of OpenSSL included in upcoming 14.0-RELEASE
On 1/28/23 07:34, Yasuhiro Kimura wrote: ... Though I'm not familiar with the incompatibility between OpenSSL 1.1.1 and 3.0, I believe it is too optimistic to regard that build of 14-CURRENT succeeds without any error just by updating /usr/src/crypto/openssl from 1.1.1 to 3.0. So it will take for a while (a few weeks?) to finish it. And it also affects build of ports. To be honest, it is rather my main concern as ports committer. I checked Bugzilla and found following PR. openssl 3 does have some breaking changes that mostly will be visible to ports. Alpine Linux switched to openssl v3 not so long ago, I will see if they posted any kind of summary.. -- Ihor Antonov
Re: local-unbound regression
I discovered that recent unbound update broke my VPN scripts, after some investigation I think I found the problem - default location of the config file was reset to upstream value. My config file is at /var/unbound/unbound.conf (as created by local-unbound-setup) but when I use local-unbound-control I see this error message: # local-unbound-control flush_stats [1673972554] unbound-control[16206:0] error: Could not open /usr/local/etc/unbound/unbound.conf: No such file or directory [1673972554] unbound-control[16206:0] fatal error: could not read config file I have not yet created bugzilla bug https://cgit.freebsd.org/src/commit/?id=1838dec31895fd4752fa8631322ab93be0705a66 /* Pathname to the Unbound configuration file */ -#define CONFIGFILE "/var/unbound/unbound.conf" +#define CONFIGFILE "/usr/local/etc/unbound/unbound.conf" It looks like it was intentional, but then my local-unbound-setup keeps creating configuration in the old destination... And it looks like a POLA violation - I can imagine lot's of users might have configs in /var/unbound -- Ihor Antonov
local-unbound regression
I discovered that recent unbound update broke my VPN scripts, after some investigation I think I found the problem - default location of the config file was reset to upstream value. My config file is at /var/unbound/unbound.conf (as created by local-unbound-setup) but when I use local-unbound-control I see this error message: # local-unbound-control flush_stats [1673972554] unbound-control[16206:0] error: Could not open /usr/local/etc/unbound/unbound.conf: No such file or directory [1673972554] unbound-control[16206:0] fatal error: could not read config file I have not yet created bugzilla bug -- Ihor Antonov
Re: netlink socket does not accept SOCK_DGRAM
On 1/15/23 12:43, Alexander V. Chernikov wrote: The snl(3) manpage itself includes some examples, including the complete working program in the end man snl No manual entry for snl Looks like not connected to the buildworld.. -- Ihor Antonov
Re: netlink socket does not accept SOCK_DGRAM
It’s a bug. The manage should state SOCK_RAW, but both options should be supported, which is not the case ATM. I’ll fix it in a couple of days. Meanwhile it may be worth looking into snl(3) which abstracts issues like this one. Thanks, I will take a look. Is there an example somewhere of snl usage? (my C skills are not very strong)
netlink socket does not accept SOCK_DGRAM
man 4 rtnetlink says:
int socket(AF_NETLINK, SOCK_DGRAM, NETLINK_ROUTE);
The following snippet fails
int fd = socket(AF_NETLINK, SOCK_DGRAM, NETLINK_ROUTE);
if (fd < 0) {
perror("Failed to open netlink socket");
return -1;
}
printf("all good\n");
close(fd);
return 0;
I get: Failed to open netlink socket: Protocol wrong type for socket
but if I change
int fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE);
I get "all good"
Am I doing something wrong or is this a bug?
$ uname -a FreeBSD zen.hq 14.0-CURRENT FreeBSD 14.0-CURRENT #0
main-n259967-11b5b9e8a520: Sat Jan 7 16:39:30 UTC 2023
r...@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
$ kldstat | grep netl
391 0x839fc00014af8 netlink.ko
--
Ihor Antonov
mention AF_NETLINK in man 2 socket
Looks like there are no references to {PF|AF}_NETLINK in the socket manpage.
--
Ihor Antonov
