Re: [DNG] [Dng] vdev status update: device properties and udev compatibility
On Fri, Jun 26, 2015 at 04:40:33PM +0200, Didier Kryn wrote: Le 25/06/2015 07:04, Jude Nelson a écrit : In time, it will also work the busybox tools, but I am still working on getting busybox's blkid to work here (specifically, we need the -p flag to work in order to get low-level partition table information and filesystem metadata). Contrary to the other options, which only deal with formatting the output, option -p cannot be emulated with a wrapper script. Have you any idea of how to proceed. I am only a consumer of Busybox and have never tried to dig into the source. Are you, or anybody in this list, able to submit a patch to busybox blkid? What information fields from -p are needed? Thanks, Isaac Dunham ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] [Dng] vdev status update: device properties and udev compatibility
Hi Isaac, On Mon, Jun 29, 2015 at 10:12 AM, Isaac Dunham ibid...@gmail.com wrote: On Fri, Jun 26, 2015 at 04:40:33PM +0200, Didier Kryn wrote: Le 25/06/2015 07:04, Jude Nelson a écrit : In time, it will also work the busybox tools, but I am still working on getting busybox's blkid to work here (specifically, we need the -p flag to work in order to get low-level partition table information and filesystem metadata). Contrary to the other options, which only deal with formatting the output, option -p cannot be emulated with a wrapper script. Have you any idea of how to proceed. I am only a consumer of Busybox and have never tried to dig into the source. Are you, or anybody in this list, able to submit a patch to busybox blkid? What information fields from -p are needed? For libudev compatibility, we will need: * usage (e.g. filesystem, crypto_LUKS, etc.) * filesystem type * filesystem UUID * subvolume UUID (where applicable) * partition table type (e.g. dos, GPT, etc.) * partition table UUID * partition table entry scheme * partition table entry type * partition table entry size * partition table entry offset * partition table entry UUID * partition table entry number * parent disk device Examples: # blkid -p -o export /dev/sda DEVNAME=/dev/sda PTUUID=0003da58 PTTYPE=dos # blkid -p -o export /dev/sda1 DEVNAME=/dev/sda1 UUID=67378bcc-ba07-4074-9a7e-9daef3537ad5 VERSION=1.0 TYPE=ext4 USAGE=filesystem PART_ENTRY_SCHEME=dos PART_ENTRY_UUID=0003da58-01 PART_ENTRY_TYPE=0x83 PART_ENTRY_NUMBER=1 PART_ENTRY_OFFSET=2048 PART_ENTRY_SIZE=195309568 PART_ENTRY_DISK=8:0 Thanks, Jude ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] [Dng] vdev status update: device properties and udev compatibility
On 25/06/15 17:04, Jude Nelson wrote: Hey everyone, After a longer-than-expected development cycle, I have the latest news for vdev. The TL;DR is that vdev has gained enough infrastructure to generate the information that normally gets put in /run/udev. This is important for most libudev clients, because this information gets used to detect and enumerate hardware. It's the way the X server detects input devices via udev, for example. This is still being heavily tested, and is not complete, but I am please to report that: * vdev now tracks device properties and device tags, under /dev/metadata/$DEVPATH/properties and /dev/metadata/$DEVPATH/tags/, respectively. They are added by shell library subroutines available to vdev's helpers (in subr.sh). * vdev now uses the same hardware database as udev to report human-curated hardware properties (such as human-readable vendor strings). To keep things simple, vdev comes with a script that generates a squashfs filesystem image from the *.hwdb files installed to /lib/udev/hwdb.d and /etc/udev/hwdb.d. This is our alternative to systemd's/udevd's mmap()-able on-disk hwdb trie. The filesystem image itself gets mounted to /dev/metadata/hwdb by the pre-seed script, and vdev now comes with a helper tool (hwdb.sh) that can use a combination of a modalias string, a devpath, a subsystem path, and some sysfs information to look up the device properties. * vdev now comes with a udev-compat.sh helper script that reads vdev's /dev/metadata/ directory and hardware database in order to generate /dev/metadata/udev, which is meant to be symlinked to /run/udev. This lets the udev_enumerate API detect and enumerate devices by tags and properties. It is important to emphasize that this helper is logically isolated from the rest of vdev--vdev gets along just fine without it (since all udev-isms are contained by this helper), so if you don't need /run/udev, you can safely disable this helper script. * Disk property detection has been significantly improved, and is much closer to how udev does it (including adding support for querying a disk's parent device's properties, which is necessary to handle partitions properly). Thanks to Didier for working with me on this! In time, it will also work the busybox tools, but I am still working on getting busybox's blkid to work here (specifically, we need the -p flag to work in order to get low-level partition table information and filesystem metadata). There is of course more work to be done on this, but the next big milestone (my next goal) will be to boot to X with vdevd, without having to generate an xorg.conf. I have not tried this yet, and I do not expect it to work as of the latest commit, but I believe that all the necessary infrastructure is in place to begin the painstaking process of ensuring that vdev detects enough of the host's devices' properties for commonly-used libudev clients work as they did with udev. This will confirm that we generate enough of /run/udev/ for alpha. Thanks, Jude That's awesome new Jude. Thanks to you and all who have been participating on this project. Regards, -- Daniel Reurich Centurion Computer Technology (2005) Ltd. 021 797 722 ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] [Dng] vdev status update: device properties and udev compatibility
On Thu, Jun 25, 2015 at 01:04:24AM -0400, Jude Nelson wrote: Hey everyone, After a longer-than-expected development cycle, I have the latest news for vdev. The TL;DR is that vdev has gained enough infrastructure to generate the information that normally gets put in /run/udev. This is important for most libudev clients, because this information gets used to detect and enumerate hardware. It's the way the X server detects input devices via udev, for example. This is still being heavily tested, and is not complete, but I am please to report that: Hi Jude, that's amazing news, indeed. Thank you for all your work on vdev. AdMaiora KatolaZ -- [ Enzo Nicosia aka KatolaZ --- GLUG Catania -- Freaknet Medialab ] [ me [at] katolaz.homeunix.net -- http://katolaz.homeunix.net -- ] [ GNU/Linux User:#325780/ICQ UIN: #258332181/GPG key ID 0B5F062F ] [ Fingerprint: 8E59 D6AA 445E FDB4 A153 3D5A 5F20 B3AE 0B5F 062F ] ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update (2015-05-03)
Le 14/05/2015 23:16, shraptor shraptor a écrit : am not a 100% with what toolchain means. did you compile for 32bit or 64bit? Toolchain generally means the stuff necessary to build new programs from source. It includes gcc, binutils, the system libraries, the system headers and the kernel headers. 'system', here, means the headers of the standard C library and the library itself, things that your default gcc searches automatically in /usr/include, /lib and /usr/lib unless you specify --nostdinc and --nostdlib. I compile for x86_64 on x86_64. I tried to build a cross-compiler for powerpc, without success up to now and I also failed to build a native powerpc one. My gcc-musl is sysrooted, which means it searches the standard files in its own file tree and hence can live in the same host as Debian Wheezy's stock compiler. But I have noticed that many packages systematically add -I/usr/include in their CFLAGS, which is useless if you compile with the stock gcc and which fucks up the sysrooted one by including the headers of the glibc instead of the ones of musl. Therefore this is the first thing to check. Le 15/05/2015 03:07, Laurent Bercot a écrit : the annoying thing IIRC is the Perl dependency at build-time, I have built Perl, Miniperl and Python, statically linked against Musl. I don't understand a single word of these languages, but they are prerequisite to build other packages. the problem is I've never been able to find a *native* toolchain here, so you're kinda stuck compiling for x86_64 using x86 gcc binaries. I think that's what I've done, on x86_64; it is all statically linked and, therefore, can be used right away on any Linux x86_64. And it can compile more than C and C++ :-) I can produce a gcc-4.8 with it, but this one doesn't work. Didier ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update (2015-05-03)
On Thu, May 14, 2015 at 03:33:06AM -0400, Jude Nelson wrote: I'm not a Slackware or Arch user, but please let me know if there's anything I can do to make packaging easier? I don't want to make things needlessly difficult :) I've dropped an issue on github about the new Makefiles, yeah :) signature.asc Description: Digital signature ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update (2015-05-03)
I would be interested in a static Vdev. Didier could you please give some info how this was done especially any gotcha's you found? On Thursday, May 14, 2015, Jack L. Frost f...@fleshless.org wrote: On Thu, May 14, 2015 at 03:33:06AM -0400, Jude Nelson wrote: I'm not a Slackware or Arch user, but please let me know if there's anything I can do to make packaging easier? I don't want to make things needlessly difficult :) I've dropped an issue on github about the new Makefiles, yeah :) ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update (2015-05-03)
There's no issues, but due to the variable in the way Slackware packages things, I have to create directories in the temp install, then use cp to installing things which ends up being messy. Not to complain, but the lack of a traditional configure file to set the install paths does make things more hectic, especially with some of the config files and other shell scripts. While unnecessary, yet, it does make packing vdev a bit more hectic. Slackware uses a sort of fakeroot style packager, pkgtools, that creates a temp directory for source and one for the build output. It then uses configure to create $DEST set paths for installing into the temp directory before packaging. It's not anything major needed, but it would help in the future, especially installing to /lib64 for the lib path instead of /lib which is symlinked on other 64-bit distros. Sent from my Windows Phone From: Jude Nelsonmailto:jud...@gmail.com Sent: 5/14/2015 12:33 AM To: James Powellmailto:james4...@hotmail.com Cc: dng@lists.dyne.orgmailto:dng@lists.dyne.org Subject: Re: [Dng] [dng] vdev status update (2015-05-03) Hi James, On Thu, May 14, 2015 at 12:04 AM, James Powell james4...@hotmail.com wrote: Thanks Jude. The Slackware package is delayed, but only due to the lack of a Slackware style script. I might try to ask one of the Slackware team for assistance when I resume work. However, the package is fairly much completed in terms of installation points and directory setup. Hey, that's pretty cool! First the AUR packages (from Jack Frost aka openfbt), now Slackware. I'm not a Slackware or Arch user, but please let me know if there's anything I can do to make packaging easier? I don't want to make things needlessly difficult :) -Jude Sent from my Windows Phone -- From: Jude Nelson jud...@gmail.com Sent: 5/13/2015 8:24 PM To: dng@lists.dyne.org Subject: [Dng] [dng] vdev status update (2015-05-03) Hey everyone, I have the latest news for vdev: * vdev now creates symlinks for: -- /dev/v4l/by-path -- /dev/disk/by-partuuid -- /dev/disk/by-partlabel Thank you Scooby for helping me confirm this! * vdev now sets the proper ownership and permissions for: -- /dev/mISDNtimer -- /dev/mwave -- /dev/parport[0-9] -- /dev/printer -- /dev/ppdev -- /dev/lp[0-9] -- /dev/irlpt[0-9] -- /dev/sch[0-9] -- /dev/pktcdvd[0-9] -- /dev/qft[0-9] -- /dev/zqft[0-9] -- /dev/nqft[0-9] -- /dev/nzqft[0-9] -- /dev/rawqft[0-9] -- /dev/nraqft[0-9] -- /dev/rawctl -- /dev/aoe -- /dev/lirc[0-9] -- /dev/legousbtower -- /dev/sonypi -- /dev/mmtimer -- /dev/sgi_* -- /dev/z90crypt * Didier Kryn has gotten vdevd and vdevfs to statically link and compile with musl libc. I'm looking forward to merging his contributions :) * the build system has been refactored to use Makefiles more appropriately: -- every generated file gets written to a separate build/ directory -- every generated file is created by a non-phony target (i.e. nothing happens by side-effect) -- global buildconf.mk for setting variables Thanks to Didier again for the suggestion! * the initramfs hook conditionally pulls in iproute2, lvm, dmsetup, blkid, etc. Moreover, vdev's helpers conditionally use these programs if they are present. This means that vdev doesn't strictly depend on them anymore, which is good if you're not using mapped devices or logical volumes. Thanks to Anto for the suggestion! * the initramfs hook pulls in the correct GNU libc files now, so chown(1) will work correctly in the initramfs. Thanks to Scooby, Isaac Dunham, Tom H for their help! [For Next Time] * The next immediate objective is to package vdevd for Devuan, to make it easier to test (in particular, easier to get a working initramfs). I'm currently dissecting udev's packaging and reading through Isaac's mdev scripts to see what to expect and what we can recycle. I've never packaged something this complex before. Thanks to Scooby, Anto, Tom H, and others for all their help on getting me this far! * Symlinks and permissions for RAID devices (/dev/md[0-9] and friends). I don't run a RAID myself, so I'll need to go set one up in QEMU. If someone wants to test vdev with a RAID, let me know, and I'll try to write a helper script to play around with sooner rather than later. * In some distros and configurations, a separate device manager (like mdev) can populate just enough of /dev to mount root, but not try to set up any symlinks. Vdevd must have a way for the admin to tell it to run helper scripts for the device even if its device node was already created (indicating that it has been set up), so the symlinks and permissions still get set appropriately. Thanks to Scooby for reporting this! Thanks, -Jude ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update (2015-05-03)
Hi Steve, On Thu, May 14, 2015 at 11:03 AM, Steve Litt sl...@troubleshooters.com wrote: On Wed, 13 May 2015 23:24:42 -0400 Jude Nelson jud...@gmail.com wrote: Hey everyone, I have the latest news for vdev: * vdev now creates symlinks for: -- /dev/v4l/by-path -- /dev/disk/by-partuuid -- /dev/disk/by-partlabel Thank you Jude. It's astounding what you've been able to accomplish. Thanks! Fortunately, it's not as bad as it looks--it's mostly a matter of reading the udev rules files and writing a script that does the same thing :) * the initramfs hook conditionally pulls in iproute2, lvm, dmsetup, blkid, etc. I can think of few things more brain taxing than writing a program to tweak a RAM filesystem and accompanying init script that will be used on the *next* boot. Troubleshooting and experimentation must be a slow process, with boots intervening. Even on a VM on a fast host. Yeah, it can get pretty tedious. Fortunately, I'm using a minimal QEMU image, so reboots happen pretty fast (i.e. the working set fits entirely in my disk cache). I have developed a new-found appreciation for the people who designed Debian's initramfs system. Despite the tediousness of building cpio images, Debian's initramfs-tools package is surprisingly flexible and well-documented. There are a couple little things it could make the process less painful, though; I'm in the process of getting together some patches to send to Debian, if they're interested. [For Next Time] * The next immediate objective is to package vdevd for Devuan, to make it easier to test (in particular, easier to get a working initramfs). Could you please make sure that there's a way for us Qemu users to use it? I don't know whether I'm the only one, but Vagrant errors out on my Debian Wheezy machine. So I'm still using the Valentine edition. Absolutely! I intend to make the first iteration of packages available as direct downloads (will post hashes to the ML), and will continue to do so until I can figure out how to hook the vdev repository into Devuan's CI system. Thanks, Jude ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update (2015-05-03)
Le 14/05/2015 15:40, shraptor shraptor a écrit : I would be interested in a static Vdev. Didier could you please give some info how this was done especially any gotcha's you found? Hi Shraptor. I have been working for almost one year, partial time, on building a sysrooted gcc-4.7 bundled with musl-1.1.5. This has been pretty difficult for me, since I am not an expert in Gcc and I could not find it ready-made because I wanted this gcc to understand all languages, including Ada. With Ada, there is a bootstrapping problem because it can be compiled only with the current version or the previous. I started from a chroot containing Sabotage-Linux, which has no Ada, in which I introduced Gnat from Debian Wheezy and I don't remember exactly how I finally succeeded, last november :-) the fact is that I have now a working sysrooted toolchain which can recompile itself and can compile a lot of applications. Sabotage provides the necessary patches for gcc to work with Musl - Big thanks to them - and I patched Gnat myself. The whole process looks like a horrible bricolage, and it was just that. With this (kind of cross) toolchain, installed on my Debian Wheezy laptop, I have succeeded to compile the whole package from Jude, including the filesystem, which is not even an alpha release AFAIU, but has some dependencies and was a good exercise. The main work was to modify the Makefiles so that they can produce static archive libraries and replacing one glibc non-standard macro with the standard one. I am now working on producing a bootable USB flash disk with two partitions, one containing the kernel and the Syslinux bootloader files and one with the root filesystem containing Busybox-1.23.1, Vdevd and its helpers, and even Bash, all statically linked against Musl. I tried to boot it 15mn ago but my kernel is still missing some drivers to be able to mount the usb key. I chose this configuration, rather than an initramfs, to be able to make persistent changes to my root filesystem, but it means that all the drivers needed to mount the root partition must be compiled in the kernel instead of loadable modules. I am very excited at seeing vdev in action and I don't think I will do anything else before I see if it works. Then I promised to Jude to send him the necessary patches. I think that, with the patched version of vdev, you could compile it statically with any ready-made gcc-musl toolchain (eg Sabotage). Could you wait a few days more? Didier ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
cheers to vdev! On Tue, 14 Apr 2015, Jude Nelson wrote: I'm using it as the device manager in my copy of the alpha Vagrant image, and others have tested it locally (but not for booting, AFAIK).* I'm working on getting it to boot the the qcow2 image, which boots from LVM. FYI and to sync a bit on progress, Nextime is completing a total rebuild of all packages for all target architectures via our CI. it took at least 3 days for that, phew. I'm spinning nightlies and testing them when I have time, they get packed with latest packages from our merged repo and the last one of today (150415) is a pretty good one, available on the temporary repository https://files.dyne.org/devuan in case you prefer virtualbox its easy to convert the qcow2 nightlies to that format using qemu-img and I confirm it works. I'm actually using Devuan already as devops base system in one of my software projects eheheh couldn't resist, its so nice to be able to continue to work undisturbed without having to cope with the device renaming b-shit and all that stuff that systemd is imposing... ciao ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
On Tue, Apr 14, 2015 at 07:40:41PM +0200, Anto wrote:
On 14/04/15 17:48, Jude Nelson wrote:
I'm using it as the device manager in my copy of the alpha Vagrant image,
and others have tested it locally (but not for booting, AFAIK).?? I'm
working on getting it to boot the the qcow2 image, which boots from LVM.
Thanks,
-Jude
Thanks Jude,
I am not sure if at this stage, it is worth it to report the problems that I
have just experienced.
I followed your 3 steps instruction to have vdev on the PC that I am
currently using for writing this email (HP EliteBook 8530w), and
surprisingly it does not work (yet) :)
Before anyone ask: no, this is not a production PC as I have been beating
it up so bad in the last 3 years since I got it used from the company I work
for.
In the grub.cfg, I used the same set of files as the ones from the running
kernel generated by update-initramfs, except the initrd that I took from the
example directory under vdev source directory.
It booted the initrd, but just ended up on the (initramfs) prompt. It is
hard to get the error messages during boot. I tried to record them using my
mobile phone and the messages at the beginning are the following:
Loading, please wait...
/init: line 175: resolve_device: not found
00071:859055872: [ vdev.c:0425] vdev_init: ERROR:
vdev_config_load('/etc/vdev/vdevd.conf') rc = -2
00071:859055872: [ main.c:0042] main: ERROR: vdev_init rc = -2
some messages that are too fast to be captured
Not being very familiar with vdev, I can't tell you what this means beyond
that something didn't get copied or ended up in the wrong place.
The last messages that I could see on the screen are about modprobe:
modules some modules not found in modules.dep.
If you needed more information, I am quite happy to provide them.
This sounds like what I ran into with mdev.
https://github.com/idunham/mdev/blob/master/hooks/mdev-hook, last two lines:
# Apparently, this somehow only gets copied if udev is installed.
copy_modules_dir kernel/drivers/scsi
The directory where that is would probably be helpful for integration
with initramfs-tools; it's what I worked out for making mdev replace
udev.
HTH,
Isaac Dunham
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
Hi Anto, Thanks a lot for all your efforts on this. My pleasure :) It looks like we are getting closer now, or at least to be able to compile vdev as a deb package. And it seems the only package that needs to be re-compile is initramfs. Do you think it is reasonable now to start packaging vdev? There are still a few things to be done before I'd recommend using it regularly (i.e. outside of a VM): * finish the remaining device scripts (in particular, scripts for setting up LVM). * integrate vdev, mdev, udev, static /dev, and other device management strategies into Devuan's alternatives system. * fork Debian's initramfs-tools to use the alternatives system to figure out which device manager's files (if any) to include in the initramfs. * finish designing, writing and testing libudev-compat with applications that depend on libudev0 or libudev1 (this one's pretty open-ended still). or did anybody try it already? I'm using it as the device manager in my copy of the alpha Vagrant image, and others have tested it locally (but not for booting, AFAIK). I'm working on getting it to boot the the qcow2 image, which boots from LVM. Thanks, -Jude ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
On 14/04/15 17:11, Jude Nelson wrote: Hey everyone, I didn't have much time to work on vdev this weekend, but I did manage to make the vdev installation process easier. The build process automatically generates the host-specific /etc files for you (i.e. including rules for persistent interface names), as well as an initramfs with vdev installed so you can use it to boot. I've added preliminary instructions on how to install vdev to https://git.devuan.org/pkgs-utopia-substitution/vdev/blob/master/INSTALL.md. DISCLAIMER: the above process has only been lightly tested. I apologize in advance if you encounter bugs. Please do not try this on a production system, and please read *all* of the instructions before attempting to install (since it will mess with your initramfs). I hope to do more next weekend, but it's currently a very busy time of year for me right now (I have a work-related deadline at the end of April that I have to meet). Thanks, Jude Hello Jude, Thanks a lot for all your efforts on this. It looks like we are getting closer now, or at least to be able to compile vdev as a deb package. And it seems the only package that needs to be re-compile is initramfs. Do you think it is reasonable now to start packaging vdev? or did anybody try it already? Cheers, Anto ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
Hi Isaac, On Wed, Apr 8, 2015 at 7:04 PM, Isaac Dunham ibid...@gmail.com wrote: On Tue, Apr 07, 2015 at 05:22:55PM -0400, Jude Nelson wrote: report every kind of device, since it listens to the kernel's driver core (i.e. libudev learns about network interfaces, buses, power supplies, etc.--stuff for which there are no device files Currently, it doesn't *report* devices; that takes something longer term, like inotify, polling a netlink socket, or listening to a daemon. It also has no clue about events or hardware that could not have a corresponding device, since it uses block/char and major:minor to find the hardware. I have a general idea of how to get information like this, by recursing through /sys or /dev, and I know of some code I could use as a starting point, but I don't know what the ideal format is. If someone points me at a program they'd like to use without libudev (preferably C with minimal dependencies) that doesn't cover a lot of ground (ie, it's clear what functionality udev provides, and I wouldn't need to duplicate much of libudev to get it working), that would be a good starting point for expanding libsysdev. You might find something useful in vdev_linux_sysfs_register_devices() and vdev_linux_sysfs_find_devices() functions in vdevd/os/linux.c. They're both involved in generating the initial coldplug device listing. They only need libc to work, and libvdev/sglib.h for basic data structures. I know how to get the devices that show up in /dev; I'm not sure about getting the sysfs entries that *don't* show up there. I'm also not sure how anything beyond this is used. Ah, my bad for misinterpreting your request. I found the sysfs rules overview from kernel.org helpful when I was working on this [1]. Basically, all devices (even ones without a major/minor number) are enumerated under /sys/devices/; the other directories under /sys/ all contain symlinks to device directories in /sys/devices/. Each device in /sys/devices/ has a uevent file that, when read, produces the payload of the netlink packet that the driver core would have sent when the device was added (note that some of them will be empty). The kernel organizes devices into a tree internally, which gets exported via sysfs. Each device has a globally-unique DEVPATH, which is the path to the device's directory under /sys/devices (except, DEVPATH omits the /sys prefix). Moreover, each device has a SUBSYSTEM that identifies the device's parent node in the device tree (which may or may not be a device itself). For example, PCI slot :ff:02.2 on my laptop has a DEVPATH of /devices/pci:ff/:ff:02.2. If you look in /sys/devices/pci:ff/:ff:02.2/, you'll see a symlink called subsystem that contains a symlink to the device's subsystem's device--in this case, ../../../bus/pci (the basename of the path in subystem is the device's subsystem name--in this case, pci). The contents of the DEVPATH directory in sysfs include device-specific attributes--usually stuff like serial numbers, vendor strings, power-related data, etc. Is this the information you were looking for? [1] https://www.kernel.org/doc/Documentation/sysfs-rules.txt To avoid the troublesome corner case where a libudev client crashes and potentially leaves behind a directory in /dev/uevents/, I would recommend mounting runfs [1] on /dev/uevents. Runfs is a special FUSE filesystem I wrote a while back that ensures that the files created in it by a particular process get automatically unlinked when that process dies (it was originally meant for holding PID files). Hmm... Do we need to have a subdirectory of the mountpoint? Could you just use ACLs if you need to make a limited subset available? I get the impression that we can do this for mdev via a script along these lines: FILENAME=`env | sha512sum | cut -d' ' -f1` for f in /dev/uevents/* do env $f/$FILENAME done but it would be *nicer* if we only needed to write one file. I agree that one file per event is ideal (or even a circular logfile of events, if we could guarantee only one writer). However, I'm not sure yet what a fine-grained ACL for device events would look like. My motivation for per-client directories is that unprivileged clients can be made to see only its own events and no one else's by default (i.e. by chmod'ing the directory to 0700), and that they make it easy reason about sending post-processed events only to the clients you want--just change the list of directories to iterate over in that for-loop :) Which is not trivial in shell, unless you have a special command to do the work of figuring out which which directories get what. ...which seems to make doing this in shell pointless, since the corresponding C is nearly as trivial. I don't yet know what the policy declaration and
Re: [Dng] [dng] vdev status update
On Tue, Apr 07, 2015 at 05:22:55PM -0400, Jude Nelson wrote: report every kind of device, since it listens to the kernel's driver core (i.e. libudev learns about network interfaces, buses, power supplies, etc.--stuff for which there are no device files Currently, it doesn't *report* devices; that takes something longer term, like inotify, polling a netlink socket, or listening to a daemon. It also has no clue about events or hardware that could not have a corresponding device, since it uses block/char and major:minor to find the hardware. I have a general idea of how to get information like this, by recursing through /sys or /dev, and I know of some code I could use as a starting point, but I don't know what the ideal format is. If someone points me at a program they'd like to use without libudev (preferably C with minimal dependencies) that doesn't cover a lot of ground (ie, it's clear what functionality udev provides, and I wouldn't need to duplicate much of libudev to get it working), that would be a good starting point for expanding libsysdev. You might find something useful in vdev_linux_sysfs_register_devices() and vdev_linux_sysfs_find_devices() functions in vdevd/os/linux.c. They're both involved in generating the initial coldplug device listing. They only need libc to work, and libvdev/sglib.h for basic data structures. I know how to get the devices that show up in /dev; I'm not sure about getting the sysfs entries that *don't* show up there. I'm also not sure how anything beyond this is used. To avoid the troublesome corner case where a libudev client crashes and potentially leaves behind a directory in /dev/uevents/, I would recommend mounting runfs [1] on /dev/uevents. Runfs is a special FUSE filesystem I wrote a while back that ensures that the files created in it by a particular process get automatically unlinked when that process dies (it was originally meant for holding PID files). Hmm... Do we need to have a subdirectory of the mountpoint? Could you just use ACLs if you need to make a limited subset available? I get the impression that we can do this for mdev via a script along these lines: FILENAME=`env | sha512sum | cut -d' ' -f1` for f in /dev/uevents/* do env $f/$FILENAME done but it would be *nicer* if we only needed to write one file. I agree that one file per event is ideal (or even a circular logfile of events, if we could guarantee only one writer). However, I'm not sure yet what a fine-grained ACL for device events would look like. My motivation for per-client directories is that unprivileged clients can be made to see only its own events and no one else's by default (i.e. by chmod'ing the directory to 0700), and that they make it easy reason about sending post-processed events only to the clients you want--just change the list of directories to iterate over in that for-loop :) Which is not trivial in shell, unless you have a special command to do the work of figuring out which which directories get what. ...which seems to make doing this in shell pointless, since the corresponding C is nearly as trivial. Also, wouldn't mounting that with runfs result in records of uevents getting erased if they're written by a helper rather than a daemon? Yes; good catch. There are a couple straightforward ways to address this: (1) have a separate, unprivileged device-event-log daemon curate /dev/uevents/ and have the helper scripts forward device events to it, or (2) fork and/or patch runfs to allow files to persist if they're generated by a certain whitelist of programs (i.e. all programs in a particular set of directories, like /lib/vdev/), but disappear otherwise once the creating process dies. What about (3) having an option for runfs that lets it erase directories (with their subentries) on process termination, but lets regular files persist until then? Thanks for your feedback! -Jude You're welcome. Thanks, Isaac ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
Scooby, John Carline, lepoitr, and others (who wish to remain anonymous) have been sending me logs filesystem listings from running vdev locally. I very much appreciate it--it helped me discover and fix bugs relating to persistent paths for disk devices, seeding /dev with initial device files, and adding support for Virtualbox USB devices. Thank you to everyone who helped me with this! Thank you for taking on this hard task However, this has proven to be a somewhat challenging problem, for a couple reasons. First, inotify(2) does not work on pseudo-filesystems like sysfs or devtmpfs, so we can't just watch /sys for changes, and we can't use devtmpfs for /dev. This means that systems using libudev-compat will require a vanilla tmpfs for /dev (or nothing at all), so it can detect when devices are added or removed. Second, only block and character devices show up in /dev. However, udevd reports every kind of device, since it listens to the kernel's driver core (i.e. libudev learns about network interfaces, buses, power supplies, etc.--stuff for which there are no device files). Clients will expect this behavior, so it's not enough to simply look up a new block/character device's sysfs data. Is this listen to kernel driver core something you don't want to do? If so is it because one of the project goals is to support several platforms i.e BSD? Or is it to keep it modular and not so interdependent? My tentative solution is to require the device manager (whatever it happens to be) to take one extra step in addition to adding/removing device files: record driver core uevents in a well-defined location in /dev (let's say /dev/uevents/), so libudev clients can discover them (with inotify(2)), read them, and send them off to their applications. This can be done without loss of generality in udev, vdev, and mdev, and I can make a script that takes the appropriate action with mknod (so those with a static /dev can alias mknod to the script, if desired). So you want to split up the listen to kernel driver core uevents part and vdev right? Trying to figure this out I saw you mentioning Isaac Dunham's libsysdev in another message. It is a replacement for libudev, right? Can libsysdev do this part quoted below? report every kind of device, since it listens to the kernel's driver core (i.e. libudev learns about network interfaces, buses, power supplies, etc.--stuff for which there are no device files Is vdev and libsysdev tested together? To avoid the troublesome corner case where a libudev client crashes and potentially leaves behind a directory in /dev/uevents/, I would recommend mounting runfs [1] on /dev/uevents. Runfs is a special FUSE filesystem I wrote a while back that ensures that the files created in it by a particular process get automatically unlinked when that process dies (it was originally meant for holding PID files). I tried runfs on my system that is using aufs and couldn't get it to run I got something like Transport endpoint is not connected This is not a problem for me since dev/ won't be persistent between boots so can run without runfs. best regards Scooby ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
On Tue 07 April 2015 14:04:36 Jude Nelson wrote: Over in systemd-land, I'm pretty sure the plan going forward is to replace netlink with kdbus to do the same task. me starts throwing up a little :-o But yes, sounds plausible /j signature.asc Description: This is a digitally signed message part. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
Hi Scooby, [Snip] However, this has proven to be a somewhat challenging problem, for a couple reasons. First, inotify(2) does not work on pseudo-filesystems like sysfs or devtmpfs, so we can't just watch /sys for changes, and we can't use devtmpfs for /dev. This means that systems using libudev-compat will require a vanilla tmpfs for /dev (or nothing at all), so it can detect when devices are added or removed. Second, only block and character devices show up in /dev. However, udevd reports every kind of device, since it listens to the kernel's driver core (i.e. libudev learns about network interfaces, buses, power supplies, etc.--stuff for which there are no device files). Clients will expect this behavior, so it's not enough to simply look up a new block/character device's sysfs data. Is this listen to kernel driver core something you don't want to do? If so is it because one of the project goals is to support several platforms i.e BSD? Or is it to keep it modular and not so interdependent? Libudev clients expect to receive post-processed driver events from udev via a dedicated netlink multicast group (defined as UDEV_MONITOR_UDEV in libudev-monitor.c). Nothing stops libudev clients from opening their own netlink socket and listening directly to messages in the NETLINK_KOBJECT_UEVENT multicast group, but the advantage of listening to udevd is that udevd takes care of parsing the uevent packet, looking up extra metadata from sysfs and the hardware database, querying the device (!!), and filling in a struct udev_device instance with all the information it gathers. The reason why libudev doesn't just do all of the above is because querying the device is often a privileged operation, since it usually entails doing specialized ioctls. This is why udevd (and vdevd and mdev) need to run as root--the helper programs they run need to have sufficient privileges. This isn't an option for libudev clients, though, which is why they need to get device information from a device manager. My plan is to make vdevd make this information available via the filesystem (for portability and better access control), and patch libudev so clients can retrieve it. Over in systemd-land, I'm pretty sure the plan going forward is to replace netlink with kdbus to do the same task. My tentative solution is to require the device manager (whatever it happens to be) to take one extra step in addition to adding/removing device files: record driver core uevents in a well-defined location in /dev (let's say /dev/uevents/), so libudev clients can discover them (with inotify(2)), read them, and send them off to their applications. This can be done without loss of generality in udev, vdev, and mdev, and I can make a script that takes the appropriate action with mknod (so those with a static /dev can alias mknod to the script, if desired). So you want to split up the listen to kernel driver core uevents part and vdev right? Trying to figure this out I saw you mentioning Isaac Dunham's libsysdev in another message. It is a replacement for libudev, right? libsysdev is meant to help clients pull information from sysfs. I don't think it is supposed to help clients run privileged ioctls (Isaac, please correct me if I'm wrong!). Can libsysdev do this part quoted below? report every kind of device, since it listens to the kernel's driver core (i.e. libudev learns about network interfaces, buses, power supplies, etc.--stuff for which there are no device files Is vdev and libsysdev tested together? They're currently not used together. Libsysdev is meant to help programs that only need libudev for a few things (like querying sysfs) do so without linking against libudev. To avoid the troublesome corner case where a libudev client crashes and potentially leaves behind a directory in /dev/uevents/, I would recommend mounting runfs [1] on /dev/uevents. Runfs is a special FUSE filesystem I wrote a while back that ensures that the files created in it by a particular process get automatically unlinked when that process dies (it was originally meant for holding PID files). I tried runfs on my system that is using aufs and couldn't get it to run I got something like Transport endpoint is not connected This is not a problem for me since dev/ won't be persistent between boots so can run without runfs. Interesting. Were you able to get runfs to run somewhere else, i.e. on a temporary directory? Was there something else mounted there already that crashed earlier? If you run runfs with -f, it will run in the foreground, and that can tell you if something is amiss (you can also run it in valgrind this way). Runfs takes the standard FUSE options (-f for foreground, -s for single-threaded, -o $FUSE_OPTION for FUSE-specific options, -d for FUSE debugging output). best regards Scooby Thanks, Jude ___ Dng mailing list
Re: [Dng] [dng] vdev status update
Hi Isaac, [snip] libsysdev is meant to help clients pull information from sysfs. I don't think it is supposed to help clients run privileged ioctls (Isaac, please correct me if I'm wrong!). Not at present. It basically maps a device to a sysfs directory and possibly gets a *little* more info from that (currently just PCI IDs). You cannot make a library do anything privileged without either helper programs or a helper daemon. Personally, I'm inclined to think it would be nicer to use helpers... but there are limitations to both approaches. I suppose it would be *possible* to make it configureable which it does. Certainly :) I'd like to support that in libudev-compat as an option at some point. Can libsysdev do this part quoted below? report every kind of device, since it listens to the kernel's driver core (i.e. libudev learns about network interfaces, buses, power supplies, etc.--stuff for which there are no device files Currently, it doesn't *report* devices; that takes something longer term, like inotify, polling a netlink socket, or listening to a daemon. It also has no clue about events or hardware that could not have a corresponding device, since it uses block/char and major:minor to find the hardware. I have a general idea of how to get information like this, by recursing through /sys or /dev, and I know of some code I could use as a starting point, but I don't know what the ideal format is. If someone points me at a program they'd like to use without libudev (preferably C with minimal dependencies) that doesn't cover a lot of ground (ie, it's clear what functionality udev provides, and I wouldn't need to duplicate much of libudev to get it working), that would be a good starting point for expanding libsysdev. You might find something useful in vdev_linux_sysfs_register_devices() and vdev_linux_sysfs_find_devices() functions in vdevd/os/linux.c. They're both involved in generating the initial coldplug device listing. They only need libc to work, and libvdev/sglib.h for basic data structures. Is vdev and libsysdev tested together? They're currently not used together. Libsysdev is meant to help programs that only need libudev for a few things (like querying sysfs) do so without linking against libudev. To avoid the troublesome corner case where a libudev client crashes and potentially leaves behind a directory in /dev/uevents/, I would recommend mounting runfs [1] on /dev/uevents. Runfs is a special FUSE filesystem I wrote a while back that ensures that the files created in it by a particular process get automatically unlinked when that process dies (it was originally meant for holding PID files). Hmm... Do we need to have a subdirectory of the mountpoint? Could you just use ACLs if you need to make a limited subset available? I get the impression that we can do this for mdev via a script along these lines: FILENAME=`env | sha512sum | cut -d' ' -f1` for f in /dev/uevents/* do env $f/$FILENAME done but it would be *nicer* if we only needed to write one file. I agree that one file per event is ideal (or even a circular logfile of events, if we could guarantee only one writer). However, I'm not sure yet what a fine-grained ACL for device events would look like. My motivation for per-client directories is that unprivileged clients can be made to see only its own events and no one else's by default (i.e. by chmod'ing the directory to 0700), and that they make it easy reason about sending post-processed events only to the clients you want--just change the list of directories to iterate over in that for-loop :) Also, wouldn't mounting that with runfs result in records of uevents getting erased if they're written by a helper rather than a daemon? Yes; good catch. There are a couple straightforward ways to address this: (1) have a separate, unprivileged device-event-log daemon curate /dev/uevents/ and have the helper scripts forward device events to it, or (2) fork and/or patch runfs to allow files to persist if they're generated by a certain whitelist of programs (i.e. all programs in a particular set of directories, like /lib/vdev/), but disappear otherwise once the creating process dies. Thanks, Isaac Dunham Thanks for your feedback! -Jude ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
Given the fraction of replies you got so far devoted to the puppy (100%) and to the technical content (0%), I suspect this community's hearts are in the right place. -- hendrik ^ +1 Linux O'Beardly @LinuxOBeardly http://o.beard.ly linux.obear...@gmail.com On Tue, Mar 31, 2015 at 11:45 AM, Hendrik Boom hend...@topoi.pooq.com wrote: On Tue, Mar 31, 2015 at 11:44:25AM -0400, Jude Nelson wrote: Thankfully, the puppy should be fine :) She only ate a few capsules, and we caught her just after the fact, so we were able to get her proper treatment before the toxicity could set in. We get to take her home from the vet either tonight or tomorrow. -Jude Given the fraction of replies you got so far devoted to the puppy (100%) and to the technical content (0%), I suspect this community's hearts are in the right place. -- hendrik ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
How is the puppy?!? On 31 March 2015 at 06:20, Jude Nelson jud...@gmail.com wrote: Hey everyone, I don't have as much to report this week as I usually do, since I spent half my weekend dealing with a pet medical emergency (TL;DR: ibuprofen is toxic to puppies, and puppies can chew through child-proof lids). I am happy to report that I was able to extract the code for libudev 219 from systemd 219, and get it to compile independently. The resulting library is ABI-compatible with systemd's (no real surprise). My next steps are working through a couple vdev bugs that Scooby helped me find, as well as get libudev to work without udevd. More next week, Jude ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update
Thankfully, the puppy should be fine :) She only ate a few capsules, and we caught her just after the fact, so we were able to get her proper treatment before the toxicity could set in. We get to take her home from the vet either tonight or tomorrow. -Jude On Tue, Mar 31, 2015 at 3:15 AM, Archangel Amael archangel.am...@gmail.com wrote: Well how are the puppies? Hey everyone, I don't have as much to report this week as I usually do, since I spent half my weekend dealing with a pet medical emergency (TL;DR: ibuprofen is toxic to puppies, and puppies can chew through child-proof lids). I am happy to report that I was able to extract the code for libudev 219 from systemd 219, and get it to compile independently. The resulting library is ABI-compatible with systemd's (no real surprise). My next steps are working through a couple vdev bugs that Scooby helped me find, as well as get libudev to work without udevd. More next week, Jude ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
Le 25/03/2015 18:04, Jude Nelson a écrit : 3) I don't think this is a great place for unsubstantiated attacks on Ulrich Drepper or on Ulrich Drepper's leadership of glibc. Ulrich Drepper's bad attitude is cited as one of the main reasons Debian switched from glibc to eglibc. Source: http://blog.aurel32.net/47. -Jude I see two flaws in the glibc: first it is bloated, second the static version does not contain all functions. To my knowledge, the second flaw is by the personal will of Ulrich Drepper. These are two reasons to prefer Musl libc. Nevertheless, Glibc designs the standard and this is not without merit. And I am not advocating to convert Devuan to Musl :-) there are more important things to do. Thanks Jude! Didier ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
3) I don't think this is a great place for unsubstantiated attacks on Ulrich Drepper or on Ulrich Drepper's leadership of glibc. Ulrich Drepper's bad attitude is cited as one of the main reasons Debian switched from glibc to eglibc. Source: http://blog.aurel32.net/47. -Jude On Wed, Mar 25, 2015 at 1:01 PM, Tor Myklebust tmykl...@csclub.uwaterloo.ca wrote: On Wed, 25 Mar 2015, T.J. Duchene wrote: For some, it does have to do with the GPL, but for many it has to do with the fact that GCC is a mess. One of the previous maintainers, Ulrich Drepper, was famous for ignoring bug reports. Are you for real? If so: 1) Drepper maintained glibc, not gcc. These are two separate projects. 2) Clang is a C and C++ frontend for LLVM. It does not contain a C library. 3) I don't think this is a great place for unsubstantiated attacks on Ulrich Drepper or on Ulrich Drepper's leadership of glibc. 4) I also don't think this is a great place for unsubstantiated attacks on a compiler suite. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
-Original Message- From: Joerg Reisenweber [mailto:reisenwe...@web.de] Sent: Tuesday, March 24, 2015 10:33 PM To: dng@lists.dyne.org Subject: Re: [Dng] [dng] vdev status update and milestone On Tue 24 March 2015 22:17:20 Steve Litt wrote: This systemd debacle increased by an order of magnitude the Linux users who understand the underpinnings of the system and are prepared to take control. [T.J. ] I don't really think so, honestly. I think that those who have would have been motivated for other reasons. 2015-02-18T14:43:35.751937+01:00 localhost kernel: [1566879.692218] systemd[1]: segfault at 1fc6ac0 ip 01fc6ac0 sp 7fff9f2ab858 error 15 [T.J. ] It should be important to note that a segfault can be caused by any number of things, that can be unrelated to systemd itself. I do grant you that systemd has its share of undesirables, but it could be exposing a flaw in the lower libraries as well. A lot of the time, the glibc library is also to blame. If there was ever any piece of software on Linux that needs a serious overhaul, beyond X11, it is the libc and GCC suite. That is why a lot of people are eying over Clang very seriously. For some, it does have to do with the GPL, but for many it has to do with the fact that GCC is a mess. One of the previous maintainers, Ulrich Drepper, was famous for ignoring bug reports. It has been my experience (having used just about every major version of Linux at one time or another since the 90s) that Linux versions get pushed out the door, regardless of how many critical bugs are still unfixed. The usual attitude is that we will fix them later, or they wait for upstream to do so. Some versions are better than others. I must admit that Debian seems to do a good job at squashing bugs. on my desktop system, with all the collateral damage like var/log/messages empty after logrotate etc, and NO way to debug. *No more* 'customize and tweak everything I like' :-/ [T.J. ] To be fair, there is always a way to be debug, but like most things these days, systemd is designed to be hands off unless you are a C programmer. Which makes it very attractive to people who just want to build a distribution and not get into the guts. That also means that it gets in the way of the people who do want to get into the details. As for your comments about logging, I agree - systemd should not be logging by default. On Debian, it doesn't actually - even if it is installed. Most other distributions have systemd logging on be default though. It is also true that you can disable systemd logging should you wish it. Now devuan is my only hope for the future of linux - alternative: move to another unix. When I wanted an OS like systemd cabal and RH is planning for, I would have chosen the original from beginning: Redmond M$ lock-in for sheep. [T.J. ] I think you are being just a bit overdramatic. I do agree that the number of binary distributions that do not use systemd are shrinking to zero, but then the binary distributions were created in the first place for people who do not care about the internal guts of the system. You can always build Linux yourself to exclude systemd or you can even try Gentoo if you are so inclined. T.J. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
Are you for real? [T.J. ] Just to clear things up. If so: 1) Drepper maintained glibc, not gcc. These are two separate projects. True. I always treat GCC and glibc as somewhat synonymous since they go hand in hand. You can't have one without the other for all intents as far as Linux is concerned. 2) Clang is a C and C++ frontend for LLVM. It does not contain a C library. Good point. It is however far superior to GCC in many regards. 3) I don't think this is a great place for unsubstantiated attacks on Ulrich Drepper or on Ulrich Drepper's leadership of glibc. Attacking Drepper or his reputation was never my intention, so clearly this is only a misunderstanding on your part. I know my comments about him ignoring bug reports are correct, I've read the reports myself. That is 100% accurate. Although obviously when I wrote the comment I should have said glibc instead of GCC. I used both in the previous sentences and did not specifically separate them when I mentioned Drepper. Like everyone else, I am only human and occasionally make errors. Thank you for the correction. Mea Culpa. 4) I also don't think this is a great place for unsubstantiated attacks on a compiler suite. Unsubstantiated? I must humbly disagree. GCC has a long history of open bugs that do not get corrected for years. This one took 10 years - 10 YEARS - for to be corrected: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=18501. I'd hardly call that criticism unsubstantiated. Saying that the GCC is a mess is perhaps subjective, but it is hardly unfair. It is a complex piece of software that like X11, to needs its development be shaken up every now and again. GCC has been completely replaced by a fork once already. EGCS replaced GCC and simply took the old name GCC name. T.J. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
KatolaZ, [T.J. ] What I said was: It should be important to note that a segfault can be caused by any number of things, that can be unrelated to systemd itself. I do grant you that systemd has its share of undesirables, but it could be exposing a flaw in the lower libraries as well. A lot of the time, the glibc library is also to blame. If there was ever any piece of software on Linux that needs a serious overhaul, beyond X11, it is the libc and GCC suite. I said that systemd COULD be (not IS) be exposing a flaw in lower libraries (which is not unreasonable to say, given that it DOES happen). I said that a lot of the time the glibc library is ALSO to blame - as in addition to when certain problems arise. I agree with you that I definitely should have qualified that better. Defaults are not necessarily caused by glibc itself, but glibc DOES have certain quirks. It is not perfect and sometimes does not follow conformant behavior. One example would be realloc violates C99. I don't know if it still does, you will have to look for yourself. Software that is compiled against one libc is not the same as compiled against another.Glibc has a history of shortcomings, just as much as its successes. That does not make it a bad piece of software, but it is hardly flawless. That is has been forked or replaced multiple times in its history says something about it. So please, do not blame glibc for faults that are not its own. On the other If you can prove that glibc/gcc is causing the segfaults of systemd then please provide links to those *facts*, otherwise what you are saying is just unsopported FUD. I agree that it is good that you desire accuracy, but you could ask for it without saying I am chanting unsupported FUD. Laters T.J. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
Defaults are not necessarily caused by glibc itself Defaults = Segfaults. Don't you just hate autocorrect? T.J. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
If something goes wrong somewhere and X11 segfaults (which I think does not happen more than once in a few decades, at least with the stable version of Xorg), then we might complain and make a fuss, but in the end is not that big deal. Having systemd as PID 1 segfaulting is a completely different story, and blaming other libraries does not help systemd acquiring more credibility. PID1 *cannot* segfault, or we are just back to the dark days of BSODs. Fullstop. [T.J. ] That's fine. =) I only meant that glibc and GCC are far from flawless and which may cause problems with software that is built with them. I have personally seen C/C++ code segfault on Linux simply because a single line (having nothing to do with anything other than standard C/C++ libraries) was written one way over another, when both ways should have worked. I apologize the commentary is not specific enough for some, it has been a number of years since the last instance, and I do not remember the exact code. It might have been on an Alpha processor if that matters - perhaps it was x86. I have not worked with anything besides X64 now for some time, hence why I do not remember the specifics. As for my mentioning realloc and C99, you are certainly welcome to look that up to see what the general state of things are. In fact, I will go so far as to publicly withdraw my comments as too generalized, and save anyone else the trouble. I have absolutely no more time to waste on the topic, and I do not think that it is fair to anyone else to spend any more time on this either. T.J. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
On Wed, Mar 25, 2015 at 04:31:47PM -0500, T.J. Duchene wrote: KatolaZ, [T.J. ] What I said was: It should be important to note that a segfault can be caused by any number of things, that can be unrelated to systemd itself. I do grant you that systemd has its share of undesirables, but it could be exposing a flaw in the lower libraries as well. A lot of the time, the glibc library is also to blame. If there was ever any piece of software on Linux that needs a serious overhaul, beyond X11, it is the libc and GCC suite. I said that systemd COULD be (not IS) be exposing a flaw in lower libraries (which is not unreasonable to say, given that it DOES happen). I still don't get the relationship between segfaults in systemd and quirks in glibc, so I am sorry but your statement remains unclear and unsubstantiated, IMHO. Anyway, the fact that systemd depends on mamy more libraries than sysv-init is exactly why we don't want systemd as PID 1, controlling everything from mounting of devices to networking to logging. It is just a single point of fault with too many potential (and actual) reasons that can cause it to fail. If something goes wrong somewhere and X11 segfaults (which I think does not happen more than once in a few decades, at least with the stable version of Xorg), then we might complain and make a fuss, but in the end is not that big deal. Having systemd as PID 1 segfaulting is a completely different story, and blaming other libraries does not help systemd acquiring more credibility. PID1 *cannot* segfault, or we are just back to the dark days of BSODs. Fullstop. My2Cents KatolaZ -- [ Enzo Nicosia aka KatolaZ --- GLUG Catania -- Freaknet Medialab ] [ me [at] katolaz.homeunix.net -- http://katolaz.homeunix.net -- ] [ GNU/Linux User:#325780/ICQ UIN: #258332181/GPG key ID 0B5F062F ] [ Fingerprint: 8E59 D6AA 445E FDB4 A153 3D5A 5F20 B3AE 0B5F 062F ] ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
moreover on libudev looking at Debian bug 735275 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=735275 and the wide impact it has on upstream (nodejs, webkit, chrome etc. are affected by the bump to libudev.1) I'd rather keep libudev.0 around as a vdev patched version, would that be possible? I understand this would gain Devuan even more backward compatibility with upstream distributed binaries. ciao ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
HOORAY! KUDOS! can't wait giving it a try /j On Tue 24 March 2015 05:37:04 Jude Nelson wrote: Hey everyone, I'm pleased to announce that vdev can successfully boot to a console on the Devuan vagrant image! [...] -Jude signature.asc Description: This is a digitally signed message part. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
On Tue, Mar 24, 2015 at 05:37:04AM -0400, Jude Nelson wrote: Hey everyone, I'm pleased to announce that vdev can successfully boot to a console on the Devuan vagrant image! It creates all requisite device files and loads all requisite kernel drivers, both for the pre-boot initramfs environment (so init can mount root) and in the early boot environment (while root is mounted read-only). I'd like to thank you all for your patience and That's great news Jude, indeed! Looking forward to seeing vdev up and running, as soon as I manage to solve some problems I have with vagrant :) AdMaiora KatolaZ -- [ Enzo Nicosia aka KatolaZ --- GLUG Catania -- Freaknet Medialab ] [ me [at] katolaz.homeunix.net -- http://katolaz.homeunix.net -- ] [ GNU/Linux User:#325780/ICQ UIN: #258332181/GPG key ID 0B5F062F ] [ Fingerprint: 8E59 D6AA 445E FDB4 A153 3D5A 5F20 B3AE 0B5F 062F ] ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
Jude Nelson wrote: Hey everyone, I'm pleased to announce that vdev can successfully boot to a console on the Devuan vagrant image! It creates all requisite device files and loads all requisite kernel drivers, both for the pre-boot initramfs environment (so init can mount root) and in the early boot environment (while root is mounted read-only). I'd like to thank you all for your patience and support, with a special shoutout to the individual who goes by the name Scooby on this list who helped me find and fix early-boot bugs. Thanks Jude. I think it's a great contribution to the community. -- Joel Roth ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
On Tue 24 March 2015 22:17:20 Steve Litt wrote: This systemd debacle increased by an order of magnitude the Linux users who understand the underpinnings of the system and are prepared to take control. Indeed. I never really bothered much, and happily lived along with (Open)Suse since... it became available. Never really cared about which distro I use, they all needed heavy customization to make me feel at home anyway. The reason why I chosen unix/linux to start with, even for my phone: I can customize and tweak everything I like. And then recently I first ran into public systemd schism (though I failed to notice it when deciding which init to use in Opensuse12(?), otherwise I should have known more early) and - ironically - later on into 2015-02-18T14:43:35.751937+01:00 localhost kernel: [1566879.692218] systemd[1]: segfault at 1fc6ac0 ip 01fc6ac0 sp 7fff9f2ab858 error 15 on my desktop system, with all the collateral damage like var/log/messages empty after logrotate etc, and NO way to debug. *No more* 'customize and tweak everything I like' :-/ Now devuan is my only hope for the future of linux - alternative: move to another unix. When I wanted an OS like systemd cabal and RH is planning for, I would have chosen the original from beginning: Redmond M$ lock-in for sheep. /j signature.asc Description: This is a digitally signed message part. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
On Tue, 24 Mar 2015 22:21:30 -0400 Clarke Sideroad clarke.sider...@gmail.com wrote: As history marks the rise and fall of systemd, contributions such as yours will make the the bigger picture of this time period net positive. Thank you, Clarke That's so true, Clarke! This systemd debacle increased by an order of magnitude the Linux users who understand the underpinnings of the system and are prepared to take control. This is a Renaissance in the Linux community. Well, except for Debian :-) SteveT Steve Litt* http://www.troubleshooters.com/ Troubleshooting Training * Human Performance ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
On Tue, 24 Mar 2015 05:37:04 -0400 Jude Nelson jud...@gmail.com wrote: Hey everyone, I'm pleased to announce that vdev can successfully boot to a console on the Devuan vagrant image! It creates all requisite device files and loads all requisite kernel drivers, both for the pre-boot initramfs environment (so init can mount root) and in the early boot environment (while root is mounted read-only). I'd like to thank you all for your patience and support, with a special shoutout to the individual who goes by the name Scooby on this list who helped me find and fix early-boot bugs. We're well on our way now to replacing udev entirely. The only big-ticket item remaining prior to an official alpha release is patching libudev so it does not need to talk to udevd to query devices. This is only necessary for applications that require libudev. -Jude * * \ o / \|/ | O U T S T A N D I N G ! ! ! / \ _ / \/ / - This is spectacular news. Thank you Jude. I wasn't looking forward to all the manual driver setup and mknodding I would have had to do in order to solder-bridge around udev. Now I don't have to. This is also a great moment of credibility for Devuan. Keep up the excellent work! SteveT Steve Litt* http://www.troubleshooters.com/ Troubleshooting Training * Human Performance ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
As history marks the rise and fall of systemd, contributions such as yours will make the the bigger picture of this time period net positive. Thank you, Clarke ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [Dng] [dng] vdev status update and milestone
Hi Jaromil, outstanding good news! congrats :^) if we manage to include vdev in Devuan for upcoming releases you may acquire rather wide testing grounds for your software. Thanks for the encouragement :) When it comes to packaging vdev, what I'll do is add an /etc/alternatives entry for the system's device manager, and modify the device manager init script and initramfs hooks to use it. It should be possible for the user to install and select between multiple different device managers. This way, people can use whatever works for them. I'm curious how you think we should plan this? as with all udev, also libudev is now part of systemd, part of their tactical move to lockin everyone rather than keep the codebases separate. The tie-in to systemd comes through its increasing dependence on some string utility methods systemd has (nothing that can't be put into a small util.c file in libudev), as well as the recent switch-over to including the hardware database parser in systemd. I'll just lift out the hwdb parser and put it back into libudev (or get a copy from eudev). libudev doesn't really do that much. It's basically a front-end for (1) reading data from sysfs, (2) querying the hardware database, and (3) getting information from udevd. I've already had to address a good chunk of (1) in the Linux port of vdevd, and most of (3) can be handled simply by watching /dev for changes and looking up newly-added device node metadata from sysfs. how many applications actually require using libudev? Lots of middleware programs do. udisks2, lvm2, mountall, ConsoleKit, bluez, gvfs, kde-workspace-bin, pulseaudio, plymouth, multipath-tools, spacefm, v4l-utils, vlc-nox, and xserver-xorg-core are all compiled to rely on libudev, for example. shall we fork libudev back out of systemd (libvdev?) That's my plan. I'm calling it libudev-compat, though (libvdev is already taken--it holds common code between vdevd and vdevfs). vdevd doesn't need a library to communicate with it. It stores all the information you'd get from udevadm under /dev/vdev/NAME_OF_DEVICE as a set of files, so programs can just read it or inotify-watch it directly. what else? we may also advertise to upstream developers the alternative and ask them to include a --with-vdev flag or so, linking to your api. Hopefully not even that. If I can supply a libudev workalike, they shouldn't have to do anything. -Jude On Tue, Mar 24, 2015 at 6:36 AM, Jaromil jaro...@dyne.org wrote: On Tue, 24 Mar 2015, Jude Nelson wrote: I'm pleased to announce that vdev can successfully boot to a console on the Devuan vagrant image!* It creates all requisite device files and loads all requisite kernel drivers, both for the pre-boot initramfs environment (so init can mount root) and in the early boot environment (while root is mounted read-only).* outstanding good news! congrats :^) if we manage to include vdev in Devuan for upcoming releases you may acquire rather wide testing grounds for your software. We're well on our way now to replacing udev entirely.* The only big-ticket item remaining prior to an official alpha release is patching libudev so it does not need to talk to udevd to query devices.* This is only necessary for applications that require libudev. I'm curious how you think we should plan this? as with all udev, also libudev is now part of systemd, part of their tactical move to lockin everyone rather than keep the codebases separate. how many applications actually require using libudev? shall we fork libudev back out of systemd (libvdev?) what else? we may also advertise to upstream developers the alternative and ask them to include a --with-vdev flag or so, linking to your api. ciao ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
