Re: [gentoo-user] what about dracut and systemd?
On Sunday 30 Jul 2017 17:38:21 Rich Freeman wrote: > The one thing I will say about git is that you'll never appreciate it > until you understand how it works. In many ways git is a data model > masquerading as a version control system. The data model is very > elegant, but also very different to what came before, and until you > understand it you'll be copy/pasting commands off of webpages that > don't seem very intuitive. In that case I have a lot of reading to do. Ho hum... -- Regards Peter
Re: [gentoo-user] what about dracut and systemd?
On Sun, Jul 30, 2017 at 5:27 PM, Peter Humphreywrote: > On Sunday 30 Jul 2017 11:02:41 Rich Freeman wrote: > >> The general sense is that Changelogs represent the old way of doing >> things. Most projects have gone away from having them, or they just >> auto-generate them from git. I don't think most projects routinely >> distribute them either - they just stick them on a webpage and only >> people who care about them look at them. The linux kernel only >> includes the changes in the last release in their change logs as well >> (which is nothing more than a dump of git log). > > This has me a bit bothered, wearing my ISO9000 hat. Change logs seem to me > to be the most important traceability tool we can have. Is it not essential > to be able to demonstrate clearly how we got to where we are today? > Preferably without having to jump through a load of abstruse hoops. Git tells you everything that Changelogs did, and more. It isn't all that obscure either, considering that just about everybody is using it. Aside from the fact that nobody has gotten around to fixing the hash function it also can provide gpg-signed histories that are pretty-much impossible to tamper with. In the case of Gentoo they can also track correlated changes across the entire repository and not just per-package histories. If somebody changes 3 packages at the same time, git will show that they changed 3 packages at the same time. With the old changelogs you'd have to hunt through all of them to find out that 3 things were changed with the timestamp, but it wasn't actually atomic. The one thing I will say about git is that you'll never appreciate it until you understand how it works. In many ways git is a data model masquerading as a version control system. The data model is very elegant, but also very different to what came before, and until you understand it you'll be copy/pasting commands off of webpages that don't seem very intuitive. -- Rich
Re: [gentoo-user] what about dracut and systemd?
On Sunday 30 Jul 2017 11:02:41 Rich Freeman wrote: > The general sense is that Changelogs represent the old way of doing > things. Most projects have gone away from having them, or they just > auto-generate them from git. I don't think most projects routinely > distribute them either - they just stick them on a webpage and only > people who care about them look at them. The linux kernel only > includes the changes in the last release in their change logs as well > (which is nothing more than a dump of git log). This has me a bit bothered, wearing my ISO9000 hat. Change logs seem to me to be the most important traceability tool we can have. Is it not essential to be able to demonstrate clearly how we got to where we are today? Preferably without having to jump through a load of abstruse hoops. -- Regards Peter
Re: [gentoo-user] what about dracut and systemd?
On Sun, Jul 30, 2017 at 10:53 AM, John Coviciwrote: > > Thanks. clone-depth seems not to be available, so I amnot sure whats > best here. I thinkI like the history, so I will see how to do a git > clone. I do havethe type as git in the gentoo.conf, but I don't know > what happened to clone-depth -- its not in the portage man page. > Feel free to attach the file and we can take a quick look at it. I'm also not sure if it only applies on the initial sync. I think the option has been around for a while so I'm not sure why it isn't working if you really do have the type set to git. And you can always browse the history from the web viewers. The reason that we ended up removing Changelogs is that they end up adding a lot of bandwidth to the sync process (and of course they take up space), when you only rarely look at them. Even if you occassionally do look at one, in order for it to be there you have thousands of them constantly being synced. The other challenge was that with the way rsync worked it greatly increases the bandwidth transferred if new entries are added to the top (if they were added to the bottom rsync would only transmit the last block of the file). However, from a convenience standpoint it is usually nicer to have new entries at the top. The general sense is that Changelogs represent the old way of doing things. Most projects have gone away from having them, or they just auto-generate them from git. I don't think most projects routinely distribute them either - they just stick them on a webpage and only people who care about them look at them. The linux kernel only includes the changes in the last release in their change logs as well (which is nothing more than a dump of git log). Encouraging users to use git is also a relatively positive thing, because this is a tool with extremely widespread use. The exact same commands you use to find out what is going on in the Gentoo repo will serve you just as well when trying to figure out what is going on in some other project. It also makes it trivial to do historical checkouts - users sometimes are looking for old portage snapshots to try to deal with updating outdated systems, and with git this is pretty trivial to accomplish. -- Rich
Re: [gentoo-user] what about dracut and systemd?
On Sun, 30 Jul 2017 09:53:18 -0400, Rich Freeman wrote: > > On Sun, Jul 30, 2017 at 9:36 AM, John Coviciwrote: > > On Sun, 30 Jul 2017 09:07:03 -0400, > > Rich Freeman wrote: > >> > >> On Sat, Jul 29, 2017 at 10:58 PM, John Covici > >> wrote: > >> > > >> > Well, clone-depth = 0 gave me a syntax error > >> > >> Can you provide the entire contents of your repos.conf, and the error > >> it gives you? > >> > >> I wouldn't use a manual checkout for /usr/portage. You can of course > >> do a checkout elsewhere but your permissions will probably get messed > >> up if you checkout /usr/portage by hand. > > > > Well, I don't have the error message anymore and I did what I thought > > you suggested -- beffore I got your last message -- I just did a git > > clone of the repository to /usr/portage and the only unusual thing was > > the first time I ran portage, it did its global updates and it seems > > to be working. > > > > Well, if your repos.conf still says to use rsync then you can't use > clone-depth in it, since that doesn't work with rsync. It might also > mean that you now have files owned by root in /usr/portage which an > emerge --sync is going to have trouble dealing with, since it runs > with reduced permissions. It isn't a big deal - just get repos.conf > the way you want it (either with git or rsync), delete /usr/portage, > and let an emerge --sync re-create it. > > If you want to do a git clone I'd do it someplace other than /usr/portage. > > You really have a few options here: > > 1. If you look at history all the time just set up /usr/portage to be > a git repository with clone-depth=0. Make sure you use the repository > I linked which has metadata pre-generated - if you sync from the > official Gentoo repo you'll have to rebuild the metadata all the time. > Plus the mirror I linked also has QA checks so that when a developer > breaks something it just won't update until somebody fixes it, which > will save you some grief (maybe 1-2x a year it ends up being a few > days stale). > > 2. If you almost never look at history and you care about disk space > then either use rsync or sync with clone-depth=1 to minimize the size > of /usr/portage, and then do a temporary git clone when you really > need it, or browse history online. > > Personally I do #1. I have full history, and it syncs a lot faster > than rsync. Git is smarter about syncing since it knows about history > and thus what has changed. However, it has to send all the commits > in-between your current and target state. If you only sync once a > month it might be faster to use rsync, since rsync only sends the > final version, and not every intermediate one, so files that were both > added and removed in the interim don't get sent. However, rsync > doesn't know anything about history and has to scan the entire tree to > figure out what has changed. > > Oh, if /usr/portage is a git repository refrain from editing anything > in it unless you know what you're doing. With rsync portage will just > overwrite anything you mess with on the next sync. With git it will > try to keep those changes around, and it might result in the tree not > syncing if things conflict. If you're messing with a file for testing > purposes you need to remember to reset/clean when you're done. > > > -- > Rich Thanks. clone-depth seems not to be available, so I amnot sure whats best here. I thinkI like the history, so I will see how to do a git clone. I do havethe type as git in the gentoo.conf, but I don't know what happened to clone-depth -- its not in the portage man page. Thanks again -- this has beeninstructive for me. -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] what about dracut and systemd?
On Sun, Jul 30, 2017 at 9:36 AM, John Coviciwrote: > On Sun, 30 Jul 2017 09:07:03 -0400, > Rich Freeman wrote: >> >> On Sat, Jul 29, 2017 at 10:58 PM, John Covici wrote: >> > >> > Well, clone-depth = 0 gave me a syntax error >> >> Can you provide the entire contents of your repos.conf, and the error >> it gives you? >> >> I wouldn't use a manual checkout for /usr/portage. You can of course >> do a checkout elsewhere but your permissions will probably get messed >> up if you checkout /usr/portage by hand. > > Well, I don't have the error message anymore and I did what I thought > you suggested -- beffore I got your last message -- I just did a git > clone of the repository to /usr/portage and the only unusual thing was > the first time I ran portage, it did its global updates and it seems > to be working. > Well, if your repos.conf still says to use rsync then you can't use clone-depth in it, since that doesn't work with rsync. It might also mean that you now have files owned by root in /usr/portage which an emerge --sync is going to have trouble dealing with, since it runs with reduced permissions. It isn't a big deal - just get repos.conf the way you want it (either with git or rsync), delete /usr/portage, and let an emerge --sync re-create it. If you want to do a git clone I'd do it someplace other than /usr/portage. You really have a few options here: 1. If you look at history all the time just set up /usr/portage to be a git repository with clone-depth=0. Make sure you use the repository I linked which has metadata pre-generated - if you sync from the official Gentoo repo you'll have to rebuild the metadata all the time. Plus the mirror I linked also has QA checks so that when a developer breaks something it just won't update until somebody fixes it, which will save you some grief (maybe 1-2x a year it ends up being a few days stale). 2. If you almost never look at history and you care about disk space then either use rsync or sync with clone-depth=1 to minimize the size of /usr/portage, and then do a temporary git clone when you really need it, or browse history online. Personally I do #1. I have full history, and it syncs a lot faster than rsync. Git is smarter about syncing since it knows about history and thus what has changed. However, it has to send all the commits in-between your current and target state. If you only sync once a month it might be faster to use rsync, since rsync only sends the final version, and not every intermediate one, so files that were both added and removed in the interim don't get sent. However, rsync doesn't know anything about history and has to scan the entire tree to figure out what has changed. Oh, if /usr/portage is a git repository refrain from editing anything in it unless you know what you're doing. With rsync portage will just overwrite anything you mess with on the next sync. With git it will try to keep those changes around, and it might result in the tree not syncing if things conflict. If you're messing with a file for testing purposes you need to remember to reset/clean when you're done. -- Rich
Re: [gentoo-user] what about dracut and systemd?
On Sun, 30 Jul 2017 09:07:03 -0400, Rich Freeman wrote: > > On Sat, Jul 29, 2017 at 10:58 PM, John Coviciwrote: > > > > Well, clone-depth = 0 gave me a syntax error > > Can you provide the entire contents of your repos.conf, and the error > it gives you? > > I wouldn't use a manual checkout for /usr/portage. You can of course > do a checkout elsewhere but your permissions will probably get messed > up if you checkout /usr/portage by hand. Well, I don't have the error message anymore and I did what I thought you suggested -- beffore I got your last message -- I just did a git clone of the repository to /usr/portage and the only unusual thing was the first time I ran portage, it did its global updates and it seems to be working. -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] what about dracut and systemd?
On Sat, Jul 29, 2017 at 10:58 PM, John Coviciwrote: > > Well, clone-depth = 0 gave me a syntax error Can you provide the entire contents of your repos.conf, and the error it gives you? I wouldn't use a manual checkout for /usr/portage. You can of course do a checkout elsewhere but your permissions will probably get messed up if you checkout /usr/portage by hand. -- Rich
Re: [gentoo-user] what about dracut and systemd?
On Sat, 29 Jul 2017 16:05:49 -0400, Rich Freeman wrote: > > On Sat, Jul 29, 2017 at 4:24 AM, John Coviciwrote: > > OK, I changed to a git repository and did a git whatchanged, but in > > the directory I was looking at namely sys-kernel/dracut, I just got a > > generated commit from about the time I created the repository and none > > of the previous history. Maybe from now on I will get something > > better. > > > > If you obtained your repository by editing repos.conf then you also > need to set sync-depth = 0 if you want to actually get any history. > By default portage will not pull any history when syncing git. I'm > not sure if it will fetch the old history or not if you add it after > the initial sync. The default is intended to keep the size down while > allowing more efficient syncs (git is more efficient than rsync if you > are doing frequent updates). > > If you just do a git clone then you should get everything by default. Well, clone-depth = 0 gave me a syntax error and it said in the portage manpage clone-depth Specifies clone depth to use for DVCS repositories. Defaults to 1 (only the newest commit). If set to 0, the depth is unlimited. So, maybe I have to do just a git clone by hand? -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] what about dracut and systemd?
On Sat, Jul 29, 2017 at 4:24 AM, John Coviciwrote: > OK, I changed to a git repository and did a git whatchanged, but in > the directory I was looking at namely sys-kernel/dracut, I just got a > generated commit from about the time I created the repository and none > of the previous history. Maybe from now on I will get something > better. > If you obtained your repository by editing repos.conf then you also need to set sync-depth = 0 if you want to actually get any history. By default portage will not pull any history when syncing git. I'm not sure if it will fetch the old history or not if you add it after the initial sync. The default is intended to keep the size down while allowing more efficient syncs (git is more efficient than rsync if you are doing frequent updates). If you just do a git clone then you should get everything by default. -- Rich
Re: [gentoo-user] what about dracut and systemd?
On Sat, 29 Jul 2017 06:17:31 -0400, Rich Freeman wrote: > > On Sat, Jul 29, 2017 at 12:47 AM, John Coviciwrote: > > On Fri, 28 Jul 2017 22:01:24 -0400, > > Rich Freeman wrote: > >> > >> On Fri, Jul 28, 2017 at 9:47 PM, John Covici wrote: > >> > On Fri, 28 Jul 2017 21:13:24 -0400, > >> > > >> > I wonder if its because I am still using rsync to sync the portage > >> > directory? There are no changelogs anywhere! or nothing by that name. > >> > > >> > >> Ah, looks like they were removed entirely from rsync. It was months > >> ago and I don't use rsync so I'd half forgotten what the outcome was. > >> > >> There is apparently an rsync repository that only contains Changelogs > >> if you want them, but honestly it is probably easier to just check git > >> logs. If you run git whatchanged path you'll get the equivalent of > >> the Changelog for that path. > > > > If I do that I get the following: fatal: Not a git repository (or any > > parent up to mount point /usr) > > Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not > > set). > > > > This only works in a git repository. > > cd /var/tmp > git clone https://github.com/gentoo-mirror/gentoo.git > cd gentoo/cat/package > git whatchanged . > > You can also just browse the history online: > https://github.com/gentoo-mirror/gentoo > > Browse to the desired directory, and then hit the history button. OK, I changed to a git repository and did a git whatchanged, but in the directory I was looking at namely sys-kernel/dracut, I just got a generated commit from about the time I created the repository and none of the previous history. Maybe from now on I will get something better. Thanks again. -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] what about dracut and systemd?
On Sat, Jul 29, 2017 at 12:47 AM, John Coviciwrote: > On Fri, 28 Jul 2017 22:01:24 -0400, > Rich Freeman wrote: >> >> On Fri, Jul 28, 2017 at 9:47 PM, John Covici wrote: >> > On Fri, 28 Jul 2017 21:13:24 -0400, >> > >> > I wonder if its because I am still using rsync to sync the portage >> > directory? There are no changelogs anywhere! or nothing by that name. >> > >> >> Ah, looks like they were removed entirely from rsync. It was months >> ago and I don't use rsync so I'd half forgotten what the outcome was. >> >> There is apparently an rsync repository that only contains Changelogs >> if you want them, but honestly it is probably easier to just check git >> logs. If you run git whatchanged path you'll get the equivalent of >> the Changelog for that path. > > If I do that I get the following: fatal: Not a git repository (or any > parent up to mount point /usr) > Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not > set). > This only works in a git repository. cd /var/tmp git clone https://github.com/gentoo-mirror/gentoo.git cd gentoo/cat/package git whatchanged . You can also just browse the history online: https://github.com/gentoo-mirror/gentoo Browse to the desired directory, and then hit the history button. -- Rich
Re: [gentoo-user] what about dracut and systemd?
On Fri, 28 Jul 2017 22:01:24 -0400, Rich Freeman wrote: > > On Fri, Jul 28, 2017 at 9:47 PM, John Coviciwrote: > > On Fri, 28 Jul 2017 21:13:24 -0400, > > > > I wonder if its because I am still using rsync to sync the portage > > directory? There are no changelogs anywhere! or nothing by that name. > > > > Ah, looks like they were removed entirely from rsync. It was months > ago and I don't use rsync so I'd half forgotten what the outcome was. > > There is apparently an rsync repository that only contains Changelogs > if you want them, but honestly it is probably easier to just check git > logs. If you run git whatchanged path you'll get the equivalent of > the Changelog for that path. If I do that I get the following: fatal: Not a git repository (or any parent up to mount point /usr) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] what about dracut and systemd?
On Fri, Jul 28, 2017 at 9:47 PM, John Coviciwrote: > On Fri, 28 Jul 2017 21:13:24 -0400, > > I wonder if its because I am still using rsync to sync the portage > directory? There are no changelogs anywhere! or nothing by that name. > Ah, looks like they were removed entirely from rsync. It was months ago and I don't use rsync so I'd half forgotten what the outcome was. There is apparently an rsync repository that only contains Changelogs if you want them, but honestly it is probably easier to just check git logs. If you run git whatchanged path you'll get the equivalent of the Changelog for that path. -- Rich
Re: [gentoo-user] what about dracut and systemd?
On Fri, 28 Jul 2017 21:13:24 -0400, Rich Freeman wrote: > > On Fri, Jul 28, 2017 at 9:04 PM, John Coviciwrote: > > On Fri, 28 Jul 2017 21:00:50 -0400, > > Rich Freeman wrote: > >> > >> On Fri, Jul 28, 2017 at 8:52 PM, John Covici wrote: > >> > I have not done a world update in a few weeks and I like to check > >> > things out first before doing this. I am seeing that the latest > >> > update of dracut has the -systemd as a mandatory flag. So, since I > >> > have to use systemd, does this mean I can no longer use dracut -- I > >> > have found dracut very nice and would be sad to give it up. > >> > > >> > Any ideas on what is happening here? > >> > >> I'm not running 045 but it looks to me that systemd is just supported > >> automatically if it is installed. The USE flag was removed, but > >> support was not. > > > > OK, thanks a lot, I wish they would have news items for such things, > > or at least a changelog which seems to be missing in that directory. > > > > I have no idea why you don't have a changelog - those are > system-generated from the git logs and maintainers have nothing to do > with them. > > The relevant commit message seems to be: > commit 90bb5611b5aa9279ff3b0d4f5c25240bf4b0f9db > Author: Mike Gilbert > Date: Sun Jul 2 19:06:08 2017 -0400 > > sys-kernel/dracut: bump to 045 > > - Clean up ebuild code in general > - Remove unnecessary version setting > - Simplify gentoo.conf config file > - Depend on >=sys-apps/kmod-15 for libkmod > - Stop removing systemd and other modules > - Drop systemd USE flag > - Depend on sys-apps/coreutils[xattr] (bug 613516) > > Bug: https://bugs.gentoo.org/615898 > Bug: https://bugs.gentoo.org/613516 > Package-Manager: Portage-2.3.6_p9, Repoman-2.3.2_p77 > > :100644 100644 6525882cc17... 04a511a44f1... M sys-kernel/dracut/Manifest > :00 100644 000... 86fddcc57bb... A > sys-kernel/dracut/dracut-045.ebuild > I wonder if its because I am still using rsync to sync the portage directory? There are no changelogs anywhere! or nothing by that name. -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] what about dracut and systemd?
On Fri, Jul 28, 2017 at 9:04 PM, John Coviciwrote: > On Fri, 28 Jul 2017 21:00:50 -0400, > Rich Freeman wrote: >> >> On Fri, Jul 28, 2017 at 8:52 PM, John Covici wrote: >> > I have not done a world update in a few weeks and I like to check >> > things out first before doing this. I am seeing that the latest >> > update of dracut has the -systemd as a mandatory flag. So, since I >> > have to use systemd, does this mean I can no longer use dracut -- I >> > have found dracut very nice and would be sad to give it up. >> > >> > Any ideas on what is happening here? >> >> I'm not running 045 but it looks to me that systemd is just supported >> automatically if it is installed. The USE flag was removed, but >> support was not. > > OK, thanks a lot, I wish they would have news items for such things, > or at least a changelog which seems to be missing in that directory. > I have no idea why you don't have a changelog - those are system-generated from the git logs and maintainers have nothing to do with them. The relevant commit message seems to be: commit 90bb5611b5aa9279ff3b0d4f5c25240bf4b0f9db Author: Mike Gilbert Date: Sun Jul 2 19:06:08 2017 -0400 sys-kernel/dracut: bump to 045 - Clean up ebuild code in general - Remove unnecessary version setting - Simplify gentoo.conf config file - Depend on >=sys-apps/kmod-15 for libkmod - Stop removing systemd and other modules - Drop systemd USE flag - Depend on sys-apps/coreutils[xattr] (bug 613516) Bug: https://bugs.gentoo.org/615898 Bug: https://bugs.gentoo.org/613516 Package-Manager: Portage-2.3.6_p9, Repoman-2.3.2_p77 :100644 100644 6525882cc17... 04a511a44f1... M sys-kernel/dracut/Manifest :00 100644 000... 86fddcc57bb... A sys-kernel/dracut/dracut-045.ebuild -- Rich
Re: [gentoo-user] what about dracut and systemd?
On Fri, 28 Jul 2017 21:00:50 -0400, Rich Freeman wrote: > > On Fri, Jul 28, 2017 at 8:52 PM, John Coviciwrote: > > I have not done a world update in a few weeks and I like to check > > things out first before doing this. I am seeing that the latest > > update of dracut has the -systemd as a mandatory flag. So, since I > > have to use systemd, does this mean I can no longer use dracut -- I > > have found dracut very nice and would be sad to give it up. > > > > Any ideas on what is happening here? > > I'm not running 045 but it looks to me that systemd is just supported > automatically if it is installed. The USE flag was removed, but > support was not. OK, thanks a lot, I wish they would have news items for such things, or at least a changelog which seems to be missing in that directory. -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] what about dracut and systemd?
On Fri, Jul 28, 2017 at 8:52 PM, John Coviciwrote: > I have not done a world update in a few weeks and I like to check > things out first before doing this. I am seeing that the latest > update of dracut has the -systemd as a mandatory flag. So, since I > have to use systemd, does this mean I can no longer use dracut -- I > have found dracut very nice and would be sad to give it up. > > Any ideas on what is happening here? I'm not running 045 but it looks to me that systemd is just supported automatically if it is installed. The USE flag was removed, but support was not. -- Rich