Re: Git for backup storage
On Fri, Oct 06, 2023 at 01:44:34PM -0700, Mike Castle wrote: > Something I played with recently was > https://packages.debian.org/stable/vcs/git-filter-repo Yes, it does work. My typical use case is when someone has put a password in the repo you don't even want to have in the history. But you aren't going to use that in a Git backup with gigabytes of data, believe me :-) > But you definitely want to run tests on real data before you decide > that deleting old data saves your anything, particularly with respect > to time. > > If git is so efficient at storing this kind of data, then what do you > expect to gain by deleting old stuff, outside of a smaller log to go > through? The backup idea is a good one for medium amounts of smallish files (/etc comes to mind). Once big hunks like videos are involved, things get sluggish. Try doing "time sha1sum foo" where foo is an 1.2G video file to see what I mean. Cheers -- t signature.asc Description: PGP signature
Re: Web functionality;
On Fri, Oct 06, 2023 at 01:27:14PM -0700, pe...@easthope.ca wrote: > From: > Date: Sun, 8 Jan 2023 17:39:20 +0100 > > I do disagree with much of what the Mozilla foundation does, and at > > the end, they see the world through ad-industry coloured goggles, but > > they are the last credible ditch we have. > > Considering how minimal Dillo is, it has worthwhile capability. I > wonder why addition of JavaScript isn't mentioned. I don't mean > compete with Firefox. Just display fewer blank windows and allow some > commonly occuring gizmos to work. Heh. I used Dillo long,long time ago. That time whence this whining of some Web pages comes from (some still have it!): "You don't have Javascript? Oh, you poor sod! Here, we explain to you how to switch it on. That's how nice we are!" > > You thought the situation with Microsoft and computing in the 1980s > > and 1990s was grotesque? It's much, much worse these days. The > > difference is that the monopoly watchdogs are fast asleep at the > > wheel these days. > > For sure ... > > ... but even dinosaurs weren't invincible. That's my hope, yes. A faint one. > In many areas, Europe is more progressive than N. Am. Any encouraging > developments there? Yes, there's GDPR, which went in the right direction. The giants of surveillance capitalism are making a mock of it, because they are massively better at that game. And, to be fair. the book on that was written by a person in the US: Shoshana Zuboff: "The Age of Surveillance Capitalism". I'm buying a second copy because I gave away my first one. Cheers -- t signature.asc Description: PGP signature
Re: Does the debian kernel sends the gratuitous arp ?
On Fri, Oct 06, 2023 at 05:20:22PM -0400, Jeffrey Walton wrote: > On Fri, Oct 6, 2023 at 2:04 PM Balaji G wrote: > > > > Hi, > > I am using "Debian GNU/Linux 11 (bullseye)" with kernel version 5.16.12. > > When i do a link up/down i don't see any Gratuitous ARP being sent. > > > > # echo 1 > /proc/sys/net/ipv4/conf/eno5np0/arp_notify > > # ip link set down dev eno5np0 > > # ip link set up dev eno5np0 > > > > Captured all the packets via tcpdump & the tcpdump is not showing any > > Gratuitous ARP packets. > > > > But, with the same commands i could see the Gratuitous ARP being sent in > > Red hat.9.0 (Plow). > > So, please let me know if this is a specific scenario in Debian 11 ?? > > I think that's now Poettering: > https://github.com/systemd/systemd/blob/main/src/libsystemd-network/sd-ipv4acd.c#L302 As Geert says, probably it needs an ip address to be able to send an ARP. Perhaps the Redhat box has one set by default? What does "ip addr show dev " say? Cheers -- t signature.asc Description: PGP signature
Re: usrmerge in bookworm
Hello, On Fri, Oct 06, 2023 at 04:42:56PM +0200, Urs Thuermann wrote: > Greg Wooledge writes: > > > Yeah, usrmerge is a bit wonky in these early stages. > > $ apt-get changelog usrmerge | tail -n2 > -- Marco d'Itri Tue, 04 Nov 2014 22:42:44 +0100 > Fetched 11.0 kB in 0s (58.9 kB/s) > > Not what I'd call 'early' stages. It is for Debian, when the maintainer of dpkg is strongly opposed to, and refuses to co-operate with, anything related to usrmerge. It got done years ago in Ubuntu, and their dpkg doesn't have this issue, as they've carried patches for that for all those years. Though Debian has a much more diverse set of packages and user base and real issues HAVE been found in usrmerge so it's not a no-brainer, it's just that we would have gone through it and out the other side by now. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting
Re: Web functionality;
From: Date: Sun, 8 Jan 2023 17:39:20 +0100 > I do disagree with much of what the Mozilla foundation does, and at > the end, they see the world through ad-industry coloured goggles, but > they are the last credible ditch we have. Considering how minimal Dillo is, it has worthwhile capability. I wonder why addition of JavaScript isn't mentioned. I don't mean compete with Firefox. Just display fewer blank windows and allow some commonly occuring gizmos to work. > You thought the situation with Microsoft and computing in the 1980s > and 1990s was grotesque? It's much, much worse these days. The > difference is that the monopoly watchdogs are fast asleep at the > wheel these days. For sure ... ... but even dinosaurs weren't invincible. In many areas, Europe is more progressive than N. Am. Any encouraging developments there? Thx, ... P. - VoIP: +1 604 670 0140 work: https://en.wikibooks.org/wiki/User:PeterEasthope
Re: Does the debian kernel sends the gratuitous arp ?
On Fri, Oct 6, 2023 at 2:04 PM Balaji G wrote: > > Hi, > I am using "Debian GNU/Linux 11 (bullseye)" with kernel version 5.16.12. > When i do a link up/down i don't see any Gratuitous ARP being sent. > > # echo 1 > /proc/sys/net/ipv4/conf/eno5np0/arp_notify > # ip link set down dev eno5np0 > # ip link set up dev eno5np0 > > Captured all the packets via tcpdump & the tcpdump is not showing any > Gratuitous ARP packets. > > But, with the same commands i could see the Gratuitous ARP being sent in Red > hat.9.0 (Plow). > So, please let me know if this is a specific scenario in Debian 11 ?? I think that's now Poettering: https://github.com/systemd/systemd/blob/main/src/libsystemd-network/sd-ipv4acd.c#L302 Jeff
Re: Git for backup storage
Something I played with recently was https://packages.debian.org/stable/vcs/git-filter-repo But you definitely want to run tests on real data before you decide that deleting old data saves your anything, particularly with respect to time. If git is so efficient at storing this kind of data, then what do you expect to gain by deleting old stuff, outside of a smaller log to go through? mrc
Re: Does the debian kernel sends the gratuitous arp ?
On Fri, Oct 06, 2023 at 05:52:16PM +0530, Balaji G wrote: > Hi, > I am using "Debian GNU/Linux 11 (bullseye)" with kernel version 5.16.12. > When i do a link up/down i don't see any Gratuitous ARP being sent. ARP > # echo 1 > /proc/sys/net/ipv4/conf/eno5np0/arp_notify Probably transmitted "I tried to enforce ARP notification" > # ip link set down dev eno5np0 > # ip link set up dev eno5np0 link > Captured all the packets via tcpdump & the tcpdump is not showing any > Gratuitous ARP packets. Why should it? (Yes, that is a serious question.) > But, with the same commands i could see the Gratuitous ARP being sent in > Red hat.9.0 (Plow). I think more factors have been change as kernel. > So, please let me know if this is a specific scenario in Debian 11 ?? I state that the OP, Original Poster, is the specific scenario. Yeah, that is a blunt statement. Now hear me out. Gratuitous ARP is "linking" MAC-address and IP-address. During `ip link set down dev eno5np0` and `ip link set up dev eno5np0` are NO IP-addresses involved. So there is no need for ARP. I do hope to see a follow-up message like: Re-did the test with a static IP-address on the interface and indeed see a gratuitous ARP or some thing like To reproduce the missing gratuitous ARP do ... Yeah, that might reveal more information about dev eno5np0. > Thanks, > Balaji Groeten Geert Stappers -- Silence is hard to parse
Re: Git for backup storage
>> `git gc` does delete the old data (if it's not reachable any more).
> And it is very expensive. My point exactly.
It's fairly expensive indeed, but it's usually an operation that is not
very time-sensitive: it can usually be delayed to a convenient time, and
you can run it infrequently and as a low-priority background task.
A good reason why you usually don't want to run it frequently, is that
due to the sharing ("deduplication"), there's usually not that much
garbage to collect.
[ IOW, often a thousand backups (of the same machine) don't take up
much more space than a single backup. ]
>> BTW, if you want to (ab)use a Git repository to do backups, you should
>> definitely look at `bup`.
> Thanks, it might be exactly what I am looking for.
Bup uses the same format as Git, but has its own implementation for most
operations because the performance of Git is tuned for a very different
use-case. With Bup it's common to have a repository that is much larger
than 100GB, whereas Git very rarely manages repositories of such size.
Stefan
Re: usrmerge in bookworm
Greg Wooledge writes: > Yeah, usrmerge is a bit wonky in these early stages. $ apt-get changelog usrmerge | tail -n2 -- Marco d'Itri Tue, 04 Nov 2014 22:42:44 +0100 Fetched 11.0 kB in 0s (58.9 kB/s) Not what I'd call 'early' stages. > Part of the reason for this is that it's not *mandatory*, not > really. That seems to have changed in Debian 12: $ aptitude show init-system-helpers | egrep ^Prio\|^Dep Priority: required Depends: usrmerge | usr-is-merged In Debian 11, init-system-helpers did not depend on usrmerge. > There are still Debian systems -- important ones, which run the > internal infrastructure of Debian itself -- that aren't using it. Are those running older versions of Debian than stable? urs
Re: usrmerge in bookworm
Marco writes: > Am 06.10.2023 schrieb Steve Keller : > > > I have always been sceptical about /usr merge, since all binaries now > > appear in two places, "type sh" in bash gives the strange looking > > /usr/bin/sh where all Uni*ers are strongly used to /bin/sh. But also > > things like the following don't work anymore: > > Historic reasons, they don't exist anymore on current systems. > > > $ dpkg -S $(type -p sh) > > dpkg-query: no path found matching pattern /usr/bin/sh > > That is because /bin is now a symlink to /usr/bin and the file isn't > provided in /usr/bin by the package. I assume Steve (seemingly multi Un*x user) is aware about the historic reasons and how and why /usr-merge is implemented using symlinks. > > And I don't see a comfortable way around this. > > One way would be to strip off /usr from the string to search. That, of course, doesn't really help $ type -Pa ls /usr/bin/ls /bin/ls $ dpkg -S $(PATH=/bin type -Pa ls) coreutils: /bin/ls $ type -Pa gcc /usr/bin/gcc /bin/gcc $ dpkg -S $(PATH=/bin type -Pa gcc) dpkg-query: no path found matching pattern /bin/gcc because some packages have their binaries in /bin and others in /usr/bin, and likewise with /sbin vs. /usr/sbin, and /lib vs. /usr/lib. But the -a option to bash's 'type' builtin command helps: $ dpkg -S $(type -Pa ls gcc) dpkg-query: no path found matching pattern /usr/bin/ls coreutils: /bin/ls gcc: /usr/bin/gcc dpkg-query: no path found matching pattern /bin/gcc urs
Re: Git for backup storage
Stefan Monnier (12023-10-06): > `git gc` does delete the old data (if it's not reachable any more). And it is very expensive. My point exactly. > BTW, if you want to (ab)use a Git repository to do backups, you should > definitely look at `bup`. Thanks, it might be exactly what I am looking for. Regards, -- Nicolas George
Re: Git for backup storage
> Have you tried? The very principle of Git makes it necessary, to remove > or update old data, to rewrite the whole subsequent history. > Furthermore, it is done by creating a new branch, the original data is > not actually deleted. `git gc` does delete the old data (if it's not reachable any more). BTW, if you want to (ab)use a Git repository to do backups, you should definitely look at `bup`. Stefan
Re: Git for backup storage
john doe (12023-10-06): > Please elaborate on why Git is so bad at removing data from a single > repository? Have you tried? The very principle of Git makes it necessary, to remove or update old data, to rewrite the whole subsequent history. Furthermore, it is done by creating a new branch, the original data is not actually deleted. I asked a question and somehow it has become my responsibility to explain things… Regards, -- Nicolas George
Does the debian kernel sends the gratuitous arp ?
Hi, I am using "Debian GNU/Linux 11 (bullseye)" with kernel version 5.16.12. When i do a link up/down i don't see any Gratuitous ARP being sent. # echo 1 > /proc/sys/net/ipv4/conf/eno5np0/arp_notify # ip link set down dev eno5np0 # ip link set up dev eno5np0 Captured all the packets via tcpdump & the tcpdump is not showing any Gratuitous ARP packets. But, with the same commands i could see the Gratuitous ARP being sent in Red hat.9.0 (Plow). So, please let me know if this is a specific scenario in Debian 11 ?? Thanks, Balaji
Re: Git for backup storage
On 10/6/23 13:26, Nicolas George wrote: john doe (12023-10-06): I do not understand why you would want multiple repos, to me this looks like this would fit the bill for a Git branching workflow. Please elaborate. How do you work around the fact that Git is terrible at removing data with a single repository? Please elaborate on why Git is so bad at removing data from a single repository? We clearly do not understand eachother! -- John Doe
Re: Git for backup storage
Max Nikulin (12023-10-06): > I have no idea if it is possible to do it in place, but "git clone" and "git > fetch" have the --depth option. So you can specify how many last commits you > would like to have in the cloned repository. Using "git rebase I know. They only allow to keep the last commits, not decimate them. > --interactive" it is possible to squash e.g. daily commits into weekly or > monthly ones. The drawback is that git rebase changes commit hashes. git rebase is too inefficient for that kind of use. Regards, -- Nicolas George
Re: Git for backup storage
john doe (12023-10-06): > I do not understand why you would want multiple repos, to me this looks > like this would fit the bill for a Git branching workflow. Please elaborate. How do you work around the fact that Git is terrible at removing data with a single repository? Regards, -- Nicolas George
Re: usrmerge in bookworm
On Fri, Oct 06, 2023 at 11:06:56AM +0200, Steve Keller wrote: > $ dpkg -S $(type -p sh) > dpkg-query: no path found matching pattern /usr/bin/sh > > And I don't see a comfortable way around this. Yeah, usrmerge is a bit wonky in these early stages. Part of the reason for this is that it's not *mandatory*, not really. There are still Debian systems -- important ones, which run the internal infrastructure of Debian itself -- that aren't using it. So, in order to continue to accomodate the non-merged systems, packages that provide certain well-known legacy pathnames like /bin/ls must continue to do so. They can't move to providing /usr/bin/ls because that may break things on non-merged systems. Corner cases like the one you showed are going to remain in place for many years.
Re: Bullseye wi-fi. I have to restart NetworkManager every 5 minutes
On 06/10/2023 16:23, Ottavio Caruso wrote: I have the latest kernel from Bullseye backports. I don't want to upgrade yet. You may try to boot from a live image and to check if up to date software stack behaves better without touching of installed OSes.
Re: Git for backup storage
On 10/6/23 11:14, Nicolas George wrote: Hi. There is a project I have that requires some scripting, but I am wondering if somebody already did something similar and there is a package that I can just apt-get install. The idea is to use Git to store backups of text files that change rather rarely or not a lot, because Git is very efficient at compressing very similar files in time sequences. That would be used for dumps of SQL databases for example, or for records of hashes of all the files on a system. Unfortunately, Git is very bad at removing old data, that makes a problem for rotating / decimating the oldest backups. To work around this, I am considering using several Git repositories with a spillover system: - The files are committed into a monthly repository, each repository being created on the fly for the first commit on the month. - Old monthly repositories can be deleted. - But before they are deleted, one commit each five days can be extracted and committed into a yearly repository. - And similarly, one commit per month can be committed into a decennial repository before old yearly repositories are removed. Of course the month / year / five days parameters can be tweaked. So, does anybody know of existing packages in Debian that could make my work easier? Thanks in advance. I do not understand why you would want multiple repos, to me this looks like this would fit the bill for a Git branching workflow. -- John Doe
Re: Bullseye wi-fi. I have to restart NetworkManager every 5 minutes
On 06/10/2023 16:23, Ottavio Caruso wrote: For some reason, every time I am connected to wifi, pages can't load after 5 minutes of activity and I have to restart NetworkManager to get them back. Does it help to just disable/enable WiFi in the NetworkManager applet? You may try to disable power management in kernel module parameters that is driver of your WiFi adapter.
Re: Git for backup storage
On 06/10/2023 16:14, Nicolas George wrote: Unfortunately, Git is very bad at removing old data I have no idea if it is possible to do it in place, but "git clone" and "git fetch" have the --depth option. So you can specify how many last commits you would like to have in the cloned repository. Using "git rebase --interactive" it is possible to squash e.g. daily commits into weekly or monthly ones. The drawback is that git rebase changes commit hashes.
Re: Bullseye wi-fi. I have to restart NetworkManager every 5 minutes
On Fri, Oct 06, 2023 at 09:23:55AM +, Ottavio Caruso wrote: > Hi, > > $ uname -a > Linux t440 6.0.0-0.deb11.6-amd64 #1 SMP PREEMPT_DYNAMIC Debian > 6.0.12-1~bpo11+1 (2022-12-19) x86_64 GNU/Linux > > $ cat /etc/debian_version > 11.2 > > First things first: bring your system up to date with bullseye and, ideally, lose the backports kernel or upgrade to bookworm. Bookworm is now just past that kernel version at 6.0.13 At that point, with everything current, we can revisit this :) All the very best, as ever, Andy > For some reason, every time I am connected to wifi, pages can't load after 5 > minutes of activity and I have to restart NetworkManager to get them back. > > $ ping 8.8.8.8 # nothing > > # dmesg -C # clears the buffer > > # systemctl restart NetworkManager > > $ ping 8.8.8.8 # it works > > $ dmesg > [ 2862.837299] wlp3s0: deauthenticating from 74:a0:2f:a8:35:be by local > choice (Reason: 3=DEAUTH_LEAVING) > [ 2863.508089] ACPI: \: failed to evaluate _DSM > bf0212f2-788f-c64d-a5b3-1f738e285ade (0x1001) > [ 2863.508098] ACPI: \: failed to evaluate _DSM > bf0212f2-788f-c64d-a5b3-1f738e285ade (0x1001) > [ 2863.508100] ACPI: \: failed to evaluate _DSM > bf0212f2-788f-c64d-a5b3-1f738e285ade (0x1001) > [ 2863.508102] ACPI: \: failed to evaluate _DSM > bf0212f2-788f-c64d-a5b3-1f738e285ade (0x1001) > [ 2863.508104] ACPI: \: failed to evaluate _DSM > bf0212f2-788f-c64d-a5b3-1f738e285ade (0x1001) > [ 2863.508106] ACPI: \: failed to evaluate _DSM > bf0212f2-788f-c64d-a5b3-1f738e285ade (0x1001) > [ 2863.508108] ACPI: \: failed to evaluate _DSM > bf0212f2-788f-c64d-a5b3-1f738e285ade (0x1001) > > > I googled the above and this looks like a known kernel bug affecting old > Thinkpads. > > I have the latest kernel from Bullseye backports. I don't want to upgrade > yet. > > Any clue? > > -- > Ottavio Caruso > > A: Because it messes up the order in which people normally read text. > Q: Why is top-posting such a bad thing? > A: Top-posting. > Q: What is the most annoying thing in e-mail? >
Re: usrmerge in bookworm
Am 06.10.2023 schrieb Steve Keller : > I have upgraded from bullseye to bookworm and it seems the package > usrmerge is installed forcedly now. At least it has been installed > and I haven't been asked about it :-( > > I have always been sceptical about /usr merge, since all binaries now > appear in two places, "type sh" in bash gives the strange looking > /usr/bin/sh where all Uni*ers are strongly used to /bin/sh. But also > things like the following don't work anymore: Historic reasons, they don't exist anymore on current systems. > $ dpkg -S $(type -p sh) > dpkg-query: no path found matching pattern /usr/bin/sh That is because /bin is now a symlink to /usr/bin and the file isn't provided in /usr/bin by the package. > And I don't see a comfortable way around this. One way would be to strip off /usr from the string to search.
usrmerge in bookworm
I have upgraded from bullseye to bookworm and it seems the package usrmerge is installed forcedly now. At least it has been installed and I haven't been asked about it :-( I have always been sceptical about /usr merge, since all binaries now appear in two places, "type sh" in bash gives the strange looking /usr/bin/sh where all Uni*ers are strongly used to /bin/sh. But also things like the following don't work anymore: $ dpkg -S $(type -p sh) dpkg-query: no path found matching pattern /usr/bin/sh And I don't see a comfortable way around this. Steve
Git for backup storage
Hi. There is a project I have that requires some scripting, but I am wondering if somebody already did something similar and there is a package that I can just apt-get install. The idea is to use Git to store backups of text files that change rather rarely or not a lot, because Git is very efficient at compressing very similar files in time sequences. That would be used for dumps of SQL databases for example, or for records of hashes of all the files on a system. Unfortunately, Git is very bad at removing old data, that makes a problem for rotating / decimating the oldest backups. To work around this, I am considering using several Git repositories with a spillover system: - The files are committed into a monthly repository, each repository being created on the fly for the first commit on the month. - Old monthly repositories can be deleted. - But before they are deleted, one commit each five days can be extracted and committed into a yearly repository. - And similarly, one commit per month can be committed into a decennial repository before old yearly repositories are removed. Of course the month / year / five days parameters can be tweaked. So, does anybody know of existing packages in Debian that could make my work easier? Thanks in advance. -- Nicolas George
