Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On Mon 17 May 2021 at 14:39:47 -0400, Greg Wooledge wrote: > On Mon, May 17, 2021 at 07:25:38PM +0100, Brian wrote: > > On Mon 17 May 2021 at 11:01:33 -0400, Greg Wooledge wrote: > > > > [...] > > > > > Done! Now, let's try that with pinfo date. I ran pinfo date from my > > > shell, which took me to one of the pages within the tree of coreutils > > > texinfo documentation corresponding to the date program. This particular > > > page is titled "21.1 ‘date’: Print or set system date and time". > > > > > > I pressed /, typed %F, pressed Enter, and I got: > > > > > > "Search string not found..." > > > > > > That's because pinfo doesn't search beyond the current page, and the > > > current page is just a menu of links to other pages. > > > > As a longtime user of pinfo I appreciate your exposing it to a wider > > audience. > > > > Regarding seaching: does the s key do anything to cause you to modify > > your observation? > > Huh... that's interesting. OK, these key bindings are *really* not ideal, > and I think I found some kind of bug. Ideal or not, I hope you are acknowledging that pinfo can search beyond the current page. Press s, type %F, press Enter; I got a description of the option at the top of the page. -- Brian.
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On Mon, May 17, 2021 at 07:25:38PM +0100, Brian wrote: > On Mon 17 May 2021 at 11:01:33 -0400, Greg Wooledge wrote: > > [...] > > > Done! Now, let's try that with pinfo date. I ran pinfo date from my > > shell, which took me to one of the pages within the tree of coreutils > > texinfo documentation corresponding to the date program. This particular > > page is titled "21.1 ‘date’: Print or set system date and time". > > > > I pressed /, typed %F, pressed Enter, and I got: > > > > "Search string not found..." > > > > That's because pinfo doesn't search beyond the current page, and the > > current page is just a menu of links to other pages. > > As a longtime user of pinfo I appreciate your exposing it to a wider > audience. > > Regarding seaching: does the s key do anything to cause you to modify > your observation? Huh... that's interesting. OK, these key bindings are *really* not ideal, and I think I found some kind of bug. According to the man page, there's no default key binding for "search again" (KEY_SEARCH_AGAIN_1), but in Debian's /etc/pinforc, it's bound to 'f'. OK, armed with that knowledge, I did "pinfo date", then "s" and "%F" to search for the same word as before. Then I pressed "f" which advanced me to the next instance. Then "f" a second time, and it says: Tag table is corrupt, trying to fix... (press a key to continue) Pressing space takes me back to the main page of coreutils.info. I'm just gonna go ahead and report the bug. And thanks for the tip.
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On Mon 17 May 2021 at 11:01:33 -0400, Greg Wooledge wrote: [...] > Done! Now, let's try that with pinfo date. I ran pinfo date from my > shell, which took me to one of the pages within the tree of coreutils > texinfo documentation corresponding to the date program. This particular > page is titled "21.1 ‘date’: Print or set system date and time". > > I pressed /, typed %F, pressed Enter, and I got: > > "Search string not found..." > > That's because pinfo doesn't search beyond the current page, and the > current page is just a menu of links to other pages. As a longtime user of pinfo I appreciate your exposing it to a wider audience. Regarding seaching: does the s key do anything to cause you to modify your observation? -- Brian.
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
Hi, Greg Wooledge wrote: > That's in the info(1) tool. I agree, info has a better search ability > than pinfo(1). Oops. I did not make the connection from your final statement to your mentioning of pinfo. (I could make excuses that you mention "info and pinfo" on the way to the end. But actually i did not make a connection to that either.) > Let's take date as an example, and let's say that we want to know what > the %F does, because we saw someone use it. > In the man page, which has everything all together, it's simple: > unicorn:~$ man date | grep %F >%F full date; like %+4Y-%m-%d At least in this case, pinfo might not be a sincere contender. I read that it aims to be like the web browser "lynx". info is more rugged in respect to traditional shell use. As you already demonstrated it would work fine with grep. Paradoxly, i would blame the highly condensed information in "man date" on the fact that man pages in many GNU packages are more or less formatted versions of the program's help text: $ date --help | grep %F %F full date; same as %Y-%m-%d > unicorn:~$ info date | less Don't tell this Richard Stallman. :)) Whatever, neither info document nor man page really serve the needs of users who want to learn how to use a program. At best they can give an introduction of concepts and list all invocation details. This serves those who already have a good idea of what they are looking for. Examples can help. When i feel clueless then i ask the all-knowing seer g*.com in plain english. It yields lots of rumors but in most cases it gives me better ideas what to look for. (Please don't tell this Richard Stallman either ...) Have a nice day :) Thomas
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On Mon, May 17, 2021 at 04:09:37PM +0200, Thomas Schmitt wrote: > Hi, > > Greg Wooledge wrote: > > the inability to *search* within the > > info page to find occurrences of your keyword can be maddening. > > It's not _that_ terrible. Pressing in > info dd > the "/" key, i get a prompt > Regexp search []: > > The input "dsync" brings me to the ‘dsync’ explanation. > Pressing "/" again and then the Enter key, brings me to its line in the > Concept index. Doing it again brings me back to the first found occurence. That's in the info(1) tool. I agree, info has a better search ability than pinfo(1). Also, dd isn't the best choice to demonstrate the issue, because it's basically a single page anyway. Let's take date as an example, and let's say that we want to know what the %F does, because we saw someone use it. In the man page, which has everything all together, it's simple: unicorn:~$ man date | grep %F %F full date; like %+4Y-%m-%d Done! Now, let's try that with pinfo date. I ran pinfo date from my shell, which took me to one of the pages within the tree of coreutils texinfo documentation corresponding to the date program. This particular page is titled "21.1 ‘date’: Print or set system date and time". I pressed /, typed %F, pressed Enter, and I got: "Search string not found..." That's because pinfo doesn't search beyond the current page, and the current page is just a menu of links to other pages. Now, to be fair, it's a different story in info(1). I installed the info package, and ran "info date" from my shell, which took me to the same page (21.1). I pressed /%FEnter and this time, it jumped me to the description of %F on a different page (21.1.2). The only problem is now I'm in the info(1) tool, which sucks. ;-) Another approach is to attempt the search from the shell: unicorn:~$ info date | grep %F ‘%F’ ‘%F’, ‘%G’, and ‘%Y’ (all without modifiers), and requires a flag to be date -d @946684800 +"%F %T %z" This is working as intended, but the issue is that the info page doesn't have a concise one-line description of the %F format. I could use something like "grep -A10 %F" and hope. Another workaround that I've found, which doesn't *always* work, but works a good deal of the time, is: unicorn:~$ info date | less This dumps a flattened version of several of the texinfo pages into a regular pager (less, in this case). From here, I can use less's search and navigation features, which are *far* easier for me. In this particular example, searching for %F within less takes me directly to the desired section, and the 5-line description is prominently visible near the top of the terminal. Of course, doing this loses the ability to navigate the texinfo page tree by following hyperlinks, but for quick searches that's a totally reasonable trade-off.
Re: OT: minimum bs for dd?
On Mon 17 May 2021 at 08:59:43 +0300, Andrei POPESCU wrote: [...] > I'll raise you 'cp': > > cp foo.iso /dev/sdb > > > which is both fast and short to type (apparently it's smart about using > the correct block size). > > Unfortunately it's missing dd's equivalent of status=progress. As an aside: cat /dev/sdb was originally recommended in the Installation Guide but was replaced with cp foo.iso /dev/sdb because the cat command gives problems when used with sudo. -- Brian.
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
Hi, Greg Wooledge wrote: > the inability to *search* within the > info page to find occurrences of your keyword can be maddening. It's not _that_ terrible. Pressing in info dd the "/" key, i get a prompt Regexp search []: The input "dsync" brings me to the ‘dsync’ explanation. Pressing "/" again and then the Enter key, brings me to its line in the Concept index. Doing it again brings me back to the first found occurence. Said that, i agree that the form of man pages is more comfortable to me than the form of info. As maintainer of GNU xorriso i am obliged to provide an info document, which must not contain less information than the man page. My solution is to insert enough @c comments with man page entrails into the .texi file which is the source of xorriso.info https://dev.lovelyhq.com/libburnia/libisoburn/raw/branch/master/xorriso/xorriso.texi to be able to derive the man page https://dev.lovelyhq.com/libburnia/libisoburn/raw/branch/master/xorriso/xorriso.1 by help of a little C program https://dev.lovelyhq.com/libburnia/libisoburn/raw/branch/master/xorriso/make_xorriso_1.c The GNU utiliy makeinfo ignores the comments and generates the .info document from xorriso.texi https://dev.lovelyhq.com/libburnia/libisoburn/raw/branch/master/xorriso/xorriso.info (My browser refuses to show it but only offers for download.) The Debian package of xorriso installs xorriso.1.gz and xorriso.info.gz alike. Both give the same information. I look up the man page when i need to remember what i programmed ten years ago. Have a nice day :) Thomas
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On Mon, May 17, 2021 at 09:38:49AM -0400, Greg Wooledge wrote: [...] > Now, as for the info pages themselves: unlike traditional man pages, > where all of the documentation is on one page, in which you can scroll > up and down and search, info pages are chopped up into tiny little > sections, and you only see one section at a time [...] To be fair, info has links going across "pages" and you can search across several pages. Of course, tool usage is, above all else, habit. Changing from man to info is almost as painful as the other way around. There goes a missed chance for (more than one!) unified user interfaces, alas. Cheers - t signature.asc Description: Digital signature
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On 05/17/2021 08:21 AM, IL Ka wrote: I got the impression from the search hits I got info pages were available on the web complete with useful hyperlinks. In a terminal "info dd" gives an annoying blob of text. Due to visual limits I *MUCH* prefer HTML for large amounts of information. All gnu info pages are available online: https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html#dd-invocation Along with debian man pages: https://manpages.debian.org/buster/coreutils/dd.1.en.html https://www.gnu.org/software/coreutils/manual/html_node/Concept-index.html#Concept-index is now next to https://manpages.debian.org/ in my bookmarks folder. Thanks.
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On Mon, May 17, 2021 at 03:03:22PM +0200, to...@tuxteam.de wrote: > Try typing in a terminal `info dd' and see what happens :) unicorn:~$ info dd bash: info: command not found ;-) GNU's info pages are a highly debated thing, in some circles. Many people despise them, some people love them, and the vast majority are simply confused. The primary issue is the user interface for actually reading the damned things. You're supposed to be using emacs, and you're supposed to hit some bizarre hand-mutilating key chord to invoke the info page reader inside of emacs. If you're *not* an emacs user, then there's a standalone tool called "info" that they provide which is supposed to emulate the experience for you, somewhat. The problem here is that if you're not an emacs user, then the experience is *completely alien* and makes no sense at all. There's an alternative tool called "pinfo" that provides a different interface for reading info pages, which is a bit more comfortable for non-emacs users. It resembles lynx. But it has limitations and does not reproduce all of the features of the official tool. So, feel free to try both "info" and "pinfo" (some installation required) and see which one you can tolerate better. Or use the online web version of the pages, if that works better for you. Now, as for the info pages themselves: unlike traditional man pages, where all of the documentation is on one page, in which you can scroll up and down and search, info pages are chopped up into tiny little sections, and you only see one section at a time. This may be an advantage if you're actually looking for a specific feature that you suspect may exist, but you don't know what it's *called*. Navigating the hierarchical structure may lead you to a place that describes the feature you're looking for, although you'll have to do a lot of reading along the way. On the other hand, if you *do* know what something is called, or at least you have a few reasonable guesses, the inability to *search* within the info page to find occurrences of your keyword can be maddening.
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
> I got the impression from the search hits I got info pages were > available on the web complete with useful hyperlinks. > > In a terminal "info dd" gives an annoying blob of text. Due to visual > limits I *MUCH* prefer HTML for large amounts of information. > All gnu info pages are available online: https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html#dd-invocation Along with debian man pages: https://manpages.debian.org/buster/coreutils/dd.1.en.html
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On 05/17/2021 08:03 AM, to...@tuxteam.de wrote:
On Mon, May 17, 2021 at 07:56:45AM -0500, Richard Owlett wrote:
On 05/17/2021 03:00 AM, to...@tuxteam.de wrote:
... (this is from the info page, which sometimes is more complete than the man
page):
My web search turned up only compliments for info pages.
NOTHING on where to find :{
Help please.
If we are talking about `dd', this is part of coreutils, whose info file
comes as one big package, here:
/usr/share/info/coreutils.info.gz
...which is part of the coreutils Debian package. So it got installed
alongside your `dd'.
Try typing in a terminal `info dd' and see what happens :)
Cheers
- t
I got the impression from the search hits I got info pages were
available on the web complete with useful hyperlinks.
In a terminal "info dd" gives an annoying blob of text. Due to visual
limits I *MUCH* prefer HTML for large amounts of information.
Re: info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On Mon, May 17, 2021 at 07:56:45AM -0500, Richard Owlett wrote:
> On 05/17/2021 03:00 AM, to...@tuxteam.de wrote:
> >... (this is from the info page, which sometimes is more complete than the
> >man page):
> >
>
> My web search turned up only compliments for info pages.
> NOTHING on where to find :{
> Help please.
If we are talking about `dd', this is part of coreutils, whose info file
comes as one big package, here:
/usr/share/info/coreutils.info.gz
...which is part of the coreutils Debian package. So it got installed
alongside your `dd'.
Try typing in a terminal `info dd' and see what happens :)
Cheers
- t
signature.asc
Description: Digital signature
Re: OT: minimum bs for dd?
On Mon, May 17, 2021 at 03:21:20PM +0300, Andrei POPESCU wrote: [...] > If my understanding from your quotes and David's links is correct > oflag=sync may be slower in specific circumstances, but it depends on so > many factors (hardware, caches, block size used, etc.) that it is hard > to predict. The proof would be in the pudding, of course. > Quite likely it won't make a significant difference for "regular" use. For regular use, the oflag is (as I tried to explain) invaluable to me. See, I bought myself a refurbished notebook, but I didn't skimp on RAM: 16GB. My main dd use is to write some image onto a flash removable (stick, SD card). The write channel is slow, the available buffer big. Once the write is done (if I do cp or forget the sync), not much is actually written to the drive. Then I say "sync", and... the sync goes shopping, without telling me when it plans to be back :-) With "oflags=sync status=progress" (or by sending SIGHUP to the dd process) I can get a rough idea when I'll be able to pull out that stick. Needless to say, I don't mind the copy being some, say, 10% slower (totally making that up, but for a seq writing with some biggish block size, I'd expect even that to be an outrageous exaggeration). Cheers - t signature.asc Description: Digital signature
info pages WHERE? -- was [Re: OT: minimum bs for dd?]
On 05/17/2021 03:00 AM, to...@tuxteam.de wrote:
... (this is from the info page, which sometimes is more complete than the man
page):
My web search turned up only compliments for info pages.
NOTHING on where to find :{
Help please.
Re: OT: minimum bs for dd?
On Lu, 17 mai 21, 10:00:22, to...@tuxteam.de wrote: > On Mon, May 17, 2021 at 10:29:06AM +0300, Andrei POPESCU wrote: > > > > Hmm, would you (or anyone else) know what is the difference between > > oflags=sync and conv=fsync? > > Let me put the docs next to each other (this is from the info page, > which sometimes is more complete than the man page): [snip informative quote from info page] Ugh, forgot about the info/man distinction fro GNU software. > I'd say "conv=fsync" does a sync at the end of the whole thing, while > "oflag=sync" open(2)s the file with O_SYNC (likewise "oflag=dsync" > translates to an open(2) with O_DSYNC), so the sync happens after each > block write. That's what I want, since it gives me much more predictability > as to when I can plop out my usb stick :) If my understanding from your quotes and David's links is correct oflag=sync may be slower in specific circumstances, but it depends on so many factors (hardware, caches, block size used, etc.) that it is hard to predict. Quite likely it won't make a significant difference for "regular" use. Kind regards, Andrei -- http://wiki.debian.org/FAQsFromDebianUser signature.asc Description: PGP signature
Re: OT: minimum bs for dd?
On Lu, 17 mai 21, 17:50:24, David wrote: > On Mon, 17 May 2021 at 17:29, Andrei POPESCU wrote: > > > Hmm, would you (or anyone else) know what is the difference between > > oflags=sync and conv=fsync? > > https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html > > https://unix.stackexchange.com/questions/508701/dd-command-oflag-direct-and-sync-flags > > https://abbbi.github.io/dd/ Thank you, I should have guessed there is answer on StackExchange. Kind regards, Andrei -- http://wiki.debian.org/FAQsFromDebianUser signature.asc Description: PGP signature
Re: OT: minimum bs for dd?
On 17/05/21 6:30 pm, to...@tuxteam.de wrote: This is one point. The other, which adds more convenience is that dd has an explicit argument for (input and) output file name, whereas cat relies on redirection. This becomes relevant when you try to sudo cat thing > that_other_thing and realise that the ">" is *not* in the sudo context (and what you would have to do to "fix" that). Instead, sudo dd if=thing of=this_other_thing Just Works out of the box. More relevant when doing ">>" (use dd's oflag=append then). And if you already have a long pipeline in your command history, that didn't work because of the above issue, you can use: ... |sudo dd of=this_other_thing Note: I'm not sure what the difference is between that_other_thing and this_other_thing :-) Richard
Re: OT: minimum bs for dd?
On Sun, May 16, 2021 at 06:43:54PM -0400, The Wanderer wrote: > > cat /dev/sdb > > > > dd /dev/sdb > Is there really no functional difference between the baseline trivial > functionalities of cat and dd? There are two differences: 1) dd is specified to use default input and output block sizes of 512 bytes. 2) dd is specified to write a summary to stderr upon completion. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html
Re: OT: minimum bs for dd?
On 2021-05-17 at 02:57, Thomas Schmitt wrote: > Hi, > > Andrew M.A. Cater wrote: > >>> When copying a file and writing it to another medium, perhaps eg >>> when writing a DVD .iso file directly to a USB stick, it's >>> ideal. > > The Wanderer wrote: > >> Is there really no functional difference between the baseline >> trivial functionalities of cat and dd? > > cat and cp cannot do the following: > > # Remove possible GPT backup header from the end of the USB stick > dd if=/dev/zero of=/dev/sdc bs=512 seek=3915775 count=1 status=none True, but that's not the baseline trivial functionality, which is what Stefan had given in the example I was responding to; it includes a lot of extra behavior-modifying flags. -- The Wanderer The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw signature.asc Description: OpenPGP digital signature
Re: OT: minimum bs for dd?
On Mon, May 17, 2021 at 10:29:06AM +0300, Andrei POPESCU wrote: > On Lu, 17 mai 21, 08:32:28, to...@tuxteam.de wrote: > > On Mon, May 17, 2021 at 08:59:43AM +0300, Andrei POPESCU wrote: > > > > [...] > > > > > I'll raise you 'cp': > > > > > > cp foo.iso /dev/sdb > > > > > > > > > which is both fast and short to type (apparently it's smart about using > > > the correct block size). > > > > I'll be sure to try that some day :) > > > > > Unfortunately it's missing dd's equivalent of status=progress. > > > > and oflags=sync. > > Hmm, would you (or anyone else) know what is the difference between > oflags=sync and conv=fsync? Let me put the docs next to each other (this is from the info page, which sometimes is more complete than the man page): * conv=fsync: ‘conv=CONVERSION[,CONVERSION]...’ Convert the file as specified by the CONVERSION argument(s). (No spaces around any comma(s).) [...] The following “conversions” are really file flags and don’t affect internal processing: [...] ‘fsync’ Synchronize output data and metadata just before finishing. This forces a physical write of output data and metadata. * oflag=sync: ‘oflag=FLAG[,FLAG]...’ Access the output file using the flags specified by the FLAG argument(s). (No spaces around any comma(s).) [...] ‘dsync’ Use synchronized I/O for data. For the output file, this forces a physical write of output data on each write. For the input file, this flag can matter when reading from a remote file that has been written to synchronously by some other process. Metadata (e.g., last-access and last-modified time) is not necessarily synchronized. ‘sync’ Use synchronized I/O for both data and metadata. I'd say "conv=fsync" does a sync at the end of the whole thing, while "oflag=sync" open(2)s the file with O_SYNC (likewise "oflag=dsync" translates to an open(2) with O_DSYNC), so the sync happens after each block write. That's what I want, since it gives me much more predictability as to when I can plop out my usb stick :) Cheers - t signature.asc Description: Digital signature
Re: OT: minimum bs for dd?
On Mon, 17 May 2021 at 17:29, Andrei POPESCU wrote: > Hmm, would you (or anyone else) know what is the difference between > oflags=sync and conv=fsync? https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html https://unix.stackexchange.com/questions/508701/dd-command-oflag-direct-and-sync-flags https://abbbi.github.io/dd/
Re: OT: minimum bs for dd?
On Lu, 17 mai 21, 08:32:28, to...@tuxteam.de wrote: > On Mon, May 17, 2021 at 08:59:43AM +0300, Andrei POPESCU wrote: > > [...] > > > I'll raise you 'cp': > > > > cp foo.iso /dev/sdb > > > > > > which is both fast and short to type (apparently it's smart about using > > the correct block size). > > I'll be sure to try that some day :) > > > Unfortunately it's missing dd's equivalent of status=progress. > > and oflags=sync. Hmm, would you (or anyone else) know what is the difference between oflags=sync and conv=fsync? Kind regards, Andrei -- http://wiki.debian.org/FAQsFromDebianUser signature.asc Description: PGP signature
Re: OT: minimum bs for dd?
Hi, Andrew M.A. Cater wrote: > > When copying a file and writing it to another medium, perhaps eg when > > writing a DVD .iso file directly to a USB stick, it's ideal. The Wanderer wrote: > Is there really no functional difference between the baseline trivial > functionalities of cat and dd? cat and cp cannot do the following: # Remove possible GPT backup header from the end of the USB stick dd if=/dev/zero of=/dev/sdc bs=512 seek=3915775 count=1 status=none # Copy the ISO with its built-in partition table to the USB stick. # Show realistic progress messages by curbing the i/o buffering. ISO=debian-10.6.0-amd64-netinst.iso dd if="$ISO" of=/dev/'sdc' bs=1M status=progress oflag=dsync With all copy methods one should do before unplugging the stick: sync It should be noted that dd has its special pitfalls. E.g. if the input file is not a data file or block device, then one should use iflag=fullblock because with some versions of dd there is the risk to get partially read blocks written as full output blocks. (I cannot reproduce it right now by (echo hello ; sleep 1 ; echo hello) | dd of=dd_target_file obs=512 but i had undesirable results in the past.) Whatever, the main risk with a short copy command performed by the superuser is to pick the wrong device file name and thus to overwrite a disk with valuable content. I made a cautious (or paranoid) helper script for the task of copying ISOs to USB sticks: https://wiki.debian.org/XorrisoDdTarget It came too late for the freeze of Debian 11 but is planned to be available as package in Debian 12. Up to then it can be downloaded from upstream and verified by its corresponding .sig file: gpg --verify xorriso-dd-target.sig xorriso-dd-target Have a nice day :) Thomas
Re: OT: minimum bs for dd?
On Mon, May 17, 2021 at 08:59:43AM +0300, Andrei POPESCU wrote: [...] > I'll raise you 'cp': > > cp foo.iso /dev/sdb > > > which is both fast and short to type (apparently it's smart about using > the correct block size). I'll be sure to try that some day :) > Unfortunately it's missing dd's equivalent of status=progress. and oflags=sync. Cheers - t signature.asc Description: Digital signature
Re: OT: minimum bs for dd?
On Sun, May 16, 2021 at 06:33:57PM -0400, Stefan Monnier wrote: > >> On Sun, May 16, 2021 at 01:31:49PM -0500, Richard Owlett wrote: > >> > I'll bite ;} > >> > When is it the right tool? > >> > >> When you're using it to convert ebcdic to ascii, while swapping bytes and > >> reblocking an ancient file from a barely readable archival tape. > >> > >> > When is it not? > >> > >> When copying a file. > >> > > When copying a file and writing it to another medium, perhaps eg when > > writing > > a DVD .iso file directly to a USB stick, it's ideal. > > Not sure about ideal: > > cat /dev/sdb > > is one char longer than > > dd /dev/sdb > > but it's often faster (you can speed up `dd` by providing a larger > `bs=` argument, but then you've lost the length advantage ;-) Agreed. When writing to USB flash media I've the impression that there's a sweet spot somewhere between 64K and 1M block size. The maximum is pretty flat, so I just use 1M (easier to remember), but it's significant wrt the default 512B. It possibly depends on the rest of the hardware. This is one point. The other, which adds more convenience is that dd has an explicit argument for (input and) output file name, whereas cat relies on redirection. This becomes relevant when you try to sudo cat thing > that_other_thing and realise that the ">" is *not* in the sudo context (and what you would have to do to "fix" that). Instead, sudo dd if=thing of=this_other_thing Just Works out of the box. More relevant when doing ">>" (use dd's oflag=append then). Another nice thing is oflag=sync: when writing to big media, it makes sure that data doesn't end up in the buffer cache making you wait during the final sync. And oh, 'status=progress' in combination with the above. So... yay, dd. Cheers - t signature.asc Description: Digital signature
Re: OT: minimum bs for dd?
On Du, 16 mai 21, 18:33:57, Stefan Monnier wrote: > >> On Sun, May 16, 2021 at 01:31:49PM -0500, Richard Owlett wrote: > >> > I'll bite ;} > >> > When is it the right tool? > >> > >> When you're using it to convert ebcdic to ascii, while swapping bytes and > >> reblocking an ancient file from a barely readable archival tape. > >> > >> > When is it not? > >> > >> When copying a file. > >> > > When copying a file and writing it to another medium, perhaps eg when > > writing > > a DVD .iso file directly to a USB stick, it's ideal. > > Not sure about ideal: > > cat /dev/sdb > > is one char longer than > > dd /dev/sdb > > but it's often faster (you can speed up `dd` by providing a larger > `bs=` argument, but then you've lost the length advantage ;-) I'll raise you 'cp': cp foo.iso /dev/sdb which is both fast and short to type (apparently it's smart about using the correct block size). Unfortunately it's missing dd's equivalent of status=progress. Kind regards, Andrei -- http://wiki.debian.org/FAQsFromDebianUser signature.asc Description: PGP signature
Re: OT: minimum bs for dd?
>> On Sun, May 16, 2021 at 01:31:49PM -0500, Richard Owlett wrote: >> > I'll bite ;} >> > When is it the right tool? >> >> When you're using it to convert ebcdic to ascii, while swapping bytes and >> reblocking an ancient file from a barely readable archival tape. >> >> > When is it not? >> >> When copying a file. >> > When copying a file and writing it to another medium, perhaps eg when writing > a DVD .iso file directly to a USB stick, it's ideal. Not sure about ideal: cat /dev/sdb is one char longer than dd /dev/sdb but it's often faster (you can speed up `dd` by providing a larger `bs=` argument, but then you've lost the length advantage ;-) Stefan
Re: OT: minimum bs for dd?
On 2021-05-16 at 18:33, Stefan Monnier wrote: >> When copying a file and writing it to another medium, perhaps eg when writing >> a DVD .iso file directly to a USB stick, it's ideal. > > Not sure about ideal: > > cat /dev/sdb > > is one char longer than > > dd /dev/sdb > > but it's often faster (you can speed up `dd` by providing a larger > `bs=` argument, but then you've lost the length advantage ;-) Is there really no functional difference between the baseline trivial functionalities of cat and dd? From quite a long time ago, I thought there were contexts in which writing a suitably-prepared data file directly to a block device using cat would produce incorrect results, but using dd would work out fine. I can't remember any of the potential specifics, though, so it's possible I'm mixing one or both of them up with something else... -- The Wanderer The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw signature.asc Description: OpenPGP digital signature
Re: OT: minimum bs for dd?
On Sun, May 16, 2021 at 02:45:12PM -0400, Michael Stone wrote: > On Sun, May 16, 2021 at 01:31:49PM -0500, Richard Owlett wrote: > > I'll bite ;} > > When is it the right tool? > > When you're using it to convert ebcdic to ascii, while swapping bytes and > reblocking an ancient file from a barely readable archival tape. > > > When is it not? > > When copying a file. > When copying a file and writing it to another medium, perhaps eg when writing a DVD .iso file directly to a USB stick, it's ideal. [Note: Debian's .iso files that are bootable have a hybrid format which means that you can write the bits directly to a USB stick and it will be bootable - other .iso files may vary.] Andy C.
Re: OT: minimum bs for dd?
On Sun, May 16, 2021 at 01:31:49PM -0500, Richard Owlett wrote: I'll bite ;} When is it the right tool? When you're using it to convert ebcdic to ascii, while swapping bytes and reblocking an ancient file from a barely readable archival tape. When is it not? When copying a file.
Re: OT: minimum bs for dd?
On 05/16/2021 12:42 PM, Michael Stone wrote: On Sat, May 15, 2021 at 11:32:31PM +0300, IL Ka wrote: As noted, is there a minimum bs size for dd? It seems that you can set bs as small as 1. 512 is the default because of HDD block size which used to be 512 bytes for more than 30 years (before advanced format was invented) dd wasn't invented to do things with hard drives, it was for working with tapes, file conversions, etc. Since record sizes on tapes and within files can vary widely, it's very flexible about what block sizes to use, and how to pad data to convert between block sizes. In most cases when people use dd these days it's not really the right tool for the job. I'll bite ;} When is it the right tool? When is it not?
Re: OT: minimum bs for dd?
On Sat, May 15, 2021 at 11:32:31PM +0300, IL Ka wrote: As noted, is there a minimum bs size for dd? It seems that you can set bs as small as 1. 512 is the default because of HDD block size which used to be 512 bytes for more than 30 years (before advanced format was invented) dd wasn't invented to do things with hard drives, it was for working with tapes, file conversions, etc. Since record sizes on tapes and within files can vary widely, it's very flexible about what block sizes to use, and how to pad data to convert between block sizes. In most cases when people use dd these days it's not really the right tool for the job.
Re: OT: minimum bs for dd?
Thanks to all for your responses! -- "No matter how big the problem is, you can always run away from it." Dom Irrera
Re: OT: minimum bs for dd?
> > > As noted, is there a minimum bs size for dd? > It seems that you can set bs as small as 1. 512 is the default because of HDD block size which used to be 512 bytes for more than 30 years (before advanced format was invented) >
Re: OT: minimum bs for dd?
On 2021-05-15 at 16:19, Bob Bernstein wrote: > As noted, is there a minimum bs size for dd? Yes: 1. dd will not accept a block size of 0, never mind a negative block size. You can try it yourself: $ dd if=/dev/zero of=/tmp/1zerobyte bs=1 count=1 1+0 records in 1+0 records out 1 byte copied, 0.000117071 s, 8.5 kB/s That said, block sizes below a certain value will give you reduced performance et cetera; what that value is depends on the physical characteristics of the underlying hardware. -- The Wanderer The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw signature.asc Description: OpenPGP digital signature
Re: OT: minimum bs for dd?
Bob Bernstein writes: As noted, is there a minimum bs size for dd? AFAIK its one byte. HTH Linux-Fan öö pgpQux2oMAtlK.pgp Description: PGP signature
