CD9660/md(4)/UFS22 silly behaviour
I'm doing som data-mining on a pile of ISO images right now. I stuck the ISOs on a UFS2 on a flash-disk for speed, and mdconfig(8)'d them so I could mount them. The traffic pattern his interesting: dT: 1.003s w: 1.000s L(q) ops/sr/s kBps ms/rw/s kBps ms/w %busy Name [...] 1733733 14661.3 0 00.0 98.2| md39 1733733 234491.3 0 00.0 93.2| da0 Notice the 1:16 ratio on kBps but 1:1 ratio on ops/s ? da0's UFS2 has 32k block-size: magic 19540119 (UFS2) timeWed Jan 4 16:41:47 2012 superblock location 65536 id [ 4f046cf5 c30697ee ] ncg 104 size19537685blocks 19228156 bsize 32768 shift 15 mask0x8000 fsize 4096shift 12 mask0xf000 [...] It looks like every 2k read from CD9660 turns into a 32k block read in the UFS filesystem, without any beneficial caching happening. Less than optimal I'd say... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CD9660/md(4)/UFS22 silly behaviour
On Sun, Jan 08, 2012 at 10:21:50PM +, Poul-Henning Kamp wrote: I'm doing som data-mining on a pile of ISO images right now. I stuck the ISOs on a UFS2 on a flash-disk for speed, and mdconfig(8)'d them so I could mount them. The traffic pattern his interesting: dT: 1.003s w: 1.000s L(q) ops/sr/s kBps ms/rw/s kBps ms/w %busy Name [...] 1733733 14661.3 0 00.0 98.2| md39 1733733 234491.3 0 00.0 93.2| da0 Notice the 1:16 ratio on kBps but 1:1 ratio on ops/s ? da0's UFS2 has 32k block-size: magic 19540119 (UFS2) timeWed Jan 4 16:41:47 2012 superblock location 65536 id [ 4f046cf5 c30697ee ] ncg 104 size19537685blocks 19228156 bsize 32768 shift 15 mask0x8000 fsize 4096shift 12 mask0xf000 [...] It looks like every 2k read from CD9660 turns into a 32k block read in the UFS filesystem, without any beneficial caching happening. Less than optimal I'd say... What is the access patern ? Is it random access, or sequential read (from the cd9660 POV) ? pgpj4AQGx2y89.pgp Description: PGP signature
Re: CD9660/md(4)/UFS22 silly behaviour
In message 20120108222720.gn31...@deviant.kiev.zoral.com.ua, Kostik Belousov writes: What is the access patern ? Is it random access, or sequential read (from the cd9660 POV) ? Random access to files in the CD9660 filesystem, which stores files in sequential 2K blocks. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CD9660/md(4)/UFS22 silly behaviour
On Sun, Jan 08, 2012 at 10:31:06PM +, Poul-Henning Kamp wrote: In message 20120108222720.gn31...@deviant.kiev.zoral.com.ua, Kostik Belousov writes: What is the access patern ? Is it random access, or sequential read (from the cd9660 POV) ? Random access to files in the CD9660 filesystem, which stores files in sequential 2K blocks. Then it is reasonable. UFS reads full blocks. If you want/plan to use UFS volume for small reads exclusively, you can newfs it with much smaller block size, e.g. 8KB or even 4KB. pgp2u3DjXRzwg.pgp Description: PGP signature
Re: CD9660/md(4)/UFS22 silly behaviour
On Sun, Jan 8, 2012 at 2:33 PM, Kostik Belousov kostik...@gmail.com wrote: On Sun, Jan 08, 2012 at 10:31:06PM +, Poul-Henning Kamp wrote: In message 20120108222720.gn31...@deviant.kiev.zoral.com.ua, Kostik Belousov writes: What is the access patern ? Is it random access, or sequential read (from the cd9660 POV) ? Random access to files in the CD9660 filesystem, which stores files in sequential 2K blocks. Then it is reasonable. UFS reads full blocks. If you want/plan to use UFS volume for small reads exclusively, you can newfs it with much smaller block size, e.g. 8KB or even 4KB. I think the complaint is that UFS is reading 32 KB (which includes the 2 KB block and 15 others that will be needed right after) but not caching the 30 KB of data that follows the requested 2 KB. -- Freddie Cash fjwc...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CD9660/md(4)/UFS22 silly behaviour
On Sun, Jan 8, 2012 at 3:01 PM, Freddie Cash fjwc...@gmail.com wrote: On Sun, Jan 8, 2012 at 2:33 PM, Kostik Belousov kostik...@gmail.com wrote: On Sun, Jan 08, 2012 at 10:31:06PM +, Poul-Henning Kamp wrote: In message 20120108222720.gn31...@deviant.kiev.zoral.com.ua, Kostik Belousov writes: What is the access patern ? Is it random access, or sequential read (from the cd9660 POV) ? Random access to files in the CD9660 filesystem, which stores files in sequential 2K blocks. Then it is reasonable. UFS reads full blocks. If you want/plan to use UFS volume for small reads exclusively, you can newfs it with much smaller block size, e.g. 8KB or even 4KB. I think the complaint is that UFS is reading 32 KB (which includes the 2 KB block and 15 others that will be needed right after) but not caching the 30 KB of data that follows the requested 2 KB. Speaking of the increase in block size and frag size, on the FreeNAS side of the house we discovered that the new values make sense with larger disks and memory disks, but they don't make sense with rc.initdiskless's generated md's, et all. So we've hacked rc.initdiskless to use smaller sizes. Just a thought since we're talking about this change from the 8.x defaults to the 9.x defaults. Thanks! -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CD9660/md(4)/UFS22 silly behaviour
On 8 Jan, Poul-Henning Kamp wrote: I'm doing som data-mining on a pile of ISO images right now. I stuck the ISOs on a UFS2 on a flash-disk for speed, and mdconfig(8)'d them so I could mount them. The traffic pattern his interesting: dT: 1.003s w: 1.000s L(q) ops/sr/s kBps ms/rw/s kBps ms/w %busy Name [...] 1733733 14661.3 0 00.0 98.2| md39 1733733 234491.3 0 00.0 93.2| da0 Notice the 1:16 ratio on kBps but 1:1 ratio on ops/s ? da0's UFS2 has 32k block-size: magic 19540119 (UFS2) timeWed Jan 4 16:41:47 2012 superblock location 65536 id [ 4f046cf5 c30697ee ] ncg 104 size19537685blocks 19228156 bsize 32768 shift 15 mask0x8000 fsize 4096shift 12 mask0xf000 [...] It looks like every 2k read from CD9660 turns into a 32k block read in the UFS filesystem, without any beneficial caching happening. Probably some confusion about which filesystem layer owns the cached data. It would probably be inefficient to cache the data in both places. The best fix would probably be for CD9660 to think that the underlying device has 32Kb sectors. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CD9660/md(4)/UFS22 silly behaviour
On Sun, Jan 08, 2012 at 04:09:00PM -0800, Don Lewis wrote: On 8 Jan, Poul-Henning Kamp wrote: I'm doing som data-mining on a pile of ISO images right now. I stuck the ISOs on a UFS2 on a flash-disk for speed, and mdconfig(8)'d them so I could mount them. The traffic pattern his interesting: dT: 1.003s w: 1.000s L(q) ops/sr/s kBps ms/rw/s kBps ms/w %busy Name [...] 1733733 14661.3 0 00.0 98.2| md39 1733733 234491.3 0 00.0 93.2| da0 Notice the 1:16 ratio on kBps but 1:1 ratio on ops/s ? da0's UFS2 has 32k block-size: magic 19540119 (UFS2) timeWed Jan 4 16:41:47 2012 superblock location 65536 id [ 4f046cf5 c30697ee ] ncg 104 size19537685blocks 19228156 bsize 32768 shift 15 mask0x8000 fsize 4096shift 12 mask0xf000 [...] It looks like every 2k read from CD9660 turns into a 32k block read in the UFS filesystem, without any beneficial caching happening. Probably some confusion about which filesystem layer owns the cached data. It would probably be inefficient to cache the data in both places. The best fix would probably be for CD9660 to think that the underlying device has 32Kb sectors. I discussed the issue with phk further. The reason for discarding the 30K of the read 32K block is that md(4) supplies IO_DIRECT flag for VOP_READ, and FFS avoids putting the read data into any cache. Most likely, we can implement a sysctl that would disable direct reads, at the cost of double-buffering the data. For obvious reasons, it is impossible to disable caching from the filesystem living on top of md(4) volume. pgpuHe11Cxjyv.pgp Description: PGP signature
