Re: CAM Shingled Disk support patches available
> On Jan 18, 2016, at 2:37 PM, Kenneth D. Merry wrote: > > I have a new set of SMR patches available. See below for the full > explanation. > > The primary change here is that I have added SMR support to the ada(4) > driver. I spent some time considering whether to try to make the da(4) and > ada(4) probe infrastructure somewhat common, but in the end concluded it > would be too involved with not enough code reduction (if any) in the end. > > So, although the ideas are similar, the probe logic is separate. > > Note that NCQ support for SMR commands (Report Zones, Reset Write Pointer, > etc.) for SATA protocol shingled drives isn't active. For both the da(4) > and ada(4) driver this is for lack of a way to plumb the ATA Auxiliary > register down to the drive. I’ve plumbed it down, but in a gross, kludgy way to make NCQ Trim work where the only value in the Auxiliary register needs to be 1. It only takes up one bit, but it doesn’t change the size of the CCB. If the NCQ Trim work wasn’t based on the I/O scheduler, I’d have pushed it into head and would be happy to share code. AHCI can send it, but it turns out that LSI’s drivers (mpt, mps, etc) can’t do it due to firmware inadequacies. The ability to send a FIS in these firmwares looked promising, but it requires a full draining of other requests, which kind of defeats the purpose of NCQ. > In the ada(4) case, we need to add the register to struct ccb_ataio and > add support in one or more of the underlying SATA drivers, e.g. ahci(4). I believe that changes the size of the CCB, so I tried to avoid that since I didn’t want to force a recompile of camcontrol(8). Adding it to the atacmd structure wasn’t so bad, and the CCB size didn’t completely change. The problem was that the atacmd changed size and pushed all the other fields. > In the da(4) case, it will require an update of the T-10 SAT spec to > provide a way to pass the Auxiliary register down via the SCSI ATA > PASS-THROUGH command, and then a subsquent update of the SAT layer in > various vendors' SAS controller firmware. At that point, there may be > an official mapping of the SCSI ZBC commands to the ATA ZAC commands, and > we may be able to just issue the SCSI version of the commands instead of > composing ATA commands in the da(4) driver. (We'll still need to keep the > ATA passthrough version for a while at least to support controllers that > don't have the updated translation code.) I looked to implement things here, but didn’t want to invent something that the T-10 would later reinvent. > FreeBSD/head as of SVN revision 294105: > > https://people.freebsd.org/~ken/cam_smr.head.20160118.1.txt > > FreeBSD stable/10 as of SVN revision 294100: > > https://people.freebsd.org/~ken/cam_smr.stable10.20160118.1.txt > > Testing and comments are welcome. So having said all that, I’m totally open to something better. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: r294248: boot stuck: EFI loader doesn't proceed
On Mon, 18 Jan 2016 22:46:18 + Steven Hartland wrote: > On 18/01/2016 21:57, Ed Maste wrote: > > On 18 January 2016 at 15:26, Andrew Turner > > wrote: > >> the issue was we were caching reads from the first filesystem we > >> looked at. I've committed the fix in r294291 to force the code to > >> re-read on each filesystem it looks at. > > Thanks Andrew - my QEMU boot failure is not reproducible on a build > > from a clean tree with this change included. > I still think there are issues in there, specifically the lookup call > in try_load, which calls fsread should be getting bad results. How? fsread will be called with dsk_meta == 0 so it will call the force probe path. Andrew ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: r294248: boot stuck: EFI loader doesn't proceed
On 18/01/2016 21:57, Ed Maste wrote: On 18 January 2016 at 15:26, Andrew Turner wrote: the issue was we were caching reads from the first filesystem we looked at. I've committed the fix in r294291 to force the code to re-read on each filesystem it looks at. Thanks Andrew - my QEMU boot failure is not reproducible on a build from a clean tree with this change included. I still think there are issues in there, specifically the lookup call in try_load, which calls fsread should be getting bad results. I believe the code here https://reviews.freebsd.org/D4989 fixes that as well as eliminating fsstat and all the duplicated code. Regards Steve ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: r294248: boot stuck: EFI loader doesn't proceed
On 18/01/2016 19:08, Ed Maste wrote: On 18 January 2016 at 03:10, Dimitry Andric wrote: On 18 Jan 2016, at 07:20, O. Hartmann wrote: Building NanoBSD images booting off from USB Flash drives and having two GPT partitions, booting is stuck in the UEFI loader, presenting me with something like: [...] Probing 6 block devices.++. done ZFS found no pools UFS found 2 partitions And further nothing happens. A RESET is only possible by a hardreset - it seems the system is crashed/stuck/frozen or something similar. The last images working run r293654. The issue occurs with r294248. Any suggestions possible? Did I miss something? Looks to me like fallout from the recent modularisation in r294060, and/or ZFS support in r294068. Steven, any clue? In QEMU boot1 failed for me with "Failed start image provided by UFS", and I can confirm that it's fixed by reverting those two commits. I believe this is an issue with UFS caching code introduced by the UFS modularisation. Andrew fixed some of it but not all in r294291, I believe the lookup results in try_load would still have been invalid. https://reviews.freebsd.org/D4989 should fix the rest, as well as resulting in much simpler flow IMO. If you could try this that would be great. Regards Steve ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: CAM Shingled Disk support patches available
I have a new set of SMR patches available. See below for the full explanation. The primary change here is that I have added SMR support to the ada(4) driver. I spent some time considering whether to try to make the da(4) and ada(4) probe infrastructure somewhat common, but in the end concluded it would be too involved with not enough code reduction (if any) in the end. So, although the ideas are similar, the probe logic is separate. Note that NCQ support for SMR commands (Report Zones, Reset Write Pointer, etc.) for SATA protocol shingled drives isn't active. For both the da(4) and ada(4) driver this is for lack of a way to plumb the ATA Auxiliary register down to the drive. In the ada(4) case, we need to add the register to struct ccb_ataio and add support in one or more of the underlying SATA drivers, e.g. ahci(4). In the da(4) case, it will require an update of the T-10 SAT spec to provide a way to pass the Auxiliary register down via the SCSI ATA PASS-THROUGH command, and then a subsquent update of the SAT layer in various vendors' SAS controller firmware. At that point, there may be an official mapping of the SCSI ZBC commands to the ATA ZAC commands, and we may be able to just issue the SCSI version of the commands instead of composing ATA commands in the da(4) driver. (We'll still need to keep the ATA passthrough version for a while at least to support controllers that don't have the updated translation code.) FreeBSD/head as of SVN revision 294105: https://people.freebsd.org/~ken/cam_smr.head.20160118.1.txt FreeBSD stable/10 as of SVN revision 294100: https://people.freebsd.org/~ken/cam_smr.stable10.20160118.1.txt Testing and comments are welcome. Ken On Wed, Nov 18, 2015 at 12:13:09 -0500, Kenneth D. Merry wrote: > > I have work in progress patches to add SMR (Shingled Magnetic Recording) > support to CAM and GEOM here: > > FreeBSD/head as of SVN revision 290997: > > https://people.freebsd.org/~ken/cam_smr.head.20151117.1.txt > > FreeBSD stable/10 as of SVN revision 290995: > > https://people.freebsd.org/~ken/cam_smr.stable10.20151117.1.txt > > This includes support for Host Managed, Host Aware and Drive Managed SMR > drives that are either SCSI (ZBC) or ATA (ZAC) attached via a SAS > controller. This does not include support for SMR ATA drives attched via > an ATA controller. Also, I have not yet figured out how to properly detect > a Host Managed ATA drive, so this code won't do that. > > The big drive vendors are moving to SMR for at least some of their drives. > The primary challenge with SMR is that it requires writing a relatively > large zone sequentially starting at the beginning of the zone. The usual > zone size is 256MB. It is conceptually almost like having a 256MB sector > size. > > We (Spectra Logic) are working on ZFS changes that will use this CAM and > GEOM infrastructure to make ZFS play well with SMR drives. Those changes > aren't yet done. > > The patches linked above include: > o A new 'camcontrol zone' command that allows displaying and managing >drive zones via SCSI/ATA passthrough. > o A new zonectl(8) utility that uses the new DIOCZONECMD ioctl to display >and manage zones via the da(4) (and later ada(4)) driver. > o Changes to diskinfo -v to display the zone mode of a drive. > o A new disk zone API, sys/sys/disk_zone.h. > o A new bio type, BIO_ZONE, and modifications to GEOM to support it. This >new bio will allow filesystems to query zone support in a drive and >manage zoned drives. > o Extensive modifications to the da(4) driver to handle probing SCSI and >SATA behind SAS SMR drives. > o Additional CAM CDB building functions for zone commands. > > The current issues that need to be addressed are: > o The da(4) driver now has 6 additional probe states, 5 of which are >needed for probing ATA drives behind SAS controllers. I have not yet >added support for BIO_ZONE bios to ada(4), but it will be very similar >to the da(4) driver version. The ATA probe code needs to be pulled >out of the da(4) driver and changed into a form that will allow it to >work for either the ada(4) or da(4) driver. Otherwise we'll have a fair >amount of code duplication between the two drivers. > > o There is a reasonable amount of code duplication between 'camcontrol zone' >and zonectl(8). This was done for speed / expediency's sake, but it may >be possible to make more of the code common there. > > o In order to add the new BIO_ZONE bio command, I had to change the bio_cmd >field in struct bio from a uint8_t to a uint16_t. This will cause >binary compatibility problems with any 3rd party loadable modules. >Advice on how to handle this would be welcome. > > o In the process of developing these changes, I discovered that the >dxfer_len paramter for scsi_ata_pass_16() was too small (uint16_t, and >it needed to be uint32_t). I increased it, but that will potentially >cause
Re: r294248: boot stuck: EFI loader doesn't proceed
On 18 January 2016 at 15:26, Andrew Turner wrote: > > the issue was we were caching reads from the first filesystem we looked > at. I've committed the fix in r294291 to force the code to re-read on > each filesystem it looks at. Thanks Andrew - my QEMU boot failure is not reproducible on a build from a clean tree with this change included. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: r294248: boot stuck: EFI loader doesn't proceed
On Mon, 18 Jan 2016 12:04:38 + Steven Hartland wrote: > On 18/01/2016 11:34, Andrew Turner wrote: > > On Mon, 18 Jan 2016 09:10:33 +0100 > > Dimitry Andric wrote: > > > >> On 18 Jan 2016, at 07:20, O. Hartmann > >> wrote: > >>> Building NanoBSD images booting off from USB Flash drives and > >>> having two GPT partitions, booting is stuck in the UEFI loader, > >>> presenting me with something like: > >>> > >>> [...] > >>> Probing 6 block devices.++. done > >>> > >>> ZFS found no pools > >>> UFS found 2 partitions > >>> > >>> And further nothing happens. A RESET is only possible by a > >>> hardreset - it seems the system is crashed/stuck/frozen or > >>> something similar. > >>> > >>> The last images working run r293654. The issue occurs with > >>> r294248. > >>> > >>> Any suggestions possible? Did I miss something? > >> Looks to me like fallout from the recent modularisation in r294060, > >> and/or ZFS support in r294068. Steven, any clue? > >> > >> -Dimitry > >> > > I found the same issue while working on nanobsd. I'm not sure it's > > due to the recent ZFS changes as I reverted them, but found boot1 > > still failed to load loader.efi. > > > Can you update to > r294265 and test with EFI_DEBUG defined e.g. > make buildenv -DEFI_DEBUG > cd sys/boot/efi/boot1 > make clean > make > > Then install the new boot1.efifat to your efi partition. > > Hopefully this will give some more information on what the problem > may be. the issue was we were caching reads from the first filesystem we looked at. I've committed the fix in r294291 to force the code to re-read on each filesystem it looks at. Andrew ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: r294248: boot stuck: EFI loader doesn't proceed
On 18 January 2016 at 03:10, Dimitry Andric wrote: > On 18 Jan 2016, at 07:20, O. Hartmann wrote: >> Building NanoBSD images booting off from USB Flash drives and having two GPT >> partitions, booting is stuck in the UEFI loader, presenting me with something >> like: >> >> [...] >> Probing 6 block devices.++. done >> >> ZFS found no pools >> UFS found 2 partitions >> >> And further nothing happens. A RESET is only possible by a hardreset - it >> seems >> the system is crashed/stuck/frozen or something similar. >> >> The last images working run r293654. The issue occurs with r294248. >> >> Any suggestions possible? Did I miss something? > > Looks to me like fallout from the recent modularisation in r294060, > and/or ZFS support in r294068. Steven, any clue? In QEMU boot1 failed for me with "Failed start image provided by UFS", and I can confirm that it's fixed by reverting those two commits. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: r294248: boot stuck: EFI loader doesn't proceed
On Mon, 18 Jan 2016 09:10:33 +0100 Dimitry Andric wrote: > On 18 Jan 2016, at 07:20, O. Hartmann > wrote: > > Building NanoBSD images booting off from USB Flash drives and > > having two GPT partitions, booting is stuck in the UEFI loader, > > presenting me with something like: > > > > [...] > > Probing 6 block devices.++. done > > > > ZFS found no pools > > UFS found 2 partitions > > > > And further nothing happens. A RESET is only possible by a > > hardreset - it seems the system is crashed/stuck/frozen or > > something similar. > > > > The last images working run r293654. The issue occurs with r294248. > > > > Any suggestions possible? Did I miss something? > > Looks to me like fallout from the recent modularisation in r294060, > and/or ZFS support in r294068. Steven, any clue? > > -Dimitry > I found the same issue while working on nanobsd. I'm not sure it's due to the recent ZFS changes as I reverted them, but found boot1 still failed to load loader.efi. Andrew ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Will pkg(8) *ever* play nice?
On Mon, 18 Jan 2016 00:26:39 -0800 "Chris H" wrote
> Greetings,
> I was in the process of updating one of my development boxes
> that runs -CURRENT. It's lagging a bit behind. So it required
> a fresh, new "world". World built as expected. But the kernel
> failed due to pkg(8). This box has an Nvidia video card. So
> src.conf(5) has [among other ports entries]
>
> PORTS_MODULES=x11/nvidia-driver-304
>
> the [kernel] build failed due to pkg(8)
>
> ===> Ports module x11/nvidia-driver-304 (all)
> cd ${PORTSDIR:-/usr/ports}/x11/nvidia-driver-304;
> PATH=/usr/obj/usr/src/tmp/lega
> cy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/b
> in:
> /usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/sbin:/bin:/usr/sb
> in: /usr/bin:/usr/local/bin:/usr/local/sbin SRC_BASE=/usr/src
> OSVERSION=1100062 W
> RKDIRPREFIX=/usr/obj/usr/src/sys/DEVBOX make -B clean all
> ===> Cleaning for nvidia-driver-304-304.128
> ===> nvidia-driver-304-304.128 pkg(8) must be version 1.6.0 or greater, but
> you have 1.4.12. You must upgrade the ports-mgmt/pkg port first.
> *** Error code 1
>
> Stop.
> make[3]: stopped in /usr/ports/x11/nvidia-driver-304
> *** Error code 1
>
> Stop.
> make[2]: stopped in /usr/obj/usr/src/sys/DEVBOX
> *** Error code 1
>
> Stop.
> make[1]: stopped in /usr/src
> *** Error code 1
>
> ..
>
> Do I need to have FreeBSD-12 && ports/head from 12 installed *prior*
> to building (upgrading)?!?
> This feels like a chicken-v-egg thing. Is it possible to *remove*
> pkg(8) && any dependency on it. Is this a bug in pkg(8) or just
> a bug in my head?
>
> Please help. As most of you already know; building/installing
> world/kernel is a fairly time consuming process.
>
> Thanks!
>
OK. It's late, and I'm ready to call it a day.
It's probably a bug in my head. I *probably* should have
upgraded pkg(8) after svn up'ing src && ports, and *prior*
to making worls && kernel.
So I'm going to take a chance, and:
cd /usr/obj
chflags -R noschg *
cd /
rm -r /usr/obj
cd /usr/ports/ports-mgmt/pkg
make; make deinstall && make resinstall clean
cd /usr/src
and make world, and kernel, and see how it goes.
--Chris
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Will pkg(8) *ever* play nice?
Greetings,
I was in the process of updating one of my development boxes
that runs -CURRENT. It's lagging a bit behind. So it required
a fresh, new "world". World built as expected. But the kernel
failed due to pkg(8). This box has an Nvidia video card. So
src.conf(5) has [among other ports entries]
PORTS_MODULES=x11/nvidia-driver-304
the [kernel] build failed due to pkg(8)
===> Ports module x11/nvidia-driver-304 (all)
cd ${PORTSDIR:-/usr/ports}/x11/nvidia-driver-304;
PATH=/usr/obj/usr/src/tmp/lega
cy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/bin:
/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/sbin:/bin:/usr/sbin:
/usr/bin:/usr/local/bin:/usr/local/sbin SRC_BASE=/usr/src OSVERSION=1100062
W
RKDIRPREFIX=/usr/obj/usr/src/sys/DEVBOX make -B clean all
===> Cleaning for nvidia-driver-304-304.128
===> nvidia-driver-304-304.128 pkg(8) must be version 1.6.0 or greater, but
you have 1.4.12. You must upgrade the ports-mgmt/pkg port first.
*** Error code 1
Stop.
make[3]: stopped in /usr/ports/x11/nvidia-driver-304
*** Error code 1
Stop.
make[2]: stopped in /usr/obj/usr/src/sys/DEVBOX
*** Error code 1
Stop.
make[1]: stopped in /usr/src
*** Error code 1
..
Do I need to have FreeBSD-12 && ports/head from 12 installed *prior*
to building (upgrading)?!?
This feels like a chicken-v-egg thing. Is it possible to *remove*
pkg(8) && any dependency on it. Is this a bug in pkg(8) or just
a bug in my head?
Please help. As most of you already know; building/installing
world/kernel is a fairly time consuming process.
Thanks!
--Chris
--
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: r294248: boot stuck: EFI loader doesn't proceed
On 18 Jan 2016, at 07:20, O. Hartmann wrote: > Building NanoBSD images booting off from USB Flash drives and having two GPT > partitions, booting is stuck in the UEFI loader, presenting me with something > like: > > [...] > Probing 6 block devices.++. done > > ZFS found no pools > UFS found 2 partitions > > And further nothing happens. A RESET is only possible by a hardreset - it > seems > the system is crashed/stuck/frozen or something similar. > > The last images working run r293654. The issue occurs with r294248. > > Any suggestions possible? Did I miss something? Looks to me like fallout from the recent modularisation in r294060, and/or ZFS support in r294068. Steven, any clue? -Dimitry signature.asc Description: Message signed with OpenPGP using GPGMail
