Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Tue, 2014-08-26 at 10:47 -0400, Alan Stern wrote: On Mon, 25 Aug 2014, Oliver Neukum wrote: Just set US_FL_NEEDS_CAP16. If READ CAPACITY(16) fails in that case, it is clear that something is wrong. It must be set or READ CAPACITY(10) alone would be taken as giving a valid answer. You have committed a mental layering violation. :-) Indeed. US_FL_NEEDS_CAP16 is a usb-storage flag. But the capacity determination is made by the sd driver, which relies on the SCSI try_rc_10_first flag. If that flag isn't set, sd tries READ CAPACITY(16) and then falls back to READ CAPACITY(10) if an error occurs. That's what happened here. I don't think we want to add another SCSI flag to say that READ CAPACITY(10) is unreliable. Why not? It would only be friendly to tell the upper layer of a malfunction if we know about it. At that time we are sure that the drive will be unusable unless we determine the correct capacity, so we don't make matters worse if we crash it. Given the difficulty of determining the true capacity, I see two alternatives. We could set the capacity to a ridiculously large value (like 1 billion TB), or we could leave the capacity set to the low value and disable the block within bounds checks. Neither of these is attractive and they both have issues of their own -- although the second is close to what Windows does. That seems to be the most attractive solution to me. (For example, udev often tries to read the last sectors of a new drive, looking for a GPT or RAID signature. That won't work if the capacity is set to some random value.) Yes, but clipping has its own dangers. Suppose you use the medium without a partition table. Regards Oliver -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Wed, 27 Aug 2014, Oliver Neukum wrote: I don't think we want to add another SCSI flag to say that READ CAPACITY(10) is unreliable. Why not? It would only be friendly to tell the upper layer of a malfunction if we know about it. To what end? What will the upper layer do with this information? Given the difficulty of determining the true capacity, I see two alternatives. We could set the capacity to a ridiculously large value (like 1 billion TB), or we could leave the capacity set to the low value and disable the block within bounds checks. Neither of these is attractive and they both have issues of their own -- although the second is close to what Windows does. That seems to be the most attractive solution to me. I'm skeptical that you can convince the SCSI and block-layer developers about this. Maybe they'll accept it if it is applied only to USB transports... (For example, udev often tries to read the last sectors of a new drive, looking for a GPT or RAID signature. That won't work if the capacity is set to some random value.) Yes, but clipping has its own dangers. Suppose you use the medium without a partition table. What would Windows do? In the absence of a partition table, it would believe the value from READ CAPACITY, right? Isn't that the same as clipping? Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
From: Alan Stern On Mon, 25 Aug 2014, Alfredo Dal Ava Junior wrote: Well, it is causing problems anyway... from user perspective, it's a Linux compatibility issue, as it works fine on Windows. I'm not an expert, but I'm wondering that if usb-storage could set capacity as UNDETERMINED/ Zero (or keep using the readcapacity_10 as it as with some flag signalizing it as inaccurate), EFI partition check would be able to ignore size and look for secondary GPT where it really is. Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Could the code try READ CAPACITY(16) first? Really there is only one way to know whether the actual capacity is larger than the reported capacity, and that is by trying to read blocks beyond the reported capacity -- a dangerous test that many drives do not like. (And in most cases a futile test. If a device doesn't support READ CAPACITY(16), how likely is it to support READ(16)?) Yes, in theory you can believe the value in the partition table in preference to the reported capacity. But what if that value is wrong? And how do you tell partition-manager programs what the capacity should be when they modify or set up the initial partition table? I've a feeling that, historically at least, windows believes the partition table. I remember some CF cards that locked up when I tried to read the 'device info' sector, and others (apparently identical) that had the 32bit sector size misaligned. These were 'major manufacturer' cards as well. David -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Tue, 2014-08-26 at 09:58 +, David Laight wrote: Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Could the code try READ CAPACITY(16) first? Yes. It does already. That fails as the device doesn't support this version. In a way we are discussing error handling here. Regards Oliver -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
From Oliver Neukum [mailto:oneu...@suse.de]
On Tue, 2014-08-26 at 09:58 +, David Laight wrote:
Part of the problem is that usb-storage has no way to know that
anything strange is going on. It's normal for READ CAPACITY(16) to
fail (this depend on the SCSI level), and it's normal for the READ
CAPACITY(10) to report a value less than 2 TB.
Could the code try READ CAPACITY(16) first?
Yes. It does already. That fails as the device doesn't support
this version. In a way we are discussing error handling here.
I read more of the thread later (getting outluck to sort mails in
any sensible way is almost impossible.)
I'm sort of surprised that the 16byte reads work if the 16byte
read capacity doesn't.
I wonder what the manufacturer would saw in response the bug where
windows shows the incorrect size when trying to partition the disk?
Such bugs ought to fail 'fitness for purpose' - so, in the UK, the
shop would have to replace the 'faulty' goods.
David
N�r��yb�X��ǧv�^�){.n�+{��^n�r���z���h��� �G���h�(�階�ݢj���m��z�ޖ���f���h���~�m�
RES: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 26 Aug 2014, David Leight wrote: I wonder what the manufacturer would saw in response the bug where windows shows the incorrect size when trying to partition the disk? I contacted enclosure manufacturer (Welland) some weeks ago, they are supposed to escalate my questions to engineering and I'm still waiting for reply. Interesting thing is that I read AS1051 on the chip, but it is detected as AS2105 (maybe same VID and PID for different chips). I contacted ASMedia (chip bridge manufacturer), they replied that ASM1051 supports 48bit LBAs and that some FW feature and FW SPEC depends on manufacture's request. They added that this chip entered EOL this year and I have to contact Welland for support. Anyway, it show explain way we see distinct behavior for same the chip (and same VIDPID). []'s Alfredo
RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 15 Aug 2014 James Bottomley wrote: So how did the partition get on there at the correct size in the first place? Even under windows partition managers believe the disk size they get from the system if the disk is blank. The HDD can be partitioned outside the enclosure, when connected directly to one SATA port on motherboard. READ_CAPACITY(16) will return properly when talking directly to the HDD. I assume for those of us on linux-scsi who don't have the start of this thread, you already tried read capacity(16) and it has this same problem? Sorry, I forgot to include linux-scsi. On this device, READ_CAPACITY_16 fails 100% of times as unsupported command, then READ_CAPACITY_10 has a distinct behavior depending on HDD size: 1TB and 2TB: READ_CAPACITY_10 returns correct value 3TB and 4TB: READ_CAPACITY_10 returns in a 2TB modulus Hm, allowing users to set desired capacity by overriding the partition size ... I'm sure that's not going to cause support problems ... Well, it is causing problems anyway... from user perspective, it's a Linux compatibility issue, as it works fine on Windows. I'm not an expert, but I'm wondering that if usb-storage could set capacity as UNDETERMINED/ Zero (or keep using the readcapacity_10 as it as with some flag signalizing it as inaccurate), EFI partition check would be able to ignore size and look for secondary GPT where it really is. []'s Alfredo -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 25 Aug 2014 Alan Stern wrote:
On Mon, 25 Aug 2014, Alfredo Dal Ava Junior wrote:
That's right. I don't know why Windows behaves that way.
Please look this output from diskpart (Windows):
DISKPART list partition
Partition ### Type Size Offset
- --- ---
Partition 1Primary 3726 GB17 KB
DISKPART list disk
Disk ### Status Size Free Dyn Gpt
- --- --- --- ---
Disk 0Online 298 GB 0 B
* Disk 1Online 1678 GB 0 B*
DISKPART select disk 1
Disk 1 is now the selected disk.
DISKPART list partition
Partition ### Type Size Offset
- --- ---
Partition 1Primary 3726 GB17 KB
Could we do the same? Would be possible to signalize to upper layers
that capacity is not accurate (or return zero) on this device, and
tell GPT handlers to bypass it's partition_size vs disk size
consistency check?
There is no way to do this, as far as I know. But I'm not an expert in this
area.
Maybe you can figure out a way to add this capability.
(But then what happens if the size stored in the partition table really is
larger
than the disk's capacity?)
It's the first time I touch this code, but I will learn from the code and try
to find it out... but still in hope to find a clean solution...
If size stored on partition table is really larger than disk, my guess it will
cause I/O errors, and that some disks may get crazy like you pointed.
Do you think disk could cause kernel instability? I think it is acceptable if
no consequences to kernel stability, since it is a specific-device
workaround.
[]'s
Alfredo
--
To unsubscribe from this list: send the line unsubscribe linux-usb in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 25 Aug 2014, Alfredo Dal Ava Junior wrote: Well, it is causing problems anyway... from user perspective, it's a Linux compatibility issue, as it works fine on Windows. I'm not an expert, but I'm wondering that if usb-storage could set capacity as UNDETERMINED/ Zero (or keep using the readcapacity_10 as it as with some flag signalizing it as inaccurate), EFI partition check would be able to ignore size and look for secondary GPT where it really is. Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Really there is only one way to know whether the actual capacity is larger than the reported capacity, and that is by trying to read blocks beyond the reported capacity -- a dangerous test that many drives do not like. (And in most cases a futile test. If a device doesn't support READ CAPACITY(16), how likely is it to support READ(16)?) Yes, in theory you can believe the value in the partition table in preference to the reported capacity. But what if that value is wrong? And how do you tell partition-manager programs what the capacity should be when they modify or set up the initial partition table? Attaching the drive over a SATA connection when you want to partition it isn't a very satisfactory solution. After all, if you have a SATA connection available then why would you be using a USB enclosure in the first place? Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RES: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 25 Aug 2014 Alan Stern wrote: Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Really there is only one way to know whether the actual capacity is larger than the reported capacity, and that is by trying to read blocks beyond the reported capacity -- a dangerous test that many drives do not like. (And in most cases a futile test. If a device doesn't support READ CAPACITY(16), how likely is it to support READ(16)?) Yes, in theory you can believe the value in the partition table in preference to the reported capacity. But what if that value is wrong? And how do you tell partition-manager programs what the capacity should be when they modify or set up the initial partition table? We may add this device to UNUSUAL_DEV list, to keep using READ_CAPACITY(10) and pass some flag to bypass EFI GPT partition check. I'm almost sure that kernel will be able to mount the partition if we can make it available on /proc/partition. This would make this device behaves like it does on Windows 7. I see this as best effort workaround for a cheap buggy hardware, like many others on UNUSUAL_DEV list Attaching the drive over a SATA connection when you want to partition it isn't a very satisfactory solution. After all, if you have a SATA connection available then why would you be using a USB enclosure in the first place? You may want do a backup or plug it in a laptop, for example. []'s Alfredo -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 2014-08-25 at 16:21 -0400, Alan Stern wrote: On Mon, 25 Aug 2014, Alfredo Dal Ava Junior wrote: Well, it is causing problems anyway... from user perspective, it's a Linux compatibility issue, as it works fine on Windows. I'm not an expert, but I'm wondering that if usb-storage could set capacity as UNDETERMINED/ Zero (or keep using the readcapacity_10 as it as with some flag signalizing it as inaccurate), EFI partition check would be able to ignore size and look for secondary GPT where it really is. Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Just set US_FL_NEEDS_CAP16. If READ CAPACITY(16) fails in that case, it is clear that something is wrong. It must be set or READ CAPACITY(10) alone would be taken as giving a valid answer. At that time we are sure that the drive will be unusable unless we determine the correct capacity, so we don't make matters worse if we crash it. Is there an easy way for Alfredo to determine what happens if we read beyond the end? Regards Oliver -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
