Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Wed, 3 Sep 2014, James Bottomley wrote:
On Wed, 2014-09-03 at 16:30 -0400, Alan Stern wrote:
On Wed, 3 Sep 2014, James Bottomley wrote:
Before we embark on elaborate hacks, why don't we just make the capacity
writeable (by root) in sysfs from userspace (will require block change)?
We can then encode all the nasty heuristics (including gpt reading) in
userspace as a udev rule.
That's what I'm working on. Except that I don't know where to do it in
the block layer, so for now I'm adding the attribute to sd.c.
Where in the block layer would the right place be? We want this to
apply only to entire devices, not to individual partitions.
The bottom layer for this is part0.nr_sects which is the size attribute
you see in the block sysfs. However, it looks like we keep a separate
value in sdkp, but we don't ever seem to use it (except to see if the
capacity has changed).
So this could be done in two ways: add a writeable capacity attribute in
sd.c, as you were originally thinking (it would call set_capacity() on
write and that would update the block layer) or make the size parameter
writeable.
Here's my patch to the sd driver. It seems to work -- writing to the
capacity_override attribute does change the device size. But then you
have to force the kernel to re-read the partition table manually, for
example by running
blockdev --rereadpt /dev/sdX
because I couldn't see any reasonable way to make this happen
automatically.
Alan Stern
Index: usb-3.17/drivers/scsi/sd.h
===
--- usb-3.17.orig/drivers/scsi/sd.h
+++ usb-3.17/drivers/scsi/sd.h
@@ -66,6 +66,7 @@ struct scsi_disk {
struct gendisk *disk;
atomic_topeners;
sector_tcapacity; /* size in 512-byte sectors */
+ sector_tcapacity_override; /* in native-size blocks */
u32 max_xfer_blocks;
u32 max_ws_blocks;
u32 max_unmap_blocks;
Index: usb-3.17/drivers/scsi/sd.c
===
--- usb-3.17.orig/drivers/scsi/sd.c
+++ usb-3.17/drivers/scsi/sd.c
@@ -477,6 +477,38 @@ max_write_same_blocks_store(struct devic
}
static DEVICE_ATTR_RW(max_write_same_blocks);
+static ssize_t
+capacity_override_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct scsi_disk *sdkp = to_scsi_disk(dev);
+
+ return sprintf(buf, %llu\n,
+ (unsigned long long) sdkp-capacity_override);
+}
+
+static ssize_t
+capacity_override_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct scsi_disk *sdkp = to_scsi_disk(dev);
+ unsigned long long cap;
+ int err;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EACCES;
+
+ err = kstrtoull(buf, 10, cap);
+ if (err)
+ return err;
+
+ sdkp-capacity_override = cap;
+ revalidate_disk(sdkp-disk);
+
+ return count;
+}
+static DEVICE_ATTR_RW(capacity_override);
+
static struct attribute *sd_disk_attrs[] = {
dev_attr_cache_type.attr,
dev_attr_FUA.attr,
@@ -489,6 +521,7 @@ static struct attribute *sd_disk_attrs[]
dev_attr_provisioning_mode.attr,
dev_attr_max_write_same_blocks.attr,
dev_attr_max_medium_access_timeouts.attr,
+ dev_attr_capacity_override.attr,
NULL,
};
ATTRIBUTE_GROUPS(sd_disk);
@@ -2158,6 +2191,13 @@ sd_read_capacity(struct scsi_disk *sdkp,
struct scsi_device *sdp = sdkp-device;
sector_t old_capacity = sdkp-capacity;
+ /* Did the user override the reported capacity? */
+ if (!sdkp-first_scan sdkp-capacity_override) {
+ sector_size = sdkp-device-sector_size;
+ sdkp-capacity = sdkp-capacity_override;
+ goto got_data;
+ }
+
if (sd_try_rc16_first(sdp)) {
sector_size = read_capacity_16(sdkp, sdp, buffer);
if (sector_size == -EOVERFLOW)
--
To unsubscribe from this list: send the line unsubscribe linux-usb in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
From: James Bottomley james.bottom...@hansenpartnership.com Before we embark on elaborate hacks, why don't we just make the capacity writeable (by root) in sysfs from userspace (will require block change)? We can then encode all the nasty heuristics (including gpt reading) in userspace as a udev rule. Looking in from the outside, this makes sense to me. All the nasty hueristics can be put in userspace -- perhaps as even a special-purpose program, where we can directly warn the user as to what he's getting himself into. (I do not demand that this all work automatically.) And the hueristics can be improved easily, without kernel changes. The only gotcha that I see is that once the recorded device size is changed, the kernel may now consider the partition table to be valid, and should now parse it and set up the special device numbers for the partitions. So that needs to get triggered properly. I suppose there is some complexity if the block-handling layer already has blocks cached and the device size is reduced below the addresses of the cached blocks. But as long as the kernel doesn't crash or write blocks in incorrect places, I don't see that as a problem -- if you set the device size as less than the block numbers you've already written to, that's your problem. Dale -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
From: Alan Stern st...@rowland.harvard.edu On Fri, 29 Aug 2014, Matthew Dharm wrote: Is there an 'easy' way to override the detected size of a storage device from userspace? If we had that, someone could write a helper application which looked for this particular fubar and try to Do The Right Thing(tm), or at least offer the user some options. You mean, force a Media Change event and override the capacity reported by the hardware? I'm not aware of any API for doing that, although it probably wouldn't be too hard to add one. How would the user know what value to put in for the capacity? Unless the drive had been hooked up to a different computer and the user manually noted the correct capacity and typed it in, it would have to be guesswork. The value would most sanely be extracted from the partition table. (After verifying that the partition table looks correct.) That seems to be what Windows does, and it seems to work consistently enough for Windows to trust that method. Or at least, it could take the disk size to be the end of the last partition, which would at least make all the partitions accessible. As somebody else hinted at, the userspace program could check the USB device against a list of device types known to have this problem. It could even verify that the SCSI-reported size matches the size reported by the partition table (modulo two-to-the-whatever) (at least for GPT tables, I don't know if MBR tables report the disk size). Do we have any way of knowing what algorithm Windows uses in this situation? Dale -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Wed, 3 Sep 2014, Dale R. Worley wrote: From: Alan Stern st...@rowland.harvard.edu On Fri, 29 Aug 2014, Matthew Dharm wrote: Is there an 'easy' way to override the detected size of a storage device from userspace? If we had that, someone could write a helper application which looked for this particular fubar and try to Do The Right Thing(tm), or at least offer the user some options. You mean, force a Media Change event and override the capacity reported by the hardware? I'm not aware of any API for doing that, although it probably wouldn't be too hard to add one. How would the user know what value to put in for the capacity? Unless the drive had been hooked up to a different computer and the user manually noted the correct capacity and typed it in, it would have to be guesswork. The value would most sanely be extracted from the partition table. (After verifying that the partition table looks correct.) That seems to be what Windows does, and it seems to work consistently enough for Windows to trust that method. Or at least, it could take the disk size to be the end of the last partition, which would at least make all the partitions accessible. If there is a partition table. It might be worthwhile to try an ATA pass-through command as well. As somebody else hinted at, the userspace program could check the USB device against a list of device types known to have this problem. It could even verify that the SCSI-reported size matches the size reported by the partition table (modulo two-to-the-whatever) (at least for GPT tables, I don't know if MBR tables report the disk size). They don't. Just the start and end of each partition. Do we have any way of knowing what algorithm Windows uses in this situation? Ask Microsoft. I suspect you're not likely to get an answer, though. Anyway, I can try writing a patch to add this capability. We'll see if it can solve your problem. Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RES: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Sat, 30 Aug 2014, Matthew Dharm wrote I was thinking of something that could notice a USB device which is formatted NTFS and has a partition table and filesystem that indicates a much bigger capacity than what the drive reports. Under this circumstances, you could do something like pop-up a dialog box saying this drive is confused -- is it 2TB or 3TB? Well, maybe that would say Drive capacity is not consistent with partition table. This can happen with certain USB drives designed for use with Windows. Override drive capacity (emulating Windows)? You could imagine increasing complex heuristics to try to detect this scenario. Even without an automated helper program to do it, if there was a sysfs interface then when we got the periodic e-mails reporting this same type of problem, we could offer a quick-and-clean solution. Hi Matt, I did small hack to skip is_pte_valid() on efi.c and now I have sdc1 partition listed on /proc/partition, but I hit other issue: ntfs-3g mount userspace tool that comes with Fedora now fails with Failed to read last sector (7814037100): Invalid argument. I also tried to override disk capacity, but SD driver fails with Invalid command failure and Illegal Request (I'll investigate it later) Many places rely on disk capacity value, I think emulate Windows behavior for these HDD docks will not be an easy task []'s Alfredo -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
From: Alan Stern st...@rowland.harvard.edu Anyway, I can try writing a patch to add this capability. We'll see if it can solve your problem. Unfortunately, I think there is genuine value in such a hack. E.g., I've got two USB-to-SATA adapters. One works correctly. One does not. But at this point, I can't attach both of my high-capacity disks to the laptop at the same time, which makes it difficult to bulk-transfer files from one to the other. If, for the deficient adapter, the kernel assumed the disk size was the end of the last partition (or the backup GPT partition table, if that follows), then I could use it for everything other than repartitioning. And that would make life easier. Thanks for trying, Dale -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Wed, 2014-09-03 at 15:05 -0400, Alan Stern wrote: On Wed, 3 Sep 2014, Dale R. Worley wrote: From: Alan Stern st...@rowland.harvard.edu On Fri, 29 Aug 2014, Matthew Dharm wrote: Is there an 'easy' way to override the detected size of a storage device from userspace? If we had that, someone could write a helper application which looked for this particular fubar and try to Do The Right Thing(tm), or at least offer the user some options. You mean, force a Media Change event and override the capacity reported by the hardware? I'm not aware of any API for doing that, although it probably wouldn't be too hard to add one. How would the user know what value to put in for the capacity? Unless the drive had been hooked up to a different computer and the user manually noted the correct capacity and typed it in, it would have to be guesswork. The value would most sanely be extracted from the partition table. (After verifying that the partition table looks correct.) That seems to be what Windows does, and it seems to work consistently enough for Windows to trust that method. Or at least, it could take the disk size to be the end of the last partition, which would at least make all the partitions accessible. If there is a partition table. It might be worthwhile to try an ATA pass-through command as well. As somebody else hinted at, the userspace program could check the USB device against a list of device types known to have this problem. It could even verify that the SCSI-reported size matches the size reported by the partition table (modulo two-to-the-whatever) (at least for GPT tables, I don't know if MBR tables report the disk size). They don't. Just the start and end of each partition. Do we have any way of knowing what algorithm Windows uses in this situation? Ask Microsoft. I suspect you're not likely to get an answer, though. Anyway, I can try writing a patch to add this capability. We'll see if it can solve your problem. Before we embark on elaborate hacks, why don't we just make the capacity writeable (by root) in sysfs from userspace (will require block change)? We can then encode all the nasty heuristics (including gpt reading) in userspace as a udev rule. James -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Wed, 3 Sep 2014, James Bottomley wrote: Before we embark on elaborate hacks, why don't we just make the capacity writeable (by root) in sysfs from userspace (will require block change)? We can then encode all the nasty heuristics (including gpt reading) in userspace as a udev rule. That's what I'm working on. Except that I don't know where to do it in the block layer, so for now I'm adding the attribute to sd.c. Where in the block layer would the right place be? We want this to apply only to entire devices, not to individual partitions. Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Wed, 2014-09-03 at 16:30 -0400, Alan Stern wrote:
On Wed, 3 Sep 2014, James Bottomley wrote:
Before we embark on elaborate hacks, why don't we just make the capacity
writeable (by root) in sysfs from userspace (will require block change)?
We can then encode all the nasty heuristics (including gpt reading) in
userspace as a udev rule.
That's what I'm working on. Except that I don't know where to do it in
the block layer, so for now I'm adding the attribute to sd.c.
Where in the block layer would the right place be? We want this to
apply only to entire devices, not to individual partitions.
The bottom layer for this is part0.nr_sects which is the size attribute
you see in the block sysfs. However, it looks like we keep a separate
value in sdkp, but we don't ever seem to use it (except to see if the
capacity has changed).
So this could be done in two ways: add a writeable capacity attribute in
sd.c, as you were originally thinking (it would call set_capacity() on
write and that would update the block layer) or make the size parameter
writeable.
This is how you would do the block layer one below, but there's no way
to inform the lower layer (so it better not have any information cached
that it makes use of).
James
---
diff --git a/block/genhd.c b/block/genhd.c
index 791f419..a114636 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -980,7 +980,7 @@ static DEVICE_ATTR(range, S_IRUGO, disk_range_show, NULL);
static DEVICE_ATTR(ext_range, S_IRUGO, disk_ext_range_show, NULL);
static DEVICE_ATTR(removable, S_IRUGO, disk_removable_show, NULL);
static DEVICE_ATTR(ro, S_IRUGO, disk_ro_show, NULL);
-static DEVICE_ATTR(size, S_IRUGO, part_size_show, NULL);
+static DEVICE_ATTR(size, S_IRUGO|S_IWUSR, part_size_show, part_size_store);
static DEVICE_ATTR(alignment_offset, S_IRUGO, disk_alignment_offset_show,
NULL);
static DEVICE_ATTR(discard_alignment, S_IRUGO, disk_discard_alignment_show,
NULL);
diff --git a/block/partition-generic.c b/block/partition-generic.c
index 789cdea..d0cc418 100644
--- a/block/partition-generic.c
+++ b/block/partition-generic.c
@@ -87,6 +87,20 @@ ssize_t part_size_show(struct device *dev,
return sprintf(buf, %llu\n,(unsigned long long)part_nr_sects_read(p));
}
+ssize_t part_size_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct hd_struct *p = dev_to_part(dev);
+ u64 size;
+
+ if (count 0 sscanf(buf, %llu, size) 0)
+ part_nr_sects_write(p, size);
+ else
+ return -EINVAL;
+
+ return count;
+}
+
static ssize_t part_ro_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
diff --git a/include/linux/genhd.h b/include/linux/genhd.h
index ec274e0..c9b3473 100644
--- a/include/linux/genhd.h
+++ b/include/linux/genhd.h
@@ -628,6 +628,9 @@ extern void blk_unregister_region(dev_t devt, unsigned long
range);
extern ssize_t part_size_show(struct device *dev,
struct device_attribute *attr, char *buf);
+extern ssize_t part_size_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count);
extern ssize_t part_stat_show(struct device *dev,
struct device_attribute *attr, char *buf);
extern ssize_t part_inflight_show(struct device *dev,
--
To unsubscribe from this list: send the line unsubscribe linux-usb in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Sat, 30 Aug 2014, Douglas Gilbert wrote: On 14-08-30 05:15 PM, Alan Stern wrote: On Fri, 29 Aug 2014, Matthew Dharm wrote: Is there an 'easy' way to override the detected size of a storage device from userspace? If we had that, someone could write a helper application which looked for this particular fubar and try to Do The Right Thing(tm), or at least offer the user some options. You mean, force a Media Change event and override the capacity reported by the hardware? I'm not aware of any API for doing that, although it probably wouldn't be too hard to add one. How would the user know what value to put in for the capacity? Unless the drive had been hooked up to a different computer and the user manually noted the correct capacity and typed it in, it would have to be guesswork. Might another possibility be using the SAT layer to issue the appropriate ATA command via the SCSI ATA PASS-THROUGH (12 or 16) command to find out the disk's size. That might work. Not all USB mass-storage devices support ATA pass-through but some of them do. This might be a possible strategy if READ CAPACITY(10) yields 0x for the last sector's LBA and the follow-up READ CAPACITY(16) fails or yields a truncated value. Yes. The problem in this case is that READ CAPACITY(10) yields a reasonable value (not 0x), so READ CAPACITY(16) never gets sent. And if it was sent, the device wouldn't handle it anyway. BTW Been looking at a USB-to-SATA adapter that uses the UAS(P) transport. I thought nothing could have worse SCSI compliance than USB mass storage devices. I was wrong ... Well, a USB-to-SATA adapter _is_ a USB mass-storage device. What vendor and product? Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Fri, 29 Aug 2014, Matthew Dharm wrote: Is there an 'easy' way to override the detected size of a storage device from userspace? If we had that, someone could write a helper application which looked for this particular fubar and try to Do The Right Thing(tm), or at least offer the user some options. You mean, force a Media Change event and override the capacity reported by the hardware? I'm not aware of any API for doing that, although it probably wouldn't be too hard to add one. How would the user know what value to put in for the capacity? Unless the drive had been hooked up to a different computer and the user manually noted the correct capacity and typed it in, it would have to be guesswork. Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On 14-08-30 05:15 PM, Alan Stern wrote: On Fri, 29 Aug 2014, Matthew Dharm wrote: Is there an 'easy' way to override the detected size of a storage device from userspace? If we had that, someone could write a helper application which looked for this particular fubar and try to Do The Right Thing(tm), or at least offer the user some options. You mean, force a Media Change event and override the capacity reported by the hardware? I'm not aware of any API for doing that, although it probably wouldn't be too hard to add one. How would the user know what value to put in for the capacity? Unless the drive had been hooked up to a different computer and the user manually noted the correct capacity and typed it in, it would have to be guesswork. Might another possibility be using the SAT layer to issue the appropriate ATA command via the SCSI ATA PASS-THROUGH (12 or 16) command to find out the disk's size. This might be a possible strategy if READ CAPACITY(10) yields 0x for the last sector's LBA and the follow-up READ CAPACITY(16) fails or yields a truncated value. Doug Gilbert BTW Been looking at a USB-to-SATA adapter that uses the UAS(P) transport. I thought nothing could have worse SCSI compliance than USB mass storage devices. I was wrong ... -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Sat, Aug 30, 2014 at 2:15 PM, Alan Stern st...@rowland.harvard.edu wrote: On Fri, 29 Aug 2014, Matthew Dharm wrote: Is there an 'easy' way to override the detected size of a storage device from userspace? If we had that, someone could write a helper application which looked for this particular fubar and try to Do The Right Thing(tm), or at least offer the user some options. You mean, force a Media Change event and override the capacity reported by the hardware? I'm not aware of any API for doing that, although it probably wouldn't be too hard to add one. How would the user know what value to put in for the capacity? Unless the drive had been hooked up to a different computer and the user manually noted the correct capacity and typed it in, it would have to be guesswork. I didn't say it would be easy to figure out the right value, but at least it would be possible. I was thinking of something that could notice a USB device which is formatted NTFS and has a partition table and filesystem that indicates a much bigger capacity than what the drive reports. Under this circumstances, you could do something like pop-up a dialog box saying this drive is confused -- is it 2TB or 3TB? Well, maybe that would say Drive capacity is not consistent with partition table. This can happen with certain USB drives designed for use with Windows. Override drive capacity (emulating Windows)? You could imagine increasing complex heuristics to try to detect this scenario. Even without an automated helper program to do it, if there was a sysfs interface then when we got the periodic e-mails reporting this same type of problem, we could offer a quick-and-clean solution. Matt -- Matthew Dharm Maintainer, USB Mass Storage driver for Linux -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
From: Alan Stern st...@rowland.harvard.edu If you try to repartition the drive under Windows using the deficient adapter, you'll see that the problem still exists. It just doesn't show up during normal use. So in summary, the Windows workaround is icky, but it allows any use but repartitioning to be one on the attached disk. Dale -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
Is there an 'easy' way to override the detected size of a storage device from userspace? If we had that, someone could write a helper application which looked for this particular fubar and try to Do The Right Thing(tm), or at least offer the user some options. Matt On Fri, Aug 29, 2014 at 2:07 PM, Dale R. Worley wor...@alum.mit.edu wrote: From: Alan Stern st...@rowland.harvard.edu If you try to repartition the drive under Windows using the deficient adapter, you'll see that the problem still exists. It just doesn't show up during normal use. So in summary, the Windows workaround is icky, but it allows any use but repartitioning to be one on the attached disk. Dale -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- Matthew Dharm Maintainer, USB Mass Storage driver for Linux -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Tue, 2014-08-26 at 10:47 -0400, Alan Stern wrote: On Mon, 25 Aug 2014, Oliver Neukum wrote: Just set US_FL_NEEDS_CAP16. If READ CAPACITY(16) fails in that case, it is clear that something is wrong. It must be set or READ CAPACITY(10) alone would be taken as giving a valid answer. You have committed a mental layering violation. :-) Indeed. US_FL_NEEDS_CAP16 is a usb-storage flag. But the capacity determination is made by the sd driver, which relies on the SCSI try_rc_10_first flag. If that flag isn't set, sd tries READ CAPACITY(16) and then falls back to READ CAPACITY(10) if an error occurs. That's what happened here. I don't think we want to add another SCSI flag to say that READ CAPACITY(10) is unreliable. Why not? It would only be friendly to tell the upper layer of a malfunction if we know about it. At that time we are sure that the drive will be unusable unless we determine the correct capacity, so we don't make matters worse if we crash it. Given the difficulty of determining the true capacity, I see two alternatives. We could set the capacity to a ridiculously large value (like 1 billion TB), or we could leave the capacity set to the low value and disable the block within bounds checks. Neither of these is attractive and they both have issues of their own -- although the second is close to what Windows does. That seems to be the most attractive solution to me. (For example, udev often tries to read the last sectors of a new drive, looking for a GPT or RAID signature. That won't work if the capacity is set to some random value.) Yes, but clipping has its own dangers. Suppose you use the medium without a partition table. Regards Oliver -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Wed, 27 Aug 2014, Oliver Neukum wrote: I don't think we want to add another SCSI flag to say that READ CAPACITY(10) is unreliable. Why not? It would only be friendly to tell the upper layer of a malfunction if we know about it. To what end? What will the upper layer do with this information? Given the difficulty of determining the true capacity, I see two alternatives. We could set the capacity to a ridiculously large value (like 1 billion TB), or we could leave the capacity set to the low value and disable the block within bounds checks. Neither of these is attractive and they both have issues of their own -- although the second is close to what Windows does. That seems to be the most attractive solution to me. I'm skeptical that you can convince the SCSI and block-layer developers about this. Maybe they'll accept it if it is applied only to USB transports... (For example, udev often tries to read the last sectors of a new drive, looking for a GPT or RAID signature. That won't work if the capacity is set to some random value.) Yes, but clipping has its own dangers. Suppose you use the medium without a partition table. What would Windows do? In the absence of a partition table, it would believe the value from READ CAPACITY, right? Isn't that the same as clipping? Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
From: James Bottomley james.bottom...@hansenpartnership.com Did you try read capacity 16 on it? What happened? (the AS2105 rejects read capacity 16, so there's no reliable way to deduce the capacity of drives over 2TB). OK, I had to track down which package contains sg_readcap. The adapter that fails gives this output: # sg_readcap --16 /dev/sdb bad field in READ CAPACITY (16) cdb including unsupported service action The adapter that succeeds gives this output: # sg_readcap --16 /dev/sdc Read Capacity results: Protection: prot_en=0, p_type=0, p_i_exponent=0 Logical block provisioning: lbpme=0, lbprz=0 Last logical block address=5860533167 (0x15d50a3af), Number of logical blocks=5860533168 Logical block length=512 bytes Logical blocks per physical block exponent=0 Lowest aligned logical block address=0 Hence: Device size: 3000592982016 bytes, 2861588.5 MiB, 3000.59 GB Dale -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
What I find interesting is that Windows (at least, Windows 7 Professional) seems to be able to handle the deficient adapter. What I'd like to do is log the disk commands during the mounting sequence, preferably at both the SCSI and USB layers. Then at least we'll know exactly what the driver is doing. Are there kernel options to do this? Unfortunately, I don't know any way to log what Windows is doing with the drive. Dale -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Wed, 2014-08-27 at 15:21 -0400, Dale R. Worley wrote: From: James Bottomley james.bottom...@hansenpartnership.com Did you try read capacity 16 on it? What happened? (the AS2105 rejects read capacity 16, so there's no reliable way to deduce the capacity of drives over 2TB). OK, I had to track down which package contains sg_readcap. The adapter that fails gives this output: # sg_readcap --16 /dev/sdb bad field in READ CAPACITY (16) cdb including unsupported service action OK, so that's definitely the tame taxonomy. James -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Wed, 27 Aug 2014, Dale R. Worley wrote: What I find interesting is that Windows (at least, Windows 7 Professional) seems to be able to handle the deficient adapter. So does Linux. The difference is that Windows believes the values in the partition table in preference to what the hardware says, whereas Linux believes the hardware in preference to the partition table. Thus, if the hardware says the disk contains 0.8 TB and the partition table says the first partition contains 2.8 TB, Windows will try to access all 2.8 TB but Linux will complain that the partition entry is invalid (because the partition extends beyond the end of the disk). If you try to repartition the drive under Windows using the deficient adapter, you'll see that the problem still exists. It just doesn't show up during normal use. What I'd like to do is log the disk commands during the mounting sequence, preferably at both the SCSI and USB layers. Then at least we'll know exactly what the driver is doing. Are there kernel options to do this? You can record the USB transfers by using usbmon (see Documentation/usb/usbmon.txt). The trace will include all the important SCSI data, so you don't need to record anything at the SCSI layer. This isn't really necessary, though. We already know what the driver is doing. Unfortunately, I don't know any way to log what Windows is doing with the drive. My experiments with Windows show that it does essentially the same things as Linux does. The important difference is not what commands and data get sent, but whether the OS pays attention to the result. Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
From: Alan Stern On Mon, 25 Aug 2014, Alfredo Dal Ava Junior wrote: Well, it is causing problems anyway... from user perspective, it's a Linux compatibility issue, as it works fine on Windows. I'm not an expert, but I'm wondering that if usb-storage could set capacity as UNDETERMINED/ Zero (or keep using the readcapacity_10 as it as with some flag signalizing it as inaccurate), EFI partition check would be able to ignore size and look for secondary GPT where it really is. Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Could the code try READ CAPACITY(16) first? Really there is only one way to know whether the actual capacity is larger than the reported capacity, and that is by trying to read blocks beyond the reported capacity -- a dangerous test that many drives do not like. (And in most cases a futile test. If a device doesn't support READ CAPACITY(16), how likely is it to support READ(16)?) Yes, in theory you can believe the value in the partition table in preference to the reported capacity. But what if that value is wrong? And how do you tell partition-manager programs what the capacity should be when they modify or set up the initial partition table? I've a feeling that, historically at least, windows believes the partition table. I remember some CF cards that locked up when I tried to read the 'device info' sector, and others (apparently identical) that had the 32bit sector size misaligned. These were 'major manufacturer' cards as well. David -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Tue, 2014-08-26 at 09:58 +, David Laight wrote: Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Could the code try READ CAPACITY(16) first? Yes. It does already. That fails as the device doesn't support this version. In a way we are discussing error handling here. Regards Oliver -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
From Oliver Neukum [mailto:oneu...@suse.de]
On Tue, 2014-08-26 at 09:58 +, David Laight wrote:
Part of the problem is that usb-storage has no way to know that
anything strange is going on. It's normal for READ CAPACITY(16) to
fail (this depend on the SCSI level), and it's normal for the READ
CAPACITY(10) to report a value less than 2 TB.
Could the code try READ CAPACITY(16) first?
Yes. It does already. That fails as the device doesn't support
this version. In a way we are discussing error handling here.
I read more of the thread later (getting outluck to sort mails in
any sensible way is almost impossible.)
I'm sort of surprised that the 16byte reads work if the 16byte
read capacity doesn't.
I wonder what the manufacturer would saw in response the bug where
windows shows the incorrect size when trying to partition the disk?
Such bugs ought to fail 'fitness for purpose' - so, in the UK, the
shop would have to replace the 'faulty' goods.
David
N�r��yb�X��ǧv�^�){.n�+{��^n�r���z���h��� �G���h�(�階�ݢj���m��z�ޖ���f���h���~�m�
RES: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 26 Aug 2014, David Leight wrote: I wonder what the manufacturer would saw in response the bug where windows shows the incorrect size when trying to partition the disk? I contacted enclosure manufacturer (Welland) some weeks ago, they are supposed to escalate my questions to engineering and I'm still waiting for reply. Interesting thing is that I read AS1051 on the chip, but it is detected as AS2105 (maybe same VID and PID for different chips). I contacted ASMedia (chip bridge manufacturer), they replied that ASM1051 supports 48bit LBAs and that some FW feature and FW SPEC depends on manufacture's request. They added that this chip entered EOL this year and I have to contact Welland for support. Anyway, it show explain way we see distinct behavior for same the chip (and same VIDPID). []'s Alfredo
Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
This is almost certainly a form of the problem reported in AS2105-based enclosure size issues with 2TB HDDs. I'm repeating my original message here so linux-usb can see it, and so it can be connected to the older thread. I'll address it in another message. I've appended James Bottomley's response, to avoid creating an additional message. Dale -- Date: Wed, 20 Aug 2014 16:18:00 -0400 From: wor...@alum.mit.edu (Dale R. Worley) To: linux-s...@vger.kernel.org Subject: Problem with USB-to-SATA adapters I don't know if this is the correct place for this problem, but you people can probably tell me the correct place. I have two USB to SATA adapter dongles. In general, they work fine. However I've discovered that one of them handles large (1 TB and above) drives correctly and one does not. In particular, when I use the unsuccessful one, neither fdisk nor gdisk reports the disk size correctly, and the kernel does not read the partition table. More confusingly, both dongles work correctly with large disks on Windows 7 Professional. Both of the following cases are when a 3.0 TB drive is attached. The successful device has the Diablo brand. The most interesting messages in the log when I plug in the USB are (the complete message set is appended below): Aug 18 21:56:53 hobgoblin kernel: [ 294.229462] usb 2-1.3: New USB device found, idVendor=152d, idProduct=2338 Aug 18 21:56:53 hobgoblin kernel: [ 294.229475] usb 2-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=5 Aug 18 21:56:53 hobgoblin kernel: [ 294.229482] usb 2-1.3: Product: USB to ATA/ATAPI bridge Aug 18 21:56:53 hobgoblin kernel: [ 294.229488] usb 2-1.3: Manufacturer: JMicron Aug 18 21:56:53 hobgoblin kernel: [ 294.229495] usb 2-1.3: SerialNumber: 01D91CC4 Aug 18 21:56:54 hobgoblin kernel: [ 295.236576] sd 7:0:0:0: [sdb] Very big device. Trying to use READ CAPACITY(16). Aug 18 21:56:54 hobgoblin kernel: [ 295.236955] sd 7:0:0:0: [sdb] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB) The unsuccessful device has the Ez-Connect brand. The most interesting messages in the log when I plug in the USB are (the complete message set is appended below): Aug 18 21:54:06 hobgoblin kernel: [ 127.674374] usb 2-2: new high-speed USB device number 4 using ehci-pci Aug 18 21:54:06 hobgoblin kernel: [ 127.791128] usb 2-2: New USB device found, idVendor=05e3, idProduct=0718 Aug 18 21:54:06 hobgoblin kernel: [ 127.791135] usb 2-2: New USB device strings: Mfr=0, Product=1, SerialNumber=2 Aug 18 21:54:06 hobgoblin kernel: [ 127.791138] usb 2-2: Product: USB Storage Aug 18 21:54:06 hobgoblin kernel: [ 127.791142] usb 2-2: SerialNumber: 0033 Aug 18 21:54:07 hobgoblin kernel: [ 128.847269] sd 6:0:0:0: [sdb] 1565565872 512-byte logical blocks: (801 GB/746 GiB) Dale -- For the Diablo adapter: Aug 18 21:56:53 hobgoblin kernel: [ 294.153343] usb 2-1.3: new high-speed USB device number 5 using ehci-pci Aug 18 21:56:53 hobgoblin kernel: [ 294.229462] usb 2-1.3: New USB device found, idVendor=152d, idProduct=2338 Aug 18 21:56:53 hobgoblin kernel: [ 294.229475] usb 2-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=5 Aug 18 21:56:53 hobgoblin kernel: [ 294.229482] usb 2-1.3: Product: USB to ATA/ATAPI bridge Aug 18 21:56:53 hobgoblin kernel: [ 294.229488] usb 2-1.3: Manufacturer: JMicron Aug 18 21:56:53 hobgoblin kernel: [ 294.229495] usb 2-1.3: SerialNumber: 01D91CC4 Aug 18 21:56:53 hobgoblin kernel: [ 294.231159] usb-storage 2-1.3:1.0: USB Mass Storage device detected Aug 18 21:56:53 hobgoblin kernel: [ 294.233951] scsi7 : usb-storage 2-1.3:1.0 Aug 18 21:56:53 hobgoblin mtp-probe: checking bus 2, device 5: /sys/devices/pci:00/:00:1d.7/usb2/2-1/2-1.3 Aug 18 21:56:53 hobgoblin mtp-probe: bus: 2, device: 5 was not an MTP device Aug 18 21:56:54 hobgoblin kernel: [ 295.235735] scsi 7:0:0:0: Direct-Access WDC WD30 EZRX-00SPEB0 PQ: 0 ANSI: 5 Aug 18 21:56:54 hobgoblin kernel: [ 295.236209] sd 7:0:0:0: Attached scsi generic sg2 type 0 Aug 18 21:56:54 hobgoblin kernel: [ 295.236576] sd 7:0:0:0: [sdb] Very big device. Trying to use READ CAPACITY(16). Aug 18 21:56:54 hobgoblin kernel: [ 295.236955] sd 7:0:0:0: [sdb] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB) Aug 18 21:56:54 hobgoblin kernel: [ 295.237957] sd 7:0:0:0: [sdb] Write Protect is off Aug 18 21:56:54 hobgoblin kernel: [ 295.238957] sd 7:0:0:0: [sdb] No Caching mode page found Aug 18 21:56:54 hobgoblin kernel: [ 295.238962] sd 7:0:0:0: [sdb] Assuming drive cache: write through Aug 18 21:56:54 hobgoblin kernel: [ 295.239956] sd 7:0:0:0: [sdb] Very big device. Trying to use READ CAPACITY(16). Aug 18 21:56:54 hobgoblin kernel: [ 295.242334] sd 7:0:0:0: [sdb] No Caching mode page found Aug 18 21:56:54 hobgoblin kernel: [ 295.242339] sd 7:0:0:0: [sdb] Assuming drive cache: write through Aug 18 21:56:54 hobgoblin kernel
Re: Problem with USB-to-SATA adapters (was: AS2105-based enclosure size issues with 2TB HDDs)
On Tue, 2014-08-26 at 15:39 -0400, Dale R. Worley wrote: This is almost certainly a form of the problem reported in AS2105-based enclosure size issues with 2TB HDDs. I'm repeating my original message here so linux-usb can see it, and so it can be connected to the older thread. I'll address it in another message. Did you try read capacity 16 on it? What happened? (the AS2105 rejects read capacity 16, so there's no reliable way to deduce the capacity of drives over 2TB). James -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
AS2105-based enclosure size issues with 2TB HDDs
Hi all, I'm investigating an issue with a HDD enclosure based on AS2105 chip. A 4TB GPT partition is not considered valid because reported last LBA reported by the enclosure makes kernel think that secondary GPT is outside disk plate. I found this patch [1] forcing it to use READ_CAPACITY_16 first and fall back to READ_CAPACITY_10 if no success. This enclosure has the same vendor and product IDs, but behavior is a bit different: READ_CAPACITY_16 fails 100% of times as unsupported command. READ_CAPACITY_10 has a distinct behavior depending on HDD size: - 1TB and 2TB: READ_CAPACITY_10 returns correct size value - 3TB and 4TB: READ_CAPACITY_10 returns size in a 2TB modulus If we fix capacity size by reporting (READ_CAPACITY_10 + MODULO_2TB), the result will be invalid when user plugs a 2TB HDD. An idea (bring by Oliver) is: first guess reading last sector using modulus result to check if size is valid. Any other ideas? There is better way to detect if enclosure is returning real LBA capacity or a modulo 2TB result? Thanks, Alfredo Dal'Ava Júnior [1] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/usb/storage?id=32c37fc30c52508711ea6a108cfd5855b8a07176 -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 2014-08-25 at 10:58 +, Alfredo Dal Ava Junior wrote:
- 1TB and 2TB: READ_CAPACITY_10 returns correct size value
- 3TB and 4TB: READ_CAPACITY_10 returns size in a 2TB modulus
If we fix capacity size by reporting (READ_CAPACITY_10 + MODULO_2TB), the
result
will be invalid when user plugs a 2TB HDD. An idea (bring by Oliver) is:
It is worse than that. Pretty soon a 4.7TB disk will also be plausible.
first guess reading last sector using modulus result to check if size is
valid.
It is necessary that a virgin disk also be handled correctly.
We cannot use the partition table (besides it being a layering
violation)
It would propose (in pseudo code)
if (read_capacity_16(device) 0) {
lower_limit = read_capacity_10(device);
for (i = 1; i++; i SANITY_LIMIT) {
err = read_sector(device, lower_limit + i * 2TB-SAFETY);
if (err == OUT_OF_RANGE)
break;
}
if (i SANITY_LIMIT)
return (i - 1) * 2TB + lower_limit;
else
return ERROR;
Regards
Oliver
--
To unsubscribe from this list: send the line unsubscribe linux-usb in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 2014-08-25 at 18:48 +, Alfredo Dal Ava Junior wrote: On Mon, 25 Aug 2014, Alan Stern wrote: Don't forget that lots of disks go crazy if you try to read from a nonexistent block, that is, one beyond the end of the disk. IMO, this bug cannot be worked around in any reasonable manner. The device simply cannot handle disks larger than 2 TB. This device works well on Windows 7 if HDD is already partitioned. So how did the partition get on there at the correct size in the first place? Even under windows partition managers believe the disk size they get from the system if the disk is blank. Sounds like Win7 gnores the READ_CAPACITY value on a partitioned HDD. It shows 4TB on disk manager, but will fall back to 1.8TB if I remove the partition. I assume for those of us on linux-scsi who don't have the start of this thread, you already tried read capacity(16) and it has this same problem? Could we do the same? Hm, allowing users to set desired capacity by overriding the partition size ... I'm sure that's not going to cause support problems ... Would be possible to signalize to upper layers that capacity is not accurate (or return zero) on this device, and tell GPT handlers to bypass it's partition_size vs disk size consistency check? If we can find a heuristic to set the correct capacity in the kernel, then we may be able to fix this ... but as Alain says, it looks hard. We can't ask userspace to hack tools for broken devices. James -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 25 Aug 2014, Alfredo Dal Ava Junior wrote: On Mon, 25 Aug 2014, Alan Stern wrote: Don't forget that lots of disks go crazy if you try to read from a nonexistent block, that is, one beyond the end of the disk. IMO, this bug cannot be worked around in any reasonable manner. The device simply cannot handle disks larger than 2 TB. This device works well on Windows 7 if HDD is already partitioned. Sounds like Win7 gnores the READ_CAPACITY value on a partitioned HDD. It shows 4TB on disk manager, but will fall back to 1.8TB if I remove the partition. That's right. I don't know why Windows behaves that way. Could we do the same? Would be possible to signalize to upper layers that capacity is not accurate (or return zero) on this device, and tell GPT handlers to bypass it's partition_size vs disk size consistency check? There is no way to do this, as far as I know. But I'm not an expert in this area. Maybe you can figure out a way to add this capability. (But then what happens if the size stored in the partition table really is larger than the disk's capacity?) Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 15 Aug 2014 James Bottomley wrote: So how did the partition get on there at the correct size in the first place? Even under windows partition managers believe the disk size they get from the system if the disk is blank. The HDD can be partitioned outside the enclosure, when connected directly to one SATA port on motherboard. READ_CAPACITY(16) will return properly when talking directly to the HDD. I assume for those of us on linux-scsi who don't have the start of this thread, you already tried read capacity(16) and it has this same problem? Sorry, I forgot to include linux-scsi. On this device, READ_CAPACITY_16 fails 100% of times as unsupported command, then READ_CAPACITY_10 has a distinct behavior depending on HDD size: 1TB and 2TB: READ_CAPACITY_10 returns correct value 3TB and 4TB: READ_CAPACITY_10 returns in a 2TB modulus Hm, allowing users to set desired capacity by overriding the partition size ... I'm sure that's not going to cause support problems ... Well, it is causing problems anyway... from user perspective, it's a Linux compatibility issue, as it works fine on Windows. I'm not an expert, but I'm wondering that if usb-storage could set capacity as UNDETERMINED/ Zero (or keep using the readcapacity_10 as it as with some flag signalizing it as inaccurate), EFI partition check would be able to ignore size and look for secondary GPT where it really is. []'s Alfredo -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 25 Aug 2014 Alan Stern wrote:
On Mon, 25 Aug 2014, Alfredo Dal Ava Junior wrote:
That's right. I don't know why Windows behaves that way.
Please look this output from diskpart (Windows):
DISKPART list partition
Partition ### Type Size Offset
- --- ---
Partition 1Primary 3726 GB17 KB
DISKPART list disk
Disk ### Status Size Free Dyn Gpt
- --- --- --- ---
Disk 0Online 298 GB 0 B
* Disk 1Online 1678 GB 0 B*
DISKPART select disk 1
Disk 1 is now the selected disk.
DISKPART list partition
Partition ### Type Size Offset
- --- ---
Partition 1Primary 3726 GB17 KB
Could we do the same? Would be possible to signalize to upper layers
that capacity is not accurate (or return zero) on this device, and
tell GPT handlers to bypass it's partition_size vs disk size
consistency check?
There is no way to do this, as far as I know. But I'm not an expert in this
area.
Maybe you can figure out a way to add this capability.
(But then what happens if the size stored in the partition table really is
larger
than the disk's capacity?)
It's the first time I touch this code, but I will learn from the code and try
to find it out... but still in hope to find a clean solution...
If size stored on partition table is really larger than disk, my guess it will
cause I/O errors, and that some disks may get crazy like you pointed.
Do you think disk could cause kernel instability? I think it is acceptable if
no consequences to kernel stability, since it is a specific-device
workaround.
[]'s
Alfredo
--
To unsubscribe from this list: send the line unsubscribe linux-usb in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 25 Aug 2014, Alfredo Dal Ava Junior wrote: Well, it is causing problems anyway... from user perspective, it's a Linux compatibility issue, as it works fine on Windows. I'm not an expert, but I'm wondering that if usb-storage could set capacity as UNDETERMINED/ Zero (or keep using the readcapacity_10 as it as with some flag signalizing it as inaccurate), EFI partition check would be able to ignore size and look for secondary GPT where it really is. Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Really there is only one way to know whether the actual capacity is larger than the reported capacity, and that is by trying to read blocks beyond the reported capacity -- a dangerous test that many drives do not like. (And in most cases a futile test. If a device doesn't support READ CAPACITY(16), how likely is it to support READ(16)?) Yes, in theory you can believe the value in the partition table in preference to the reported capacity. But what if that value is wrong? And how do you tell partition-manager programs what the capacity should be when they modify or set up the initial partition table? Attaching the drive over a SATA connection when you want to partition it isn't a very satisfactory solution. After all, if you have a SATA connection available then why would you be using a USB enclosure in the first place? Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RES: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 25 Aug 2014 Alan Stern wrote: Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Really there is only one way to know whether the actual capacity is larger than the reported capacity, and that is by trying to read blocks beyond the reported capacity -- a dangerous test that many drives do not like. (And in most cases a futile test. If a device doesn't support READ CAPACITY(16), how likely is it to support READ(16)?) Yes, in theory you can believe the value in the partition table in preference to the reported capacity. But what if that value is wrong? And how do you tell partition-manager programs what the capacity should be when they modify or set up the initial partition table? We may add this device to UNUSUAL_DEV list, to keep using READ_CAPACITY(10) and pass some flag to bypass EFI GPT partition check. I'm almost sure that kernel will be able to mount the partition if we can make it available on /proc/partition. This would make this device behaves like it does on Windows 7. I see this as best effort workaround for a cheap buggy hardware, like many others on UNUSUAL_DEV list Attaching the drive over a SATA connection when you want to partition it isn't a very satisfactory solution. After all, if you have a SATA connection available then why would you be using a USB enclosure in the first place? You may want do a backup or plug it in a laptop, for example. []'s Alfredo -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RES: RES: AS2105-based enclosure size issues with 2TB HDDs
On Mon, 2014-08-25 at 16:21 -0400, Alan Stern wrote: On Mon, 25 Aug 2014, Alfredo Dal Ava Junior wrote: Well, it is causing problems anyway... from user perspective, it's a Linux compatibility issue, as it works fine on Windows. I'm not an expert, but I'm wondering that if usb-storage could set capacity as UNDETERMINED/ Zero (or keep using the readcapacity_10 as it as with some flag signalizing it as inaccurate), EFI partition check would be able to ignore size and look for secondary GPT where it really is. Part of the problem is that usb-storage has no way to know that anything strange is going on. It's normal for READ CAPACITY(16) to fail (this depend on the SCSI level), and it's normal for the READ CAPACITY(10) to report a value less than 2 TB. Just set US_FL_NEEDS_CAP16. If READ CAPACITY(16) fails in that case, it is clear that something is wrong. It must be set or READ CAPACITY(10) alone would be taken as giving a valid answer. At that time we are sure that the drive will be unusable unless we determine the correct capacity, so we don't make matters worse if we crash it. Is there an easy way for Alfredo to determine what happens if we read beyond the end? Regards Oliver -- To unsubscribe from this list: send the line unsubscribe linux-usb in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
