Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Evan Leibovitch via talk]

Resolved.

$20 and a drive to CC (closest to me is the one near Vaughan Mills) for the
new enclosure solved the problem. All now looks as it should.
It was a bit of a zoo there, but obviously worth it. And the new enclosure
doesn't need screws or tools, which is a bonus.

Thanks to all who helped.

As mentioned, the old case is available to anyone who wants it (and can
pick it up at the next meeting).

- Evan


On Mon, 30 Dec 2019 at 08:35, Scott Sullivan  wrote:

> Although I don't have recent testing experince, after the whole 2TB
> 'thing', the marketing became about the largest drives tested at time of
> release.
>
> Someone else can look into this further but I think it was not handling
> Advanced Format Drives (4k sectors) that was the crux of limitation.
>
>
> On December 30, 2019 8:13:57 a.m. EST, Evan Leibovitch 
> wrote:
>>
>> Hi Scott (and everyone else who replied),
>>
>> As other on the thread have zero'ing in on, it is likely an issue with 
>> whatever
>>> USB/SATA adapter your using. This was a known limitation of early
>>> chipsets, and I recall the marketing shift to 'supports larger then
>>> 2TB!'.
>>>
>>
>> Pretty sure that's it. As I look around at what's available, even finding
>> one now that supports 10TB is not so easy, many top out at 8.
>> Since I can't yet put the drive in the chassis I have this enclosure
>> 
>> on order which will be useful later anyway.
>> Thanks for the feedback.
>>
>> Anyone want my existing enclosure? I can bring it to the next meeting.
>> You all know its limitations, but can't beat the price,
>>
>> --
>> Evan Leibovitch, Toronto Canada
>> @evanleibovitch or @el56
>>
>
> --
> Scott Sullivan
>


-- 
Evan Leibovitch, Toronto Canada
@evanleibovitch or @el56
---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Scott Sullivan via talk]

Although I don't have recent testing experince, after the whole 2TB 'thing', 
the marketing became about the largest drives tested at time of release.

Someone else can look into this further but I think it was not handling 
Advanced Format Drives (4k sectors) that was the crux of limitation.


On December 30, 2019 8:13:57 a.m. EST, Evan Leibovitch  wrote:
>Hi Scott (and everyone else who replied),
>
>As other on the thread have zero'ing in on, it is likely an issue with
>whatever
>> USB/SATA adapter your using. This was a known limitation of early
>> chipsets, and I recall the marketing shift to 'supports larger then
>2TB!'.
>>
>
>Pretty sure that's it. As I look around at what's available, even
>finding
>one now that supports 10TB is not so easy, many top out at 8.
>Since I can't yet put the drive in the chassis I have this enclosure
>
>on order which will be useful later anyway.
>Thanks for the feedback.
>
>Anyone want my existing enclosure? I can bring it to the next meeting.
>You
>all know its limitations, but can't beat the price,
>
>-- 
>Evan Leibovitch, Toronto Canada
>@evanleibovitch or @el56

-- 
Scott Sullivan---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Evan Leibovitch via talk]

Hi Scott (and everyone else who replied),

As other on the thread have zero'ing in on, it is likely an issue with whatever
> USB/SATA adapter your using. This was a known limitation of early
> chipsets, and I recall the marketing shift to 'supports larger then 2TB!'.
>

Pretty sure that's it. As I look around at what's available, even finding
one now that supports 10TB is not so easy, many top out at 8.
Since I can't yet put the drive in the chassis I have this enclosure

on order which will be useful later anyway.
Thanks for the feedback.

Anyone want my existing enclosure? I can bring it to the next meeting. You
all know its limitations, but can't beat the price,

-- 
Evan Leibovitch, Toronto Canada
@evanleibovitch or @el56
---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Nicholas Krause via talk]

On 12/29/19 12:02 PM, D. Hugh Redelmeier via talk wrote:

| From: Evan Leibovitch via talk 

| It's not an Xmas present as it was bought a few weeks ago, but I'm trying
| to install a 10TB WD (Red Pro) disk and it's not exactly going to plan.

Red Pro seem to be "top of the line".  I think that it doesn't use
SMR (good), but I don't know.

You don't say what your USB <-> SATA device is.  I think that matters.
But even if you did, they are often poorly documented with respect to
things that the manufacturers think might confuse users.  You appear
to be in that territory.

Your USB device is surely the reason that your computer sees 512 byte
sectors, even though your drive prefers/demands 4 KiB sectors.
Superstitiously, I wonder whether that alone might disturb the
geometry such that transferring the disk from the external to internal
would not work.

If your USS <-> SATA device is old, it might not even support sizes
over 2T.

(I have a couple of older NAS boxes that claim they only support SATA
drives up to 2T.)

Suggestions: one of

- Mount your drive in its ultimate destination (internal bay, with
   SATA connection) and set it up there.

- Mount your drive in a different computer (with a similar OS release)
   and set it up there.  The reason for the "similar OS release"
   suggestion is that filesystem details details might (but it isn't
   common).

- Buy a new (and well-documented) external USB<->SATA device



Western Digital called 4 KiB sectors "Advanced Formatting".  The
transition from 512 B sectors came about a decade ago, painfully.

During the transition, all sorts of hacks were used to paper over the
problem.  Some devices with 4 KiB partitions would pretend and fake
having 512 B partitions if they thought that the computer didn't
understand 4 KiB.  I remember all this faking causing its own set of
problems.

I seem to remember USB<->SATA devices presenting 512 B sectors on the
USB side, even if the drive uses 4 KiB sectors.  This might cause
problems (I don't know).

(Off topic: A single "real" blocksize for SSDs doesn't exist.  There
are several kinds of block sizes for a particular device.  All this is
hidden from the computer.  Too bad: the OS cannot optimize I/O
appropriately.)

That does not matter actually. The optimizations for most
file systems related to hard drives i.e. merging or reading
per block don't matter in a SSD. Brtfs just turns off optimizations
and there was discussion about direct IO being the default for
very fast SSDs actually. If your talking about writes, again that
is just trim or talking to firmware and does not matter unless
your in the embedded world with cheap NAND or data centers.
Most consumer SSDs do over 1000TBW so its not an issue for
most people.

Nick


The main point of Advanced Formatting was to increase density: each
sector has overhead on the disk platter and so reducing the number of
sectors reduced the overhead.

A second reason was that sector addressing of some kind was limited to
2T / 512 or 16T / 4096, so Advanced Formatting allowed a larger
capacity.  At least for MBR partitioning.  But we mostly use GPT
partitioning now, eliminating that problem (I think).

Does your computer use "legacy" booting or EFI?
---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk


---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[D. Hugh Redelmeier via talk]

| From: Evan Leibovitch via talk 

| It's not an Xmas present as it was bought a few weeks ago, but I'm trying
| to install a 10TB WD (Red Pro) disk and it's not exactly going to plan.

Red Pro seem to be "top of the line".  I think that it doesn't use
SMR (good), but I don't know.

You don't say what your USB <-> SATA device is.  I think that matters.
But even if you did, they are often poorly documented with respect to
things that the manufacturers think might confuse users.  You appear
to be in that territory.

Your USB device is surely the reason that your computer sees 512 byte
sectors, even though your drive prefers/demands 4 KiB sectors.
Superstitiously, I wonder whether that alone might disturb the
geometry such that transferring the disk from the external to internal
would not work.

If your USS <-> SATA device is old, it might not even support sizes
over 2T.

(I have a couple of older NAS boxes that claim they only support SATA
drives up to 2T.)

Suggestions: one of

- Mount your drive in its ultimate destination (internal bay, with
  SATA connection) and set it up there.

- Mount your drive in a different computer (with a similar OS release)
  and set it up there.  The reason for the "similar OS release"
  suggestion is that filesystem details details might (but it isn't
  common).

- Buy a new (and well-documented) external USB<->SATA device



Western Digital called 4 KiB sectors "Advanced Formatting".  The
transition from 512 B sectors came about a decade ago, painfully.

During the transition, all sorts of hacks were used to paper over the
problem.  Some devices with 4 KiB partitions would pretend and fake
having 512 B partitions if they thought that the computer didn't
understand 4 KiB.  I remember all this faking causing its own set of
problems.

I seem to remember USB<->SATA devices presenting 512 B sectors on the
USB side, even if the drive uses 4 KiB sectors.  This might cause
problems (I don't know).

(Off topic: A single "real" blocksize for SSDs doesn't exist.  There
are several kinds of block sizes for a particular device.  All this is
hidden from the computer.  Too bad: the OS cannot optimize I/O
appropriately.)

The main point of Advanced Formatting was to increase density: each
sector has overhead on the disk platter and so reducing the number of
sectors reduced the overhead.

A second reason was that sector addressing of some kind was limited to
2T / 512 or 16T / 4096, so Advanced Formatting allowed a larger
capacity.  At least for MBR partitioning.  But we mostly use GPT
partitioning now, eliminating that problem (I think).

Does your computer use "legacy" booting or EFI?
---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Evan Leibovitch via talk]

Hi Howard,

The motherboard is an Asus P8Z77-V LE PLUS
.
What am I looking for?
There are already two existing drives, a Seagate 4TB (ST4000DM000)and a WD
3TB (WD30EFRX), that have been working fine.

- Evan




On Sun, 29 Dec 2019 at 10:46, Howard Gibson via talk 
wrote:

> On Sun, 29 Dec 2019 00:05:17 -0500
> Evan Leibovitch via talk  wrote:
> >
> > The system already has an SSD as /dec/sda, and two disks one of 3TB and
> the
> > other of 4TB, all working fine.
> >
> > When I attach the external drive, lsblock and fdisk -l both report TWO
> new
> > drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set the disk
> > from DOS to GPT but the size didn't change;
>
> Evan,
>
>Is your motherboard MBR or GPT?  I have had a nasty experience with my
> old motherboard and a new Western Digital GPT formatted drive.
>
>http://home.eol.ca/~hgibson/RevLinux/#x1-8D
>
>I found that Western Digital GPT drives were difficult to use.  I was
> able to make Seagate drives work
>
> --
> Howard Gibson
> hgib...@eol.ca
> jhowardgib...@gmail.com
> http://home.eol.ca/~hgibson
> ---
> Post to this mailing list talk@gtalug.org
> Unsubscribe from this mailing list
> https://gtalug.org/mailman/listinfo/talk
>


-- 
Evan Leibovitch, Toronto Canada
@evanleibovitch or @el56
---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Howard Gibson via talk]

On Sun, 29 Dec 2019 00:05:17 -0500
Evan Leibovitch via talk  wrote:
> 
> The system already has an SSD as /dec/sda, and two disks one of 3TB and the
> other of 4TB, all working fine.
> 
> When I attach the external drive, lsblock and fdisk -l both report TWO new
> drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set the disk
> from DOS to GPT but the size didn't change;

Evan,

   Is your motherboard MBR or GPT?  I have had a nasty experience with my old 
motherboard and a new Western Digital GPT formatted drive.  

   http://home.eol.ca/~hgibson/RevLinux/#x1-8D

   I found that Western Digital GPT drives were difficult to use.  I was able 
to make Seagate drives work 

-- 
Howard Gibson 
hgib...@eol.ca
jhowardgib...@gmail.com
http://home.eol.ca/~hgibson
---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[James Knott via talk]

On 2019-12-29 12:05 AM, Evan Leibovitch via talk wrote:
When I attach the external drive, lsblock and fdisk -l both report TWO 
new drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set 
the disk from DOS to GPT but the size didn't change;


What does your mom board support?  Any limitation there?
---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Nicholas Krause via talk]

On 12/29/19 12:48 AM, Evan Leibovitch wrote:

Hrm.

For the 4TB disk in the same system (which has been working for 
years), fdisk -l reports:


Disk /dev/sdc: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 88C7DF91-CD80-4EEB-AC6B-110119B04DD4

Device     Start        End    Sectors  Size Type
/dev/sdc1   2048 7814035455 7814033408  3.7T Linux filesystem

Wondering if that helps. Starting to wonder if it's the firmware on 
the USB external chassis.
Maybe as those numbers look better on the physical side. Try seeing if 
connecting it directly
to a open SATA port and see if that gets you 512 bytes for logical and 
4096 for physical. I've
suspecting that may be it based on the internal drive you proved sector 
size numbers.


Nick



On Sun, 29 Dec 2019 at 00:42, Nicholas Krause > wrote:




On 12/29/19 12:19 AM, Evan Leibovitch wrote:



On Sun, 29 Dec 2019 at 00:08, Nicholas Krause
mailto:xerofo...@gmail.com>> wrote:


Your using fdisk right. There is a version for GPT disks
called gdisk and you may want to try
that or a GUI program like gparted.


Tried that. gdisk also reports 2TB and refuses to create any
partition larger than that.

- Evan

While its stating that you have a sector size of 512 bytes which
is odd. Most gpt drives should
be 4096bytes per sector, I just double checked. So even if its gpt
it may be doing it based on
issues with other things, not sure if the computer or device your
using at a firmware level
supports 4K sectors but it seems maybe that should be checked. Its
a common problem
with larger drives, I've never run into it as the systems I have
are almost all UEFI or later.

Nick



--
Evan Leibovitch, Toronto Canada
@evanleibovitch or @el56


---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Evan Leibovitch via talk]

Hrm.

For the 4TB disk in the same system (which has been working for years), fdisk
-l reports:

Disk /dev/sdc: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 88C7DF91-CD80-4EEB-AC6B-110119B04DD4

Device StartEndSectors  Size Type
/dev/sdc1   2048 7814035455 7814033408  3.7T Linux filesystem

Wondering if that helps. Starting to wonder if it's the firmware on the USB
external chassis.


On Sun, 29 Dec 2019 at 00:42, Nicholas Krause  wrote:

>
>
> On 12/29/19 12:19 AM, Evan Leibovitch wrote:
>
>
>
> On Sun, 29 Dec 2019 at 00:08, Nicholas Krause  wrote:
>
>>
>> Your using fdisk right. There is a version for GPT disks called gdisk and
>> you may want to try
>> that or a GUI program like gparted.
>>
>
> Tried that. gdisk also reports 2TB and refuses to create any partition
> larger than that.
>
> - Evan
>
> While its stating that you have a sector size of 512 bytes which is odd.
> Most gpt drives should
> be 4096bytes per sector, I just double checked. So even if its gpt it may
> be doing it based on
> issues with other things, not sure if the computer or device your using at
> a firmware level
> supports 4K sectors but it seems maybe that should be checked. Its a
> common problem
> with larger drives, I've never run into it as the systems I have are
> almost all UEFI or later.
>
> Nick
>
>

-- 
Evan Leibovitch, Toronto Canada
@evanleibovitch or @el56
---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Nicholas Krause via talk]

On 12/29/19 12:19 AM, Evan Leibovitch wrote:



On Sun, 29 Dec 2019 at 00:08, Nicholas Krause > wrote:



Your using fdisk right. There is a version for GPT disks called
gdisk and you may want to try
that or a GUI program like gparted.


Tried that. gdisk also reports 2TB and refuses to create any partition 
larger than that.


- Evan
While its stating that you have a sector size of 512 bytes which is odd. 
Most gpt drives should
be 4096bytes per sector, I just double checked. So even if its gpt it 
may be doing it based on
issues with other things, not sure if the computer or device your using 
at a firmware level
supports 4K sectors but it seems maybe that should be checked. Its a 
common problem
with larger drives, I've never run into it as the systems I have are 
almost all UEFI or later.


Nick

---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

Re: [GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Nicholas Krause via talk]

On 12/29/19 12:05 AM, Evan Leibovitch via talk wrote:
It's not an Xmas present as it was bought a few weeks ago, but I'm 
trying to install a 10TB WD (Red Pro) disk and it's not exactly going 
to plan.


My original plan was to put the disk into an external case, plug it in 
to format and copy over files, then install it in the case.


The system already has an SSD as /dec/sda, and two disks one of 3TB 
and the other of 4TB, all working fine.


When I attach the external drive, lsblock and fdisk -l both report TWO 
new drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set 
the disk from DOS to GPT but the size didn't change;


Now fdisk -l reports:

Disk /dev/sdd: 2 TiB, 2199023255040 bytes, 4294967295 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 49FDD63D-6C95-46F0-B7B1-650038D8FB9B

Disk /dev/sde: 2 TiB, 2199023255040 bytes, 4294967295 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

And cat /proc/scsi/scsi reports:

Host: scsi10 Channel: 00 Id: 00 Lun: 00
  Vendor: WDC WD10 Model: 1KFBX-68R56N0    Rev: 0200
  Type:   Direct-Access                    ANSI  SCSI revision: 04
Host: scsi10 Channel: 00 Id: 00 Lun: 01
  Vendor: WDC WD10 Model: 1KFBX-68R56NLUN1 Rev: 0200
  Type:   Direct-Access                    ANSI  SCSI revision: 04


The disk model is WD101KFBX and its specs can be found here. 
All 
of the online guides I've consulted say that the full capacity of the 
drive should be visible once I switch it to GPT mode.


Is this a USB limitation? Or is there something else I need to change? 
I've attached the output of hwinfo if that's any help.


Thanks for any suggestions!



Your using fdisk right. There is a version for GPT disks called gdisk 
and you may want to try

that or a GUI program like gparted.

Nick

--
Evan Leibovitch, Toronto Canada
@evanleibovitch or @el56

---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk


---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk

[GTALUG] 10TB drive seen as a 2TB drive (twice?)

[Evan Leibovitch via talk]

It's not an Xmas present as it was bought a few weeks ago, but I'm trying
to install a 10TB WD (Red Pro) disk and it's not exactly going to plan.

My original plan was to put the disk into an external case, plug it in to
format and copy over files, then install it in the case.

The system already has an SSD as /dec/sda, and two disks one of 3TB and the
other of 4TB, all working fine.

When I attach the external drive, lsblock and fdisk -l both report TWO new
drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set the disk
from DOS to GPT but the size didn't change;

Now fdisk -l reports:

Disk /dev/sdd: 2 TiB, 2199023255040 bytes, 4294967295 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 49FDD63D-6C95-46F0-B7B1-650038D8FB9B

Disk /dev/sde: 2 TiB, 2199023255040 bytes, 4294967295 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

And cat /proc/scsi/scsi reports:

Host: scsi10 Channel: 00 Id: 00 Lun: 00
  Vendor: WDC WD10 Model: 1KFBX-68R56N0Rev: 0200
  Type:   Direct-AccessANSI  SCSI revision: 04
Host: scsi10 Channel: 00 Id: 00 Lun: 01
  Vendor: WDC WD10 Model: 1KFBX-68R56NLUN1 Rev: 0200
  Type:   Direct-AccessANSI  SCSI revision: 04


The disk model is WD101KFBX and its specs can be found here.

All of the online guides I've consulted say that the full capacity of the
drive should be visible once I switch it to GPT mode.

Is this a USB limitation? Or is there something else I need to change? I've
attached the output of hwinfo if that's any help.

Thanks for any suggestions!

-- 
Evan Leibovitch, Toronto Canada
@evanleibovitch or @el56


evan-hwinfo.gz
Description: application/gzip
---
Post to this mailing list talk@gtalug.org
Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk