On March 30, 2021 10:11:56 AM PDT, Dr Rainer Woitok <rainer.woi...@gmail.com>
wrote:
>On Saturday, 2020-12-05 19:07:51 +0100, I myself wrote:
>
>("> >" refers to Michael <confabul...@kintzios.com>)
>
>> Michael,
>>
>> On Friday, 2020-11-27 19:07:17 +0000, you wrote:
>>
>> > ...
>> > A 4k block size is recommended for ntfs-3g which is the default
>sector created
>> > by fdisk and friends on Linux these days. This will align your
>partition
>> > optimally. In addition, mkfs.ntfs will use 4096 bytes as the
>default cluster
>> > size, so you should be good in that respect.
>> >
>> > Another setting you may want to try is mounting the USB with
>'big_writes' -
>> > check the man page. This should help particularly with large
>files, which
>> > will use larger blocks up to 128KB when copying data to the NTFS.
>>
>> Both, the VeraCrypt command line (--fs-options=big_writes) and the
>Vera-
>> Crypt GUI (under "Settings --> Preferences") allow setting this
>mount
>> option. But
>>
>> $ mount | grep veracrypt
>>
>> never shows it, initially causing me to erroneously believe it
>wasn't
>> set and to try finding on the web another way of setting it. By
>pure
>> chance I finally found out that
>>
>> $ ps -ef | grep veracrypt
>>
>> lists a "/usr/sbin/mount.ntfs" task which shows the options really
>in
>> effect. However, I haven't yet had the time to test the effect of
>this
>> option when writing plenty of really big files. I will report on
>that
>> later.
>
>Well, it's been quite a while, due to my being almost permanently
>con-
>fronted with more pressing tasks ... :-(
>
>To sum up my experience with my new 128 GB Philips USB 3.0 sticks:
>while
>the Philips sticks are significantly faster for reading operations
>than
>my old 64 GB Verbatim ones (probably USB 2.0), writing operations to
>the
>Philips sticks are unbearably slow, regardless of whether I created
>a
>normal unencrypted NTFS filesystem on them or an encrypted NTFS
>filesys-
>tem using VeraCrypt. Writing to the USB stick while at the same
>time
>reading from it in a different terminal window caused commands like
>"cd"
>or "ls" to simply stall. Thus while running
>
> $ cp --preserve=timestamps -ru $source_dir .
>
>in one terminal window, I ran
>
> $ while true
> > do n=$(ps -ef|g 'cp --preserve'|g -v grep)
> > if [[ "$n" = "${o-}" ]]
> > then sleep 10
> > else o="$n"
> > echo "$n"
> > fi
> > done
>
>in another, to get the wall clock times when copying a new file
>began.
>That way I found that copying a 30 MB file took about 40 minutes.
>
>So what are my options?
>
> - Stay away from Philips USB 3.0 sticks?
>
> - Stay away from Philips USB sticks in general?
>
> - Stay away from USB 3.0 sticks in general?
>
> - Stay away from Filesystem in User Space using a non-stable 5.10 or
> 5.11 kernel (currently I'm using stable 5.4.97)?
>
> - Stay away from Gentoo?
>
> - Stay away from Linux in general and go back to OTOS (aka the Only
> True Operating System aka Windoze)?
>
> - ...?
>
>Any ideas and comments welcome ...
>
>Sincerely,
> Rainer
There are a number of things which might be going on here.
To start with, you can get the kernel, user, and wall clock run times for
commands by prefixing it with "time". So:
time cp <large file> <USB mountpoint>
Will get you more precise answers with much less effort.
As for the performance of the USB drive in question, there are a few things
that might be tripping it up.
Firstly, writing flash memory is significantly slower than reading it. Some
drives deal with this by having some kind of internal cache mechanism. Many
deal with it by using a pile of smaller chips instead of one big one and
striping the writes. If the Phillips drive just used a few large chips
instead, then it's just slow to write to and there isn't much you can do about
it. I've seen a lot of cheaper drives that are that way.
Double check that the alignment and block size are correct for the drive's
internal structure. That can cause some pretty massive performance hits if
it's incorrect.
You can also check the output of the dmesg command for any errors the system is
encountering with regard to the drive.
I don't know of any reason to stay away from usb 3.0 on Linux, but if you have
USB 3 devices on both ends and try to hook them together with a USB 2.0 or 1.1
rated cable that could easily cause some problems... I assume you're plugging
the drive straight into the machine's socket. If you're using the front panel
though try one of the ones on the back. There may be something up with the
case wiring.
I've never had a Phillips USB stick, so maybe do some tests with another brand
of stick and see if it has the same problem. Kingston or SanDisk or something.
One of the ones where memory is their primary focus.
You could definitely check performance on a different OS. There may be
driver-related performance issues on this model of drive or even this specific
drive.
Instead of NTFS you could also try UDF. It's supported on both Linux and
Windows Vista and newer. (And even XP I think, but only an older revision of
the filesystem.) Being an open standard it is quite a bit more stable on Linux
than NTFS, doesn't need a userspace driver, and still supports larger drives
and files. You can find Linux and Windows formatting instructions on the web
pretty easily. Big thing is that on Windows it's available via the format
command, but not in the GUI.
That's all I can think of at the moment. Hope it helps.
LMP
