Re: [PLUG] Does an Open Source Thermostat exist?

[TomasK]

On Fri, 2022-07-15 at 18:07 -0700, Russell Senior wrote:
> On Fri, Jul 15, 2022 at 5:17 PM Tomas Kuchta
>  wrote:
> > 
> > On Fri, Jul 15, 2022, 17:04 Russell Senior 
> > wrote:
> > 
> > > I have used a bunch of these:
> > > 
> > >   https://www.amazon.com/gp/product/B00T0K8NXC/
> > > and
> > >   https://www.amazon.com/gp/product/B01G7BE9WK/
> > > 
> > > with https://github.com/merbanan/rtl_433 and an rtlsdr (like this:
> > > https://www.amazon.com/gp/product/B00VZ1AWQA/) to decode.
> > > 
> > > For improved accuracy, I calibrated the sensors in an icebath (in
> > > several layers of ziplock bags and desiccant) for 0 degrees offset.
> > > .
> > 
> > 
> > These are good sensors with great battery life. I also have ThermPro
> > sensors, they work, but the ones Russell lists are better, smaller and much
> > longer battery life.
> > 
> > I do not calibrate them. I care about being comfortable and saving power
> > rather than worry about whether the temperature reading is 0.2-0.3 degrees
> > different from absolutely correct value.
> 
> Fwiw, I didn't calibrate mine for a long time, but mine aren't so much
> for automated control, they were for understanding the temperature
> environments, and in particular differences and/or gradients in
> various microclimates. I noticed that a particular outside location
> seemed to get down to freezing before the other sensors. The
> significance of small temperature differences increases the closer you
> are to freezing, for example.  Eventually, I just wanted to understand
> whether the difference was due to the sensor or the microenvironment
> it happened to be in. Unless vigorously stirred, there can be
> significant temperature differences over very short distances, due to
> heat sources, stratification, illumination, etc.
> 
> I'd really like to have a lab grade temperature sensor, accurate to
> 0.01°C, to actually calibrate against. I encountered sensors when I
> worked in Oceanography with that kind of precision, but they were
> designed for water temperature and also were several thousand dollars.
> I wouldn't like to have one *that* much. Most consumer grade sensors
> only claim ±1°C.
> 

I was going to comment about - what atmospheric pressure did you calibrate your
sensors at - then I had second thought thinking about relatively high energy
needed/stored in H2O phase change which is connected to low dV (V-volume) on the
opposite sides of liquid/solid phase. H2O is very interesting substance indeed.

Anyway, I checked the theory, so I would not fool myself with the other smart
people here:

This is because of the Clausius-Clapeyron equation
dlogT / dlogP = (P dV) / L
where T is the temperature of the phase transition, dV is the change in volume,
and L is the latent heat. The water/gas transition has an enormous dV because
gas is much less dense than water, so dT/dP is large. The water/ice transition
has a dV about 10^-3, so dT/dP is small.

There is some 'cost' L to be paid doing the phase transition, most of it is paid
by thermal energy. If the volume changes during the transition, the P dV work
can help lowering the necessary temperature. So, it makes sense that dT/dP
depends on the ratio of these two contributions.

https://en.wikipedia.org/wiki/File:Phase_diagram_of_water.svg

So, as a consequence - water boils at low temperature on Mt. Everest or in
space, but ice cream is about as difficult to make at low/high pressures (unless
going above 1GPa+ (maybe there is a lot of ice(cream) in the middle of
sun/black-hole!)

Happy weekend, it should be ideal for temperature observations (with ice-cream)
-T

Re: [PLUG] Does an Open Source Thermostat exist?

[Russell Senior]

I just checked. My zero offset's run from -0.7 to +0.5 °C, based on an
hour-long fully-settled ice-bath zero-point calibration, over a sample
of about 20 sensors.

On Fri, Jul 15, 2022 at 6:07 PM Russell Senior
 wrote:
>
> On Fri, Jul 15, 2022 at 5:17 PM Tomas Kuchta
>  wrote:
> >
> > On Fri, Jul 15, 2022, 17:04 Russell Senior 
> > wrote:
> >
> > > I have used a bunch of these:
> > >
> > >   https://www.amazon.com/gp/product/B00T0K8NXC/
> > > and
> > >   https://www.amazon.com/gp/product/B01G7BE9WK/
> > >
> > > with https://github.com/merbanan/rtl_433 and an rtlsdr (like this:
> > > https://www.amazon.com/gp/product/B00VZ1AWQA/) to decode.
> > >
> > > For improved accuracy, I calibrated the sensors in an icebath (in
> > > several layers of ziplock bags and desiccant) for 0 degrees offset.
> > > .
> >
> >
> > These are good sensors with great battery life. I also have ThermPro
> > sensors, they work, but the ones Russell lists are better, smaller and much
> > longer battery life.
> >
> > I do not calibrate them. I care about being comfortable and saving power
> > rather than worry about whether the temperature reading is 0.2-0.3 degrees
> > different from absolutely correct value.
>
> Fwiw, I didn't calibrate mine for a long time, but mine aren't so much
> for automated control, they were for understanding the temperature
> environments, and in particular differences and/or gradients in
> various microclimates. I noticed that a particular outside location
> seemed to get down to freezing before the other sensors. The
> significance of small temperature differences increases the closer you
> are to freezing, for example.  Eventually, I just wanted to understand
> whether the difference was due to the sensor or the microenvironment
> it happened to be in. Unless vigorously stirred, there can be
> significant temperature differences over very short distances, due to
> heat sources, stratification, illumination, etc.
>
> I'd really like to have a lab grade temperature sensor, accurate to
> 0.01°C, to actually calibrate against. I encountered sensors when I
> worked in Oceanography with that kind of precision, but they were
> designed for water temperature and also were several thousand dollars.
> I wouldn't like to have one *that* much. Most consumer grade sensors
> only claim ±1°C.
>
> >
> > -T

Re: [PLUG] Does an Open Source Thermostat exist?

[Russell Senior]

On Fri, Jul 15, 2022 at 5:17 PM Tomas Kuchta
 wrote:
>
> On Fri, Jul 15, 2022, 17:04 Russell Senior 
> wrote:
>
> > I have used a bunch of these:
> >
> >   https://www.amazon.com/gp/product/B00T0K8NXC/
> > and
> >   https://www.amazon.com/gp/product/B01G7BE9WK/
> >
> > with https://github.com/merbanan/rtl_433 and an rtlsdr (like this:
> > https://www.amazon.com/gp/product/B00VZ1AWQA/) to decode.
> >
> > For improved accuracy, I calibrated the sensors in an icebath (in
> > several layers of ziplock bags and desiccant) for 0 degrees offset.
> > .
>
>
> These are good sensors with great battery life. I also have ThermPro
> sensors, they work, but the ones Russell lists are better, smaller and much
> longer battery life.
>
> I do not calibrate them. I care about being comfortable and saving power
> rather than worry about whether the temperature reading is 0.2-0.3 degrees
> different from absolutely correct value.

Fwiw, I didn't calibrate mine for a long time, but mine aren't so much
for automated control, they were for understanding the temperature
environments, and in particular differences and/or gradients in
various microclimates. I noticed that a particular outside location
seemed to get down to freezing before the other sensors. The
significance of small temperature differences increases the closer you
are to freezing, for example.  Eventually, I just wanted to understand
whether the difference was due to the sensor or the microenvironment
it happened to be in. Unless vigorously stirred, there can be
significant temperature differences over very short distances, due to
heat sources, stratification, illumination, etc.

I'd really like to have a lab grade temperature sensor, accurate to
0.01°C, to actually calibrate against. I encountered sensors when I
worked in Oceanography with that kind of precision, but they were
designed for water temperature and also were several thousand dollars.
I wouldn't like to have one *that* much. Most consumer grade sensors
only claim ±1°C.

>
> -T

Re: [PLUG] Does an Open Source Thermostat exist?

[Jason Barnett]

The things you listed sound a lot like a full home automation, even if it
is just controlling the thermostat. All the sensors needed to detect
presence, time, temperature, etc. can be done "simply" using something like
Home Assistant (https://www.home-assistant.io/) and no cloud or even
internet required for many configurations. There are many people that have
built Open Source thermostats that tie into their systems and will control
multiple zones based on presence, weather forecast, time of day, etc.

Beware, this is a deep rabbit hole, proceed with caution.


On Fri, Jul 15, 2022 at 5:17 PM Tomas Kuchta 
wrote:

> On Fri, Jul 15, 2022, 17:04 Russell Senior 
> wrote:
>
> > I have used a bunch of these:
> >
> >   https://www.amazon.com/gp/product/B00T0K8NXC/
> > and
> >   https://www.amazon.com/gp/product/B01G7BE9WK/
> >
> > with https://github.com/merbanan/rtl_433 and an rtlsdr (like this:
> > https://www.amazon.com/gp/product/B00VZ1AWQA/) to decode.
> >
> > For improved accuracy, I calibrated the sensors in an icebath (in
> > several layers of ziplock bags and desiccant) for 0 degrees offset.
> > .
>
>
> These are good sensors with great battery life. I also have ThermPro
> sensors, they work, but the ones Russell lists are better, smaller and much
> longer battery life.
>
> I do not calibrate them. I care about being comfortable and saving power
> rather than worry about whether the temperature reading is 0.2-0.3 degrees
> different from absolutely correct value.
>
> -T
>

Re: [PLUG] Does an Open Source Thermostat exist?

[Tomas Kuchta]

On Fri, Jul 15, 2022, 17:04 Russell Senior 
wrote:

> I have used a bunch of these:
>
>   https://www.amazon.com/gp/product/B00T0K8NXC/
> and
>   https://www.amazon.com/gp/product/B01G7BE9WK/
>
> with https://github.com/merbanan/rtl_433 and an rtlsdr (like this:
> https://www.amazon.com/gp/product/B00VZ1AWQA/) to decode.
>
> For improved accuracy, I calibrated the sensors in an icebath (in
> several layers of ziplock bags and desiccant) for 0 degrees offset.
> .


These are good sensors with great battery life. I also have ThermPro
sensors, they work, but the ones Russell lists are better, smaller and much
longer battery life.

I do not calibrate them. I care about being comfortable and saving power
rather than worry about whether the temperature reading is 0.2-0.3 degrees
different from absolutely correct value.

-T

Re: [PLUG] Need help backing up my data

[Tomas Kuchta]

>
> .

How many files/average file size?

If you are copying millions of small files, it would be pretty slow to slow
SMR disk.

That said, you have tone of unnecessary options there. I will not comment
on options, you know what you need. Definitely get rid of the print, that
slows stuff a lot. If you really need the list of files, redirect it to a
file and watch the dile separately.

Hope that helps, -T

Re: [PLUG] Does an Open Source Thermostat exist?

[Russell Senior]

I have used a bunch of these:

  https://www.amazon.com/gp/product/B00T0K8NXC/
and
  https://www.amazon.com/gp/product/B01G7BE9WK/

with https://github.com/merbanan/rtl_433 and an rtlsdr (like this:
https://www.amazon.com/gp/product/B00VZ1AWQA/) to decode.

For improved accuracy, I calibrated the sensors in an icebath (in
several layers of ziplock bags and desiccant) for 0 degrees offset.

On Fri, Jul 15, 2022 at 1:24 PM Larry Brigman  wrote:
>
> On Thu, Jul 14, 2022 at 2:53 PM Tomas Kuchta 
> wrote:
> > I have seeded house + exterior with bunch of 433MHz wireless temperature +
> > humidity sensors - which I read with cheap SDR.
> > I control this with simple raspberry PI + relay board + simple program.
>
> Which sensors did you used?

Re: [PLUG] Does an Open Source Thermostat exist?

[Larry Brigman]

On Thu, Jul 14, 2022 at 2:53 PM Tomas Kuchta 
wrote:
> I have seeded house + exterior with bunch of 433MHz wireless temperature +
> humidity sensors - which I read with cheap SDR.
> I control this with simple raspberry PI + relay board + simple program.

Which sensors did you used?

Re: [PLUG] Need help backing up my data

[Jason Barnett]

I agree, that is VERY slow.  Are you sure it is plugged into a USB3 port?
Some HDDs are fast at first, but then slow down after their buffers get
full (Shingled magnetic recording
), but even then
I would expect 20 - 30 MB/s transfer rates.

You may want to compress the data on the fly so you can increase the
effective transfer speed, if that is suitable for your purposes.

Jason

On Fri, Jul 15, 2022 at 12:37 PM Robert Citek 
wrote:

> Even for a spinning disk that seems really slow, especially if it is rated
> for USB3.0.  I haven’t played with spinning disks in years, but IIRC I was
> getting 30-40 MB/s writes.
>
> This drive sounds similar to yours and is advertised at a max of 220 MB/s.
> Even half that speed would be quite good.
>
>
> https://www.bestbuy.com/site/seagate-backup-plus-fast-4tb-external-usb-3-0-portable-hard-drive-black/5127078.p?skuId=5127078
>
> Regards,
> - Robert
>
> On Fri, Jul 15, 2022 at 1:22 PM Mark Phillips 
> wrote:
>
> > Found the problemI got a rotational drive, not ssd.
> >
> > Mark
> >
> > On Fri, Jul 15, 2022, 12:15 PM Mark Phillips  >
> > wrote:
> >
> > > Disks benchmarks 1/2 way done.
> > >
> > > Showing
> > > Avg read 2.3 MB/sec
> > > Avg write 1.8 MB/sec
> > > Avg access time 450 msec
> > >
> > > Does this seem slow for a seagate 4tb USB 3 external drive?
> > >
> > > Mark
> > >
> > > On Fri, Jul 15, 2022, 11:53 AM Mark Phillips <
> m...@phillipsmarketing.biz
> > >
> > > wrote:
> > >
> > >> It may be a cockpit error. The USB drive is brand new, and I assumed
> > >> formatted to vfat. It turns out it is ntfs. I reformatted the drive to
> > >> ext4, and I am running benchmarks.
> > >>
> > >> Initial results from hdparm (on a different machine - SurfacePro 4
> > >> running Ubuntu
> > >> Timing cache reads: 9598.53 MB/sec
> > >> Buffered disk reads: 6.47 MB/sec
> > >>
> > >> Waiting for disks benchmark on the target machine.
> > >>
> > >> Mark
> > >>
> > >> On Fri, Jul 15, 2022, 10:21 AM Robert Citek 
> > >> wrote:
> > >>
> > >>> Indeed, that sounds really slow:
> > >>>
> > >>> ( 138GB * 1000MB/GB ) / (26hr * 60min/hr * 60s/min ) = 1.5 MB/s  ~ 12
> > >>> Mbps
> > >>>
> > >>> That's in the USB1.x range.  If you use USB3.0 and can get 100MB/s
> > write
> > >>> speed, you'd be done in about 6 hours.
> > >>>
> > >>> Have a look at hdparm to get some info on read/write performance of
> > your
> > >>> drive:
> > >>>
> > >>>
> > >>>
> >
> https://linuxconfig.org/hard-drive-speed-test-using-linux-command-line-and-hdparm
> > >>>
> > >>> Good luck and let us know what you discover.
> > >>>
> > >>> Regards,
> > >>> - Robert
> > >>>
> > >>>
> > >>>
> > >>> On Fri, Jul 15, 2022 at 10:48 AM Michael Ewan <
> michaelewa...@gmail.com
> > >
> > >>> wrote:
> > >>>
> > >>> > Are you using a USB3 drive and a USB3 port, the speed of the
> > interface
> > >>> is
> > >>> > what I would think of first.
> > >>> >
> > >>> > On Fri, Jul 15, 2022 at 8:41 AM Mark Phillips <
> > >>> m...@phillipsmarketing.biz>
> > >>> > wrote:
> > >>> >
> > >>> > > I have an Ubuntu 18.04 system with two drives in an lvm with one
> > >>> logical
> > >>> > > root partition. I am trying to back up the contents of the drives
> > >>> (ie /)
> > >>> > to
> > >>> > > an external usb drive using rsync. It is taking a really long
> time.
> > >>> After
> > >>> > > 26 hours of continuous operation I have only transferred 138 GB
> out
> > >>> of 2+
> > >>> > > TB, so I am looking at about 16 days to complete the transfer.
> > >>> > >
> > >>> > > My rsync command is:
> > >>> > > sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
> > >>> > >
> > >>> > >
> > >>> >
> > >>>
> >
> --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
> > >>> > > / '/media/mark/Seagate Portable
> > Drive/tsunami-backups-Jul_13_17-39/'
> > >>> > >
> > >>> > > Any suggestions on how I can speed this up and not lose any data?
> > >>> > >
> > >>> > > Thanks!
> > >>> > >
> > >>> > > Mark
> > >>> > >
> > >>> >
> > >>>
> > >>
> >
>

Re: [PLUG] Need help backing up my data

[Mark Phillips]

This is what I bought, and I am returning it

Seagate Portable 4TB External Hard Drive HDD – USB 3.0 for PC, Mac, Xbox, &
PlayStation - 1-Year Rescue Service (STGX4000400) https://a.co/d/9MjJj7X

Mark

On Fri, Jul 15, 2022, 12:37 PM Robert Citek  wrote:

> Even for a spinning disk that seems really slow, especially if it is rated
> for USB3.0.  I haven’t played with spinning disks in years, but IIRC I was
> getting 30-40 MB/s writes.
>
> This drive sounds similar to yours and is advertised at a max of 220 MB/s.
> Even half that speed would be quite good.
>
>
> https://www.bestbuy.com/site/seagate-backup-plus-fast-4tb-external-usb-3-0-portable-hard-drive-black/5127078.p?skuId=5127078
>
> Regards,
> - Robert
>
> On Fri, Jul 15, 2022 at 1:22 PM Mark Phillips 
> wrote:
>
> > Found the problemI got a rotational drive, not ssd.
> >
> > Mark
> >
> > On Fri, Jul 15, 2022, 12:15 PM Mark Phillips  >
> > wrote:
> >
> > > Disks benchmarks 1/2 way done.
> > >
> > > Showing
> > > Avg read 2.3 MB/sec
> > > Avg write 1.8 MB/sec
> > > Avg access time 450 msec
> > >
> > > Does this seem slow for a seagate 4tb USB 3 external drive?
> > >
> > > Mark
> > >
> > > On Fri, Jul 15, 2022, 11:53 AM Mark Phillips <
> m...@phillipsmarketing.biz
> > >
> > > wrote:
> > >
> > >> It may be a cockpit error. The USB drive is brand new, and I assumed
> > >> formatted to vfat. It turns out it is ntfs. I reformatted the drive to
> > >> ext4, and I am running benchmarks.
> > >>
> > >> Initial results from hdparm (on a different machine - SurfacePro 4
> > >> running Ubuntu
> > >> Timing cache reads: 9598.53 MB/sec
> > >> Buffered disk reads: 6.47 MB/sec
> > >>
> > >> Waiting for disks benchmark on the target machine.
> > >>
> > >> Mark
> > >>
> > >> On Fri, Jul 15, 2022, 10:21 AM Robert Citek 
> > >> wrote:
> > >>
> > >>> Indeed, that sounds really slow:
> > >>>
> > >>> ( 138GB * 1000MB/GB ) / (26hr * 60min/hr * 60s/min ) = 1.5 MB/s  ~ 12
> > >>> Mbps
> > >>>
> > >>> That's in the USB1.x range.  If you use USB3.0 and can get 100MB/s
> > write
> > >>> speed, you'd be done in about 6 hours.
> > >>>
> > >>> Have a look at hdparm to get some info on read/write performance of
> > your
> > >>> drive:
> > >>>
> > >>>
> > >>>
> >
> https://linuxconfig.org/hard-drive-speed-test-using-linux-command-line-and-hdparm
> > >>>
> > >>> Good luck and let us know what you discover.
> > >>>
> > >>> Regards,
> > >>> - Robert
> > >>>
> > >>>
> > >>>
> > >>> On Fri, Jul 15, 2022 at 10:48 AM Michael Ewan <
> michaelewa...@gmail.com
> > >
> > >>> wrote:
> > >>>
> > >>> > Are you using a USB3 drive and a USB3 port, the speed of the
> > interface
> > >>> is
> > >>> > what I would think of first.
> > >>> >
> > >>> > On Fri, Jul 15, 2022 at 8:41 AM Mark Phillips <
> > >>> m...@phillipsmarketing.biz>
> > >>> > wrote:
> > >>> >
> > >>> > > I have an Ubuntu 18.04 system with two drives in an lvm with one
> > >>> logical
> > >>> > > root partition. I am trying to back up the contents of the drives
> > >>> (ie /)
> > >>> > to
> > >>> > > an external usb drive using rsync. It is taking a really long
> time.
> > >>> After
> > >>> > > 26 hours of continuous operation I have only transferred 138 GB
> out
> > >>> of 2+
> > >>> > > TB, so I am looking at about 16 days to complete the transfer.
> > >>> > >
> > >>> > > My rsync command is:
> > >>> > > sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
> > >>> > >
> > >>> > >
> > >>> >
> > >>>
> >
> --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
> > >>> > > / '/media/mark/Seagate Portable
> > Drive/tsunami-backups-Jul_13_17-39/'
> > >>> > >
> > >>> > > Any suggestions on how I can speed this up and not lose any data?
> > >>> > >
> > >>> > > Thanks!
> > >>> > >
> > >>> > > Mark
> > >>> > >
> > >>> >
> > >>>
> > >>
> >
>

Re: [PLUG] Need help backing up my data

[Robert Citek]

Even for a spinning disk that seems really slow, especially if it is rated
for USB3.0.  I haven’t played with spinning disks in years, but IIRC I was
getting 30-40 MB/s writes.

This drive sounds similar to yours and is advertised at a max of 220 MB/s.
Even half that speed would be quite good.

https://www.bestbuy.com/site/seagate-backup-plus-fast-4tb-external-usb-3-0-portable-hard-drive-black/5127078.p?skuId=5127078

Regards,
- Robert

On Fri, Jul 15, 2022 at 1:22 PM Mark Phillips 
wrote:

> Found the problemI got a rotational drive, not ssd.
>
> Mark
>
> On Fri, Jul 15, 2022, 12:15 PM Mark Phillips 
> wrote:
>
> > Disks benchmarks 1/2 way done.
> >
> > Showing
> > Avg read 2.3 MB/sec
> > Avg write 1.8 MB/sec
> > Avg access time 450 msec
> >
> > Does this seem slow for a seagate 4tb USB 3 external drive?
> >
> > Mark
> >
> > On Fri, Jul 15, 2022, 11:53 AM Mark Phillips  >
> > wrote:
> >
> >> It may be a cockpit error. The USB drive is brand new, and I assumed
> >> formatted to vfat. It turns out it is ntfs. I reformatted the drive to
> >> ext4, and I am running benchmarks.
> >>
> >> Initial results from hdparm (on a different machine - SurfacePro 4
> >> running Ubuntu
> >> Timing cache reads: 9598.53 MB/sec
> >> Buffered disk reads: 6.47 MB/sec
> >>
> >> Waiting for disks benchmark on the target machine.
> >>
> >> Mark
> >>
> >> On Fri, Jul 15, 2022, 10:21 AM Robert Citek 
> >> wrote:
> >>
> >>> Indeed, that sounds really slow:
> >>>
> >>> ( 138GB * 1000MB/GB ) / (26hr * 60min/hr * 60s/min ) = 1.5 MB/s  ~ 12
> >>> Mbps
> >>>
> >>> That's in the USB1.x range.  If you use USB3.0 and can get 100MB/s
> write
> >>> speed, you'd be done in about 6 hours.
> >>>
> >>> Have a look at hdparm to get some info on read/write performance of
> your
> >>> drive:
> >>>
> >>>
> >>>
> https://linuxconfig.org/hard-drive-speed-test-using-linux-command-line-and-hdparm
> >>>
> >>> Good luck and let us know what you discover.
> >>>
> >>> Regards,
> >>> - Robert
> >>>
> >>>
> >>>
> >>> On Fri, Jul 15, 2022 at 10:48 AM Michael Ewan  >
> >>> wrote:
> >>>
> >>> > Are you using a USB3 drive and a USB3 port, the speed of the
> interface
> >>> is
> >>> > what I would think of first.
> >>> >
> >>> > On Fri, Jul 15, 2022 at 8:41 AM Mark Phillips <
> >>> m...@phillipsmarketing.biz>
> >>> > wrote:
> >>> >
> >>> > > I have an Ubuntu 18.04 system with two drives in an lvm with one
> >>> logical
> >>> > > root partition. I am trying to back up the contents of the drives
> >>> (ie /)
> >>> > to
> >>> > > an external usb drive using rsync. It is taking a really long time.
> >>> After
> >>> > > 26 hours of continuous operation I have only transferred 138 GB out
> >>> of 2+
> >>> > > TB, so I am looking at about 16 days to complete the transfer.
> >>> > >
> >>> > > My rsync command is:
> >>> > > sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
> >>> > >
> >>> > >
> >>> >
> >>>
> --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
> >>> > > / '/media/mark/Seagate Portable
> Drive/tsunami-backups-Jul_13_17-39/'
> >>> > >
> >>> > > Any suggestions on how I can speed this up and not lose any data?
> >>> > >
> >>> > > Thanks!
> >>> > >
> >>> > > Mark
> >>> > >
> >>> >
> >>>
> >>
>

Re: [PLUG] Need help backing up my data

[Mark Phillips]

Found the problemI got a rotational drive, not ssd.

Mark

On Fri, Jul 15, 2022, 12:15 PM Mark Phillips 
wrote:

> Disks benchmarks 1/2 way done.
>
> Showing
> Avg read 2.3 MB/sec
> Avg write 1.8 MB/sec
> Avg access time 450 msec
>
> Does this seem slow for a seagate 4tb USB 3 external drive?
>
> Mark
>
> On Fri, Jul 15, 2022, 11:53 AM Mark Phillips 
> wrote:
>
>> It may be a cockpit error. The USB drive is brand new, and I assumed
>> formatted to vfat. It turns out it is ntfs. I reformatted the drive to
>> ext4, and I am running benchmarks.
>>
>> Initial results from hdparm (on a different machine - SurfacePro 4
>> running Ubuntu
>> Timing cache reads: 9598.53 MB/sec
>> Buffered disk reads: 6.47 MB/sec
>>
>> Waiting for disks benchmark on the target machine.
>>
>> Mark
>>
>> On Fri, Jul 15, 2022, 10:21 AM Robert Citek 
>> wrote:
>>
>>> Indeed, that sounds really slow:
>>>
>>> ( 138GB * 1000MB/GB ) / (26hr * 60min/hr * 60s/min ) = 1.5 MB/s  ~ 12
>>> Mbps
>>>
>>> That's in the USB1.x range.  If you use USB3.0 and can get 100MB/s write
>>> speed, you'd be done in about 6 hours.
>>>
>>> Have a look at hdparm to get some info on read/write performance of your
>>> drive:
>>>
>>>
>>> https://linuxconfig.org/hard-drive-speed-test-using-linux-command-line-and-hdparm
>>>
>>> Good luck and let us know what you discover.
>>>
>>> Regards,
>>> - Robert
>>>
>>>
>>>
>>> On Fri, Jul 15, 2022 at 10:48 AM Michael Ewan 
>>> wrote:
>>>
>>> > Are you using a USB3 drive and a USB3 port, the speed of the interface
>>> is
>>> > what I would think of first.
>>> >
>>> > On Fri, Jul 15, 2022 at 8:41 AM Mark Phillips <
>>> m...@phillipsmarketing.biz>
>>> > wrote:
>>> >
>>> > > I have an Ubuntu 18.04 system with two drives in an lvm with one
>>> logical
>>> > > root partition. I am trying to back up the contents of the drives
>>> (ie /)
>>> > to
>>> > > an external usb drive using rsync. It is taking a really long time.
>>> After
>>> > > 26 hours of continuous operation I have only transferred 138 GB out
>>> of 2+
>>> > > TB, so I am looking at about 16 days to complete the transfer.
>>> > >
>>> > > My rsync command is:
>>> > > sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
>>> > >
>>> > >
>>> >
>>> --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
>>> > > / '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/'
>>> > >
>>> > > Any suggestions on how I can speed this up and not lose any data?
>>> > >
>>> > > Thanks!
>>> > >
>>> > > Mark
>>> > >
>>> >
>>>
>>

Re: [PLUG] Need help backing up my data

[Mark Phillips]

Disks benchmarks 1/2 way done.

Showing
Avg read 2.3 MB/sec
Avg write 1.8 MB/sec
Avg access time 450 msec

Does this seem slow for a seagate 4tb USB 3 external drive?

Mark

On Fri, Jul 15, 2022, 11:53 AM Mark Phillips 
wrote:

> It may be a cockpit error. The USB drive is brand new, and I assumed
> formatted to vfat. It turns out it is ntfs. I reformatted the drive to
> ext4, and I am running benchmarks.
>
> Initial results from hdparm (on a different machine - SurfacePro 4 running
> Ubuntu
> Timing cache reads: 9598.53 MB/sec
> Buffered disk reads: 6.47 MB/sec
>
> Waiting for disks benchmark on the target machine.
>
> Mark
>
> On Fri, Jul 15, 2022, 10:21 AM Robert Citek 
> wrote:
>
>> Indeed, that sounds really slow:
>>
>> ( 138GB * 1000MB/GB ) / (26hr * 60min/hr * 60s/min ) = 1.5 MB/s  ~ 12 Mbps
>>
>> That's in the USB1.x range.  If you use USB3.0 and can get 100MB/s write
>> speed, you'd be done in about 6 hours.
>>
>> Have a look at hdparm to get some info on read/write performance of your
>> drive:
>>
>>
>> https://linuxconfig.org/hard-drive-speed-test-using-linux-command-line-and-hdparm
>>
>> Good luck and let us know what you discover.
>>
>> Regards,
>> - Robert
>>
>>
>>
>> On Fri, Jul 15, 2022 at 10:48 AM Michael Ewan 
>> wrote:
>>
>> > Are you using a USB3 drive and a USB3 port, the speed of the interface
>> is
>> > what I would think of first.
>> >
>> > On Fri, Jul 15, 2022 at 8:41 AM Mark Phillips <
>> m...@phillipsmarketing.biz>
>> > wrote:
>> >
>> > > I have an Ubuntu 18.04 system with two drives in an lvm with one
>> logical
>> > > root partition. I am trying to back up the contents of the drives (ie
>> /)
>> > to
>> > > an external usb drive using rsync. It is taking a really long time.
>> After
>> > > 26 hours of continuous operation I have only transferred 138 GB out
>> of 2+
>> > > TB, so I am looking at about 16 days to complete the transfer.
>> > >
>> > > My rsync command is:
>> > > sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
>> > >
>> > >
>> >
>> --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
>> > > / '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/'
>> > >
>> > > Any suggestions on how I can speed this up and not lose any data?
>> > >
>> > > Thanks!
>> > >
>> > > Mark
>> > >
>> >
>>
>

Re: [PLUG] Need help backing up my data

[Mark Phillips]

It may be a cockpit error. The USB drive is brand new, and I assumed
formatted to vfat. It turns out it is ntfs. I reformatted the drive to
ext4, and I am running benchmarks.

Initial results from hdparm (on a different machine - SurfacePro 4 running
Ubuntu
Timing cache reads: 9598.53 MB/sec
Buffered disk reads: 6.47 MB/sec

Waiting for disks benchmark on the target machine.

Mark

On Fri, Jul 15, 2022, 10:21 AM Robert Citek  wrote:

> Indeed, that sounds really slow:
>
> ( 138GB * 1000MB/GB ) / (26hr * 60min/hr * 60s/min ) = 1.5 MB/s  ~ 12 Mbps
>
> That's in the USB1.x range.  If you use USB3.0 and can get 100MB/s write
> speed, you'd be done in about 6 hours.
>
> Have a look at hdparm to get some info on read/write performance of your
> drive:
>
>
> https://linuxconfig.org/hard-drive-speed-test-using-linux-command-line-and-hdparm
>
> Good luck and let us know what you discover.
>
> Regards,
> - Robert
>
>
>
> On Fri, Jul 15, 2022 at 10:48 AM Michael Ewan 
> wrote:
>
> > Are you using a USB3 drive and a USB3 port, the speed of the interface is
> > what I would think of first.
> >
> > On Fri, Jul 15, 2022 at 8:41 AM Mark Phillips <
> m...@phillipsmarketing.biz>
> > wrote:
> >
> > > I have an Ubuntu 18.04 system with two drives in an lvm with one
> logical
> > > root partition. I am trying to back up the contents of the drives (ie
> /)
> > to
> > > an external usb drive using rsync. It is taking a really long time.
> After
> > > 26 hours of continuous operation I have only transferred 138 GB out of
> 2+
> > > TB, so I am looking at about 16 days to complete the transfer.
> > >
> > > My rsync command is:
> > > sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
> > >
> > >
> >
> --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
> > > / '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/'
> > >
> > > Any suggestions on how I can speed this up and not lose any data?
> > >
> > > Thanks!
> > >
> > > Mark
> > >
> >
>

Re: [PLUG] Need help backing up my data

[Mark Phillips]

Yes, they are both USB 3.x

Mark

On Fri, Jul 15, 2022, 9:48 AM Michael Ewan  wrote:

> Are you using a USB3 drive and a USB3 port, the speed of the interface is
> what I would think of first.
>
> On Fri, Jul 15, 2022 at 8:41 AM Mark Phillips 
> wrote:
>
> > I have an Ubuntu 18.04 system with two drives in an lvm with one logical
> > root partition. I am trying to back up the contents of the drives (ie /)
> to
> > an external usb drive using rsync. It is taking a really long time. After
> > 26 hours of continuous operation I have only transferred 138 GB out of 2+
> > TB, so I am looking at about 16 days to complete the transfer.
> >
> > My rsync command is:
> > sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
> >
> >
> --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
> > / '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/'
> >
> > Any suggestions on how I can speed this up and not lose any data?
> >
> > Thanks!
> >
> > Mark
> >
>

Re: [PLUG] Need help backing up my data

[Robert Citek]

Indeed, that sounds really slow:

( 138GB * 1000MB/GB ) / (26hr * 60min/hr * 60s/min ) = 1.5 MB/s  ~ 12 Mbps

That's in the USB1.x range.  If you use USB3.0 and can get 100MB/s write
speed, you'd be done in about 6 hours.

Have a look at hdparm to get some info on read/write performance of your
drive:

https://linuxconfig.org/hard-drive-speed-test-using-linux-command-line-and-hdparm

Good luck and let us know what you discover.

Regards,
- Robert



On Fri, Jul 15, 2022 at 10:48 AM Michael Ewan 
wrote:

> Are you using a USB3 drive and a USB3 port, the speed of the interface is
> what I would think of first.
>
> On Fri, Jul 15, 2022 at 8:41 AM Mark Phillips 
> wrote:
>
> > I have an Ubuntu 18.04 system with two drives in an lvm with one logical
> > root partition. I am trying to back up the contents of the drives (ie /)
> to
> > an external usb drive using rsync. It is taking a really long time. After
> > 26 hours of continuous operation I have only transferred 138 GB out of 2+
> > TB, so I am looking at about 16 days to complete the transfer.
> >
> > My rsync command is:
> > sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
> >
> >
> --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
> > / '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/'
> >
> > Any suggestions on how I can speed this up and not lose any data?
> >
> > Thanks!
> >
> > Mark
> >
>

Re: [PLUG] Need help backing up my data

[Michael Ewan]

Are you using a USB3 drive and a USB3 port, the speed of the interface is
what I would think of first.

On Fri, Jul 15, 2022 at 8:41 AM Mark Phillips 
wrote:

> I have an Ubuntu 18.04 system with two drives in an lvm with one logical
> root partition. I am trying to back up the contents of the drives (ie /) to
> an external usb drive using rsync. It is taking a really long time. After
> 26 hours of continuous operation I have only transferred 138 GB out of 2+
> TB, so I am looking at about 16 days to complete the transfer.
>
> My rsync command is:
> sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
>
> --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
> / '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/'
>
> Any suggestions on how I can speed this up and not lose any data?
>
> Thanks!
>
> Mark
>

[PLUG] Need help backing up my data

[Mark Phillips]

I have an Ubuntu 18.04 system with two drives in an lvm with one logical
root partition. I am trying to back up the contents of the drives (ie /) to
an external usb drive using rsync. It is taking a really long time. After
26 hours of continuous operation I have only transferred 138 GB out of 2+
TB, so I am looking at about 16 days to complete the transfer.

My rsync command is:
sudo rsync  --no-compress --info=progress2 -avAXEWSlHh
--exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'}
/ '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/'

Any suggestions on how I can speed this up and not lose any data?

Thanks!

Mark