RE: cp to flash drive very slow

[Hannu E K Nevalainen]

you wrote:


> What happens if you cp between two hard drives, or across the
> network?  Same
> crazy slowness?
> 
> --
> Gary R. Van Sickle

How about 'tar --diff' and '--update' - tried them?


/Hannu E K Nevalainen, B.Sc. EE Microcomputer systems--72-->

** mailing list preference; please keep replies on list **

-- printf("LocalTime: UTC+%02d\n",(DST)? 2:1); --
--END OF MESSAGE--

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: [OT] RE: cp to flash drive very slow

[Gary R. Van Sickle]

That is simultaneously so sweet and so wrong.

I wonder which Device Class that one falls under ;-).

-- 
Gary R. Van Sickle
 

> -Original Message-
> From: Igor Pechtchanski [mailto:[EMAIL PROTECTED] 
> Sent: Friday, October 01, 2004 9:36 AM
> To: Gary R. Van Sickle
> Cc: [EMAIL PROTECTED]
> Subject: [OT] RE: cp to flash drive very slow
> 
> On Thu, 30 Sep 2004, Gary R. Van Sickle wrote:
> 
> > > Well, yes.  The flash drive is a Sandisk Cruzer Mini which is USB 
> > > 2.0 with fallback to 1.1.  The computer is a Dell Dimension 4600 
> > > which claims eight USB 2.0 connectiors.
> > > Running Windows XP.
> > [snip]
> >
> > (EIGHT USB 2.0 connectors?  Wowzers!  This USB fad just might be 
> > catching on! ;-))
> 
> You better believe it! :-D
> <http://www.troubles.ru/Humor/Admins_Life/index_files/41.jpg>
>   Igor
> -- 
>   http://cs.nyu.edu/~pechtcha/
>   |\  _,,,---,,_  [EMAIL PROTECTED]
> ZZZzz /,`.-'`'-.  ;-;;,_  [EMAIL PROTECTED]
>  |,4-  ) )-,_. ,\ (  `'-' Igor Pechtchanski, Ph.D.
> '---''(_/--'  `-'\_) fL   a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!
> 
> "Happiness lies in being privileged to work hard for long 
> hours in doing whatever you think is worth doing."  -- Dr. 
> Jubal Harshaw


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: [OT] RE: cp to flash drive very slow

[Gary R. Van Sickle]

> Igor Pechtchanski wrote:
> > On Thu, 30 Sep 2004, Gary R. Van Sickle wrote:
> > 
> >>> Well, yes.  The flash drive is a Sandisk Cruzer Mini which is USB 
> >>> 2.0 with fallback to 1.1.  The computer is a Dell Dimension 4600 
> >>> which claims eight USB 2.0 connectiors.
> >>> Running Windows XP.
> >> [snip]
> >> 
> >> (EIGHT USB 2.0 connectors?  Wowzers!  This USB fad just might be 
> >> catching on! ;-))
> > 
> > You better believe it! :-D
> > 
>  
>   Igor
> 
> http://www.thinkgeek.com/interests/usb/68ce/zoom/
> http://www.thinkgeek.com/interests/usb/6984/zoom/
> http://www.thinkgeek.com/interests/usb/6db9/
> http://www.nerdorama.com/showproduct.php?ProductID=149
> 
> The last two may actually be usefull, well the last one anyway.
> Bill

Yeah, um... wow.

While a USB air purifier and a USB aquarium are nutty enough, the USB cup
warmer is nutty and FWICT a scam.  The max power you can draw from a USB
port is 2.5W, not quite enough to keep your Breakfast Of Champions much
above room temperature.

-- 
Gary R. Van Sickle


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: [OT] RE: cp to flash drive very slow

[Hughes, Bill]

Igor Pechtchanski wrote:
> On Thu, 30 Sep 2004, Gary R. Van Sickle wrote:
> 
>>> Well, yes.  The flash drive is a Sandisk Cruzer Mini which is
>>> USB 2.0 with fallback to 1.1.  The computer is a Dell
>>> Dimension 4600 which claims eight USB 2.0 connectiors.
>>> Running Windows XP.
>> [snip]
>> 
>> (EIGHT USB 2.0 connectors?  Wowzers!  This USB fad just might be
>> catching on! ;-))
> 
> You better believe it! :-D
>  Igor

http://www.thinkgeek.com/interests/usb/68ce/zoom/
http://www.thinkgeek.com/interests/usb/6984/zoom/
http://www.thinkgeek.com/interests/usb/6db9/
http://www.nerdorama.com/showproduct.php?ProductID=149

The last two may actually be usefull, well the last one anyway.
Bill
-- 
   ___
  oo  // \\  "De Chelonian Mobile"
 (_,\/ \_/ \ TortoiseSVN
   \ \_/_\_/>The coolest Interface to (Sub)Version Control
   /_/   \_\ http://tortoisesvn.tigris.org

This e-mail transmission is strictly confidential and intended solely
for the person or organisation to whom it is addressed. It may contain
privileged and confidential information and if you are not the intended
recipient, you must not copy, distribute or take any action in reliance
on it. If you have received this email in error, please reply to the
sender as soon as possible and delete the message. Please note that we
are able to, and reserve the right to, monitor e-mail communications
passing through our network.

The views expressed in this email are not that of the company unless
specified within the message.

The inclusion of this footnote indicates that the mail message and any
attachments have been checked for the presence of known viruses.

If you have any comments regarding our policy please direct them to
[EMAIL PROTECTED]

This email has been scanned for all viruses by the MessageLabs Email
Security System. For more information on a proactive email security
service working around the clock, around the globe, visit
http://www.messagelabs.com


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

[OT] RE: cp to flash drive very slow

[Igor Pechtchanski]

On Thu, 30 Sep 2004, Gary R. Van Sickle wrote:

> > Well, yes.  The flash drive is a Sandisk Cruzer Mini which is
> > USB 2.0 with fallback to 1.1.  The computer is a Dell
> > Dimension 4600 which claims eight USB 2.0 connectiors.
> > Running Windows XP.
> [snip]
>
> (EIGHT USB 2.0 connectors?  Wowzers!  This USB fad just might be catching
> on! ;-))

You better believe it! :-D

Igor
-- 
http://cs.nyu.edu/~pechtcha/
  |\  _,,,---,,_[EMAIL PROTECTED]
ZZZzz /,`.-'`'-.  ;-;;,_[EMAIL PROTECTED]
 |,4-  ) )-,_. ,\ (  `'-'   Igor Pechtchanski, Ph.D.
'---''(_/--'  `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

"Happiness lies in being privileged to work hard for long hours in doing
whatever you think is worth doing."  -- Dr. Jubal Harshaw

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cp to flash drive very slow

[Gary R. Van Sickle]

> Well, yes.  The flash drive is a Sandisk Cruzer Mini which is 
> USB 2.0 with fallback to 1.1.  The computer is a Dell 
> Dimension 4600 which claims eight USB 2.0 connectiors.  
> Running Windows XP.
> 

Ok, like Hannu said, it's a USB 2.0 connection then, as long as you don't
have any USB 1.1 hubs in between the two.  I.e., if you're plugging the
drive directly into the computer, it's 2.0 all the way.

(EIGHT USB 2.0 connectors?  Wowzers!  This USB fad just might be catching
on! ;-))

> I don't think caching is the difference.  I was able to unzip 
> the .zip file right after xcopy had copied it.
> 

Well, that's exactly what caching is intended for.  The first xcopy copies
the file to a RAM buffer, and the RAM buffer gets sputtered out to the drive
in the background.  The unzip, in the reverse direction, would end up
reading from that same RAM buffer.

> Is there anything faster than cp for copying out of the 
> cygwin tools?  I'm working out some scripts to share files 
> between my work machine and my home machine using the flash 
> drive.  I could use xcopy but then I'd have to go through the 
> filename translation doohicky.

The latter is what I ended up having to do, but it wasn't for speed reasons
alone (cp was corrupting files over the network "back in the day").  I can't
think of any other alternatives offhand that would make any sense to do a
local copy like this.

What happens if you cp between two hard drives, or across the network?  Same
crazy slowness?

-- 
Gary R. Van Sickle


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cp to flash drive very slow

[Hannu E K Nevalainen]

you ([EMAIL PROTECTED]) wrote on  :

> Well, yes.  The flash drive is a Sandisk Cruzer Mini which is USB 2.0
> with fallback to 1.1.  The computer is a Dell Dimension 4600 which
> claims eight USB 2.0 connectiors.  Running Windows XP.

Right, then we know. ;-)
 
> I don't think caching is the difference.  I was able to unzip the .zip
> file right after xcopy had copied it.

...which doesn't proove a thing about caching or not.
 
> Is there anything faster than cp for copying out of the cygwin tools?
> I'm working out some scripts to share files between my work machine
> and my home machine using the flash drive.  I could use xcopy but
> then I'd have to go through the filename translation doohicky.
> 
> dar

*** Untested  suggestion:

---
#!/bin/bash

src="/home/dar/personal-projects /projects/"
dst="/cygdrive/z" # for the USB drive

for d in $src; do
  find $d -type d | (
  while read dir; do
# cannot be used on SHARES
mkdir -p "$dst/$dir"
# assuming xcopy is in PATH
xcopy "$(cygpath -D "$dir")" "$(cygpath -D "$dst/$dir")"
  done
  )
done
---

Does it need to be more complex?

Otherwise the cygwin 'rsync' or 'unison' packages might be useful... 
Others might be able to help with them.

/Hannu E K Nevalainen, B.Sc. EE Microcomputer systems--72-->

** mailing list preference; please keep replies on list **

-- printf("LocalTime: UTC+%02d\n",(DST)? 2:1); --
--END OF MESSAGE--

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cp to flash drive very slow

[David A. Rogers]

Well, yes.  The flash drive is a Sandisk Cruzer Mini which is USB 2.0 with
fallback to 1.1.  The computer is a Dell Dimension 4600 which claims eight
USB 2.0 connectiors.  Running Windows XP.

I don't think caching is the difference.  I was able to unzip the .zip
file right after xcopy had copied it.

Is there anything faster than cp for copying out of the cygwin tools?  I'm
working out some scripts to share files between my work machine and my
home machine using the flash drive.  I could use xcopy but then I'd have
to go through the filename translation doohicky.

dar


On Thu, 30 Sep 2004, Hannu E K Nevalainen wrote:

> you wrote:
>
> > Thanks for responding, Gary.
> >
> >> Regardless, <3.7Mb/second seems like something's wrong somewhere.
> >> Are you running USB2.0 hub-to-device?
> >
> > I dunno.  I'm not very knowledgable about hardware esp. USB.  How
> > would I tell?
> >
> > dar
>
> Sorry for butting in...
>
>  I'd say it should've read "USB 2.0" very clearly, somewhere on that
> USB-thingie.
>
>  Sometimes you can tell whether there is some caching going on by simply
> moving everything away from the mounted drive (i.e. move Explorer's viewing
> off the drive) and then submit an "Right click->Eject" on the drive letter.
>
> If the Eject makes the USB-drive-led flicker, there was something cached.
>
>
> /Hannu E K Nevalainen, B.Sc. EE Microcomputer systems--72-->
>
> ** mailing list preference; please keep replies on list **
>
> -- printf("LocalTime: UTC+%02d\n",(DST)? 2:1); --
> --END OF MESSAGE--
>
>
> --
> Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:   http://cygwin.com/problems.html
> Documentation: http://cygwin.com/docs.html
> FAQ:   http://cygwin.com/faq/
>
>
>


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cp to flash drive very slow

[Hannu E K Nevalainen]

you wrote:

> Thanks for responding, Gary.
>
>> Regardless, <3.7Mb/second seems like something's wrong somewhere.
>> Are you running USB2.0 hub-to-device?
>
> I dunno.  I'm not very knowledgable about hardware esp. USB.  How
> would I tell?
>
> dar

Sorry for butting in...

 I'd say it should've read "USB 2.0" very clearly, somewhere on that
USB-thingie.

 Sometimes you can tell whether there is some caching going on by simply
moving everything away from the mounted drive (i.e. move Explorer's viewing
off the drive) and then submit an "Right click->Eject" on the drive letter.

If the Eject makes the USB-drive-led flicker, there was something cached.


/Hannu E K Nevalainen, B.Sc. EE Microcomputer systems--72-->

** mailing list preference; please keep replies on list **

-- printf("LocalTime: UTC+%02d\n",(DST)? 2:1); --
--END OF MESSAGE--


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cp to flash drive very slow

[David A. Rogers]

Thanks for responding, Gary.

> Regardless, <3.7Mb/second seems like something's wrong somewhere.  Are you
> running USB2.0 hub-to-device?

I dunno.  I'm not very knowledgable about hardware esp. USB.  How would I
tell?

dar

On Wed, 29 Sep 2004, Gary R. Van Sickle wrote:

> > I tried using cp to copy a zip file 106MB from my hard drive
> > to my flash drive (sandisk mini cruzer).  After 20 minutes it
> > still had not completed.
> >
> > xcopy copied the file in 22 seconds.
> >
> > Why would cp be so much slower?  Any ideas as to work-arounds?
> >
>
> Last I checked, cp was slower on network copies than xcopy was, but the
> difference was nowhere *near* that dramatic.  I can offer a few guesses
> here:
>
> 1.  Again last I looked, cp was using fopen()/fread() et al to do the copy.
> Good for portability, bad for efficiency.  Xcopy is probably using
> CopyFile{Ex} or some such lower-level funcion, which if MS is on the ball
> (yeah I know) involves a lot fewer layers of code, and if we're really good
> maybe is even copying raw sectors using scatter/gather (yeah I know I'm
> dreaming, but maybe).
>
> 2.  Caching.  Xcopy may be caching your writes to flash, cp may be forcing a
> flush somehow.  I've had similar copies take essentially no time, only to
> find out that the copy never actually got committed to disk until much much
> later.  XP SP1 doesn't default to that behavior IIRC, but check to make sure
> that you do NOT have that option turned on, or you WILL lose data.
>
> Regardless, <3.7Mb/second seems like something's wrong somewhere.  Are you
> running USB2.0 hub-to-device?
>
>


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cp to flash drive very slow

[Gary R. Van Sickle]

> I tried using cp to copy a zip file 106MB from my hard drive 
> to my flash drive (sandisk mini cruzer).  After 20 minutes it 
> still had not completed.
> 
> xcopy copied the file in 22 seconds.
> 
> Why would cp be so much slower?  Any ideas as to work-arounds?
> 

Last I checked, cp was slower on network copies than xcopy was, but the
difference was nowhere *near* that dramatic.  I can offer a few guesses
here:

1.  Again last I looked, cp was using fopen()/fread() et al to do the copy.
Good for portability, bad for efficiency.  Xcopy is probably using
CopyFile{Ex} or some such lower-level funcion, which if MS is on the ball
(yeah I know) involves a lot fewer layers of code, and if we're really good
maybe is even copying raw sectors using scatter/gather (yeah I know I'm
dreaming, but maybe).

2.  Caching.  Xcopy may be caching your writes to flash, cp may be forcing a
flush somehow.  I've had similar copies take essentially no time, only to
find out that the copy never actually got committed to disk until much much
later.  XP SP1 doesn't default to that behavior IIRC, but check to make sure
that you do NOT have that option turned on, or you WILL lose data.

Regardless, <3.7Mb/second seems like something's wrong somewhere.  Are you
running USB2.0 hub-to-device?

-- 
Gary R. Van Sickle
 




--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/