Re: [BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Les Mikesell
On Fri, Dec 16, 2011 at 9:00 AM, Jean Spirat wrote: > >> Excuse my "off topic"-ness, but with that many small files I kind of expect a >> filesystem to reach certain limits. Why is that webapp written to use many >> little files? Why not with a database where all that stuff is in blobs? >> That wh

Re: [BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Jean Spirat
> Excuse my "off topic"-ness, but with that many small files I kind of expect a > filesystem to reach certain limits. Why is that webapp written to use many > little files? Why not with a database where all that stuff is in blobs? > That whould be easier to maintain and easier to back up. > > Have

Re: [BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Arnold Krille
Hi, On Friday 16 December 2011 10:42:00 Jean Spirat wrote: > I use backuppc to save a webserver. The issue is that the application > used on it is making thousand of little files used for a game to create > maps and various things. The issue is that we are now at 100GB of data > and 8.030.000 fi

Re: [BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Tim Fletcher
On Fri, 2011-12-16 at 07:33 -0600, Les Mikesell wrote: > On Fri, Dec 16, 2011 at 4:49 AM, Jean Spirat wrote: > > for my understanding rsync had allways seems to be the most efficient > > of the two but i never challenged this "fact" ;p > > Rsync working natively is very efficient, but think abo

Re: [BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Steve
On Fri, Dec 16, 2011 at 4:42 AM, Jean Spirat wrote: > The issue is that we are now at 100GB of data > and 8.030.000 files so the backups takes 48H and more (to help the files > are on NFS share). I think i come to the point where file backup is at > it's limit. What about a script on this machine

Re: [BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Les Mikesell
On Fri, Dec 16, 2011 at 4:49 AM, Jean Spirat wrote: > > Hum i cannot directly use the FS i have no access to the NFS server that > is on the hosting company side i just have access to the webserver that > use the nfs partition to store  it's content. Right now i also mount the > nfs share on the b

Re: [BackupPC-users] Scary problem with USB3...

2011-12-16 Thread hansbkk
I highly recommend **against** using any protocol conversion in the mix "USB to eSATA" or whatever. True eSATA is fine - obviously the quality of the hardware is an issue. Firewire is also OK but getting rarer these days. Internal SATA to eSATA should also not be a problem, not really doing any c

Re: [BackupPC-users] Scary problem with USB3...

2011-12-16 Thread Pavel Hofman
Dne 16.12.2011 10:18, Tim Fletcher napsal(a): > On Thu, 2011-12-15 at 16:38 -0500, Zach La Celle wrote: > >> Regarding some other responses, I'll be sure to try eSATA next time >> instead of USB to see if that tends to be more stable. I could also use >> internal drives, I suppose...the real re

Re: [BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Tim Fletcher
On Fri, 2011-12-16 at 11:49 +0100, Jean Spirat wrote: > > I would suggest you try the following: > > tar+ssh backups however use more bandwidth but as you are already using > > nfs I am assuming you are on a local network of some sort. > for my understanding rsync had allways seems to be the mo

Re: [BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Jean Spirat
r. > > > I would suggest you try the following: > > Move to tar over ssh on the remote webserver, the first full backup > might well take a long time but the following ones should be faster. > > tar+ssh backups however use more bandwidth but as you are already using > nfs I am assuming you are on

Re: [BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Tim Fletcher
On Fri, 2011-12-16 at 10:42 +0100, Jean Spirat wrote: > hi, > > I use backuppc to save a webserver. The issue is that the application > used on it is making thousand of little files used for a game to create > maps and various things. The issue is that we are now at 100GB of data > and 8.030.

[BackupPC-users] 8.030.000, Too much files to backup ?

2011-12-16 Thread Jean Spirat
hi, I use backuppc to save a webserver. The issue is that the application used on it is making thousand of little files used for a game to create maps and various things. The issue is that we are now at 100GB of data and 8.030.000 files so the backups takes 48H and more (to help the files ar

Re: [BackupPC-users] Scary problem with USB3...

2011-12-16 Thread Tim Fletcher
On Thu, 2011-12-15 at 16:38 -0500, Zach La Celle wrote: > Regarding some other responses, I'll be sure to try eSATA next time > instead of USB to see if that tends to be more stable. I could also use > internal drives, I suppose...the real reason we're using external disks > is so that we can rep