Re: [BackupPC-users] BackupPC 3.3.0 released
Le 18/04/2013 23:41, backu...@kosowsky.org a écrit : > Craig Barratt wrote at about 11:16:31 -0700 on Thursday, April 18, 2013: > > Adam, > > > > Let me look at the email archives to understand this issue better. > > > > I'm happy for other capable developers to contribute to BackupPC, > including > > adding commit and release privileges. But they have to be very good. > > I agree -- I would personnaly LOVE to see somewhere : - a git repository which would encourage contributions (the project maintainer(s) may or may not merge them) - an issue tracker and a feature-requests tracker (look at us digging a mailing list archive to find out which fearure was requested, or worse, resending redundant features requests in this very same list) These tools not only ensure a good quality of developpement, they maintain a healthhy community around it and, most important, extends the lifetime of a project. Besides, SourceForge provides those services, I don't see any reason why not to do it. Cheers, jonathan -- IUEM - Service Informatique rue Dumont D'Urville Technopôle Brest-Iroise 29280 Plouzané France tel: +33 2 98 49 87 94 -- AlienVault Unified Security Management (USM) platform delivers complete security visibility with the essential security capabilities. Easily and efficiently configure, manage, and operate all of your security controls from a single console and one unified framework. Download a free trial. http://p.sf.net/sfu/alienvault_d2d ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC-Visualize - visual plot of host backup durations
Le 28/01/2013 18:46, Carl Cravens a écrit :
> I've written a little tool to help analyze BackupPC "scheduling" that I
> thought others might find useful. It is meant to generate plots that can be
> viewed from the web, but it's just as easily used from the command line.
>
> https://github.com/ravenx99/backuppc-visualize/
>
> Over the past year+ that we've been using BackupPC at my company, we've
> struggled with getting all our machines backed up every 24 hours. We have
> over 60 Linux and Windows hosts, and over half of those are virtual machines
> all attached to a single EMC storage server, and a single full backup of all
> the hosts is at 2 TB of data after dedupe.
>
> Because of the load backups create on our VMs, our default is to blackout 6
> AM to 6 PM, and then allow only trivial machines to back up during the day.
> We restrict the number of concurrent backups to 3, due to the serious load it
> creates on a VM host when more than one of its guests is being backed up at
> the same time.
>
> This has caused a problem with getting backups of critical machines on a
> 24-hour cycle. In order to see how the backups are interacting, I wrote
> BackupPC-Visualize to create a broken-bar plot (a "timeline") indicating the
> time during which each backup ran over the last N days. This aggregate view
> of the data quickly identified our problem machines. (Our solution has been
> to use cron to force the full backups of problem machines to run on the
> weekend. A kludge, and I'd like to find a more elegant solution.)
>
> bpcviz consists of a simple Perl script that gathers and massages the
> /var/lib/backuppc/pc//backups data, and a Ploticus plot script to
> convert the data to visual form. I tried to use Gnuplot initially, but it
> won't do horizontal plots, and it handles multiple-interval time data very
> poorly (via a kludge).
>
> Let me know if you find it useful, and I'm open to considering feature
> requests... you can open an "issue" on Github tagged as "enhancement", or
> just drop me an email if Github is a barrier for you.
>
> (I hope the author of BackupPC doesn't object to the name... it's the most
> obvious way to make it clear that this tool works with BackupPC.)
>
This is realy a good idea. Easy to use and easy to interpret. Thank you.
This is my small contribution :
to avoid the error when parsing a host that has never been backed up,
issue a warn instead of die :
diff --git a/bpcviz-gatherdata b/bpcviz-gatherdata
index e796637..2d1b909 100755
--- a/bpcviz-gatherdata
+++ b/bpcviz-gatherdata
@@ -88,7 +88,8 @@ my @data = ();
# merge all the data files to sort them by time
for my $host (@hosts) {
my $file = "$hostdir/$host/backups";
-open( my $IN, '<', $file ) || die( "Cannot open $file: $!\n" );
+unless (open $IN,'<', $file )
+ { warn( "Cannot open $file: $!\n" ); next ; }
while ( my $line = <$IN> ) {
chomp($line);
my @rec=split(/\t/, $line);
cheers
--
IUEM - Service Informatique
rue Dumont D'Urville
Technopôle Brest-Iroise
29280 Plouzané
France
tel: +33 2 98 49 87 94
--
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
[BackupPC-users] Some usefull features for the future
Hi list, I was recently asked by some coworkers during a backuppc presentation for some usefull functionalities missing in the current stable release. Those requests are : - managing roles for the users connected to the web UI and assigning clients to a group so that users belonging to a role are able to operate only on some of the clients - being able to see the size of a directory in the web UI when browsing a client's backups. What do you think about it, does it sound feasable ? cheers, jonathan -- IUEM - Service Informatique rue Dumont D'Urville Technopôle Brest-Iroise 29280 Plouzané France tel: +33 2 98 49 87 94 -- Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Windows7 vshadow error
Hello, this is not direcly connected to backuppc itself, but I thought I would find people here with a good experience of my problem. I am backing up windows clients to a central server using the shadow copy system. It's alright for windows XP machines. But I have an error on Windows 7. To sum up the whole thing, it's just like : $ vshadow.exe C: Error during the last asynchronous operation. - Returned HRESULT = 0x80042318 - Error text: VSS_E_WRITER_INFRASTRUCTURE - Please re-run VSHADOW.EXE with the /tracing option to get more details Of course, /tracing is not telling anything relevant (but I might be wrong). Besides, the service "Volume Shadow Copy" starts normaly on the windows machine. I have to admit that I don't understand much about how windows is doing all this shadow copy stuff. Did someone come accross this problem and know how to repair it ? Cheers, Jonathan -- IUEM - Service Informatique rue Dumont D'Urville Technopôle Brest-Iroise 29280 Plouzané France tel: +33 2 98 49 87 94 0xA8657ED2.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-dev2dev___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Can't browse bad directory name
Le 09/02/2011 11:55, Jonathan Schaeffer a écrit : > does anybody have an explanation about this error ? > > For the records, I managed to find out the problem. I added some times ago the option "--one-file-system" int the RSyncArgs As, for some machines, /home is on a separate file system, the backup would ignore it, reporting no error of course. cheers, Jonathan 0xA8657ED2.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature -- Free Software Download: Index, Search & Analyze Logs and other IT data in Real-Time with Splunk. Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. http://p.sf.net/sfu/splunk-dev2dev ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Can't browse bad directory name
Hi all, does anybody have an explanation about this error ? The context : One full backup per month, incrmental backups daily on 6 levels, so each week, the invermental level cycles. So the backups are like that : Backup# | Type | Filled | Level 15 | full | yes | 0 16 | incr | no | 1 17 | incr | no | 2 18 | incr | no | 3 19 | incr | no | 4 20 | incr | no | 5 21 | incr | no | 6 22 | incr | no | 1 23 | incr | no | 2 When I browse backups 23 to 21, some directories are missing. But ther was no error in the logs. Thus, when trying to browse a "missing" directory, the web interface give the error : "Can't browse bad directory name" Note that the field "#" indicating the backup number is empty if I browse backups 21, 22 and 23. Backup #20 and previous are clean and restorable. I saw someone else on the list reporting this problem, but there was no explanation about its origin. This is happening to several of my backupped clients. What can I check to understand the problem ? Thanks, Jonathan -- IUEM - Service Informatique rue Dumont D'Urville Technopôle Brest-Iroise 29280 Plouzané France tel: +33 2 98 49 87 94 0xA8657ED2.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Clustering Backuppc
Hi list, We are about to deploy a backup system for a large number of clients (~400), The backupPC server will be a good machine (4 CPUs, loads of RAM) but I'm still concerned about the time the full backup are taking and the number of imultaneous backups. To adress this, I'm considering a setup of N servers, connected to the same file system (through GFS for instance) wich would be the backuppc data directory. || | Server1 | | | | One File System | Server2 |-| (GFS, OCFS, ...) | | | ServerN |-| || 2 possibilities : a. Each backuppc server would be independant, having it's set of clients to backup. They are just storing everything in the same pool. b. All servers share the same set of clients, but I can't see how this would be possible Did anyone setup something like that ? Is it theoretically possible ? Actualy, I see some problems with this settings, file systems Maybe there is a more clever way to build a backuppc cluster. Any suggestion ? Thanks ! Jonathan -- IUEM - Service Informatique rue Dumont D'Urville Technopôle Brest-Iroise 29280 Plouzané France tel: +33 2 98 49 87 94 signature.asc Description: OpenPGP digital signature -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] FSArchiver?
Le 09/12/2010 06:55, hans...@gmail.com a écrit : > I've been investigating how to backup BackupPC's filesystem, > specifically the tree with all the hard links (BTW what's the right > name for it, the one that's not the pool?) > > The goal is to be able to bring a verified-good copy of the whole > volume off-site via a big cahuna sata drive. I'm answering not exactly to your question, but you might be interested in this : If you consider using ZFS as BackupPC's filesystem, there is the awesome combo : zfs snapshot # makes a snapshot of you filesystem, for instance on a daily bases zfs send snapshot | ssh backback "zfs receive" and your filesystem will be exported on host backback AND you will be able to travel in time by munting the daily snapshots. Jonathan > > I don't have enough RAM (or time!) for rsync -H and cp -a > > I was originally looking at block-level partition imaging tools, from > mdmraid (RAID1'ing to a removable drive) to dd to Acronis. > > I'm also looking at BackupPC_tarPCCopy, which seems great, but > > What I'm really looking for is to be able to just mount the resulting > filesystem on any ol' livecd, without having to restore anything, > reconstruct LVM/RAID etc complexities just to get at the data - the > source volume is an LV running on a RAID6 array, but I want the target > partition to be a normal one. > > I've come across this tool: http://www.fsarchiver.org/Main_Page > > Does anyone have experience with it? > > Any and all feedback/suggestions welcome. > > -- > This SF Dev2Dev email is sponsored by: > > WikiLeaks The End of the Free Internet > http://p.sf.net/sfu/therealnews-com > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > -- IUEM - Service Informatique place Nicolas Copernic 29280 Plouzané France tel: +33 2 98 49 87 94 0xA8657ED2.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature -- This SF Dev2Dev email is sponsored by: WikiLeaks The End of the Free Internet http://p.sf.net/sfu/therealnews-com___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Windows client : Easy deployment
Hi all. I'm in the process of bulding a large scale backup system in my institute based on backuppc. So far, things are going according to plan. We built up a backupPC server and a mechanism for seamlessly "install" the client on the machines to backup with to objectives in mind : - doing as secure as possible backups - making the client as easy to deploy as possible So we came out with a nice solution where the client package use a ssh service key to connect to the backuppc server which generates an ssh keypair, sends the public part to the client and registers the client in the configuration files. You can have a look if you are interrested : https://tucuxi.univ-brest.fr/projects/iuem-backuppc-client The deployment is very quick and easy on linux and MaxOS machines but now, I have to deal with the windows machines, and this is totally different. I managed to deploy a client but the process is fastidious and there is no way for my users to do this by themselves. I deployed the windows client according to the different docs on the backupPC Wiki (VShadow, rsync+ssh, cygwin). Question : Did someone already worked on an automated deployment on windows ? Has someone experience with one of those packaging system : * wix * nsis * izpack * inosetup Or would a perl script be more appropriate ? I would appreciate any pertinent input on the matter. Thanks Jonathan 0xA8657ED2.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature -- Virtualization is moving to the mainstream and overtaking non-virtualized environment for deploying applications. Does it make network security easier or more difficult to achieve? Read this whitepaper to separate the two and get a better understanding. http://p.sf.net/sfu/hp-phase2-d2d___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] File restore integrity
On 18/06/2010 20:59, Les Mikesell wrote: > > Again, keep in mind that anything on the system can be subverted by a > good hacker, including these tools and their databases. > Agreed with that. The replication of the backup is also a good idea. Thanks for the help. Jonathan -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] File restore integrity
On 17/06/2010 17:26, Jeffrey J. Kosowsky wrote: > Jonathan Schaeffer wrote at about 16:29:19 +0200 on Thursday, June 17, 2010: > > Is there a checking mechanism to ensure the integrity of the restored > files ? > > i.e. the server can check that the files he is about to restore is the > same as > > the one he stored previously ? > > > > Not automatically or officially. Though it might be a good feature to > add in the future. > > If you use rsync checksum caching, I have written a routine that > allows to check some or all pool or pc files for consistency between > the full file md4 checksum stored by rsync and the actual file > content. > > One could also do other checks such as checking the pool file name > against its contents using the partial file md5sum that backuppc > uses. One could also check the file size stored in the attrib file > against the actual size. Thanks for the quick and helpfull answer. This is great. I'll give it a try. This could be coupled with Nagios to report any anomality. Thanks. -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] File restore integrity
Hi, thanks for your quick answers. On 17/06/2010 17:55, Les Mikesell wrote: > On 6/17/2010 9:29 AM, Jonathan Schaeffer wrote: >> >> I'm administrating a BackupPC server and I'm concerned about the security of >> the >> whole system. > > It is based on controlling access to root and the backuppc user on the > server. I don't see a way around that. Sure, my point is to limit the risks of intrusion on the clients when the server is compromised. But the first thing to do is obviously to protect the server. >> And then, when the client restores the file, it gets an empty file. >> >> Is there a checking mechanism to ensure the integrity of the restored files ? >> i.e. the server can check that the files he is about to restore is the same >> as >> the one he stored previously ? > > If you are going to corrupt something intentionally and you have root > access, you would also be able to replace/bypass any such check. Don't > give anyone you don't trust root access... The problem is not who I give root access but who takes it in my back. First, I will not connect the BackupPC host to the internet Then, I'll use a file alteration monitor on the system to be aware of any modification in the backupPC core programs and in the data stored. iwatch looks like a good candidate. Finaly, I'll keep in mind that anybody gaining root priviledges can basicaly do all imaginable stuff. Still, would'nt it be a nice feature to check the file integrity before the restore ? Not only in case of evil intrusion but also in case of data corruption, preventing corrupted data to be restored. -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] File restore integrity
Hi all, I'm administrating a BackupPC server and I'm concerned about the security of the whole system. I configured the linux clients as unpriviledged users doing sudos for rsyncs to limit the risk of intrusion from the backupPC server to the clients as described in the FAQ : http://backuppc.sourceforge.net/faq/ssh.html#how_can_client_access_as_root_be_avoided But I found a simple way to screw up the client when the backupPC server is corrupted : It is easy to empty some (or all) files of a backup : r...@backuppc:/data/backuppc/pc/172.16.2.44/3/f%2f/fhome/fjschaeff# cat /dev/null > f.bashrc And then, when the client restores the file, it gets an empty file. Is there a checking mechanism to ensure the integrity of the restored files ? i.e. the server can check that the files he is about to restore is the same as the one he stored previously ? Cheers, Jonathan -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
