Re: [FOSSology] Combining databases from several servers

2009-07-29 Thread Bob Gobeille


On Jul 27, 2009, at 11:40 AM, thomas.j.mur...@kodak.com wrote:


I  believe your 3rd suggestion is what I'm leaning toward ---  I  
have asked each product team to do a pg_dump to archive the  
database   would be nice to then 'import/combine' them into a  
central database


Hi Tom,
I was hoping to find a postgresql migration utility that allows me to  
do the import/combine.  So far, no luck.   The only suggestion that  
has come up is to change our primary/foreign keys to prefix a server  
ID.   There are different ways we could implement this, but code/ 
schema changes are needed.


If you haven't added any licenses or agents, and you only want license  
data (not data from pkgmetagetta or specagent), and you throw away all  
the job queue history.  I think the only keys you need to update are  
for folders, pfile, upload, uploadtree, and users.


Some possibilities:
Is this something you (or any programmer) are interested in doing  
yourself?
You could hire HP Services, or a fossology developer, or  anyone to  
make the changes.

You could punt and simply reanalyze the code on your master server.
You could manually tweek the primary keys (need to set the top 3 bits  
to make unique keys for your 7 servers).  Then a normal import might  
work.

You could write a program to rewrite the primary keys.
You could wait and we can try and get this into v 1.3

You do realize that in addition to the db, you have to copy over your  
repository?  That's a simple rsync but maybe a lot of data.


Any other ideas?

Bob___
fossology mailing list
fossology@fossology.org
http://fossology.org/mailman/listinfo/fossology


Re: [FOSSology] Combining databases from several servers

2009-07-29 Thread thomas . j . murray
Bob,

Thanks for the suggestions

I've tried the pg_restore to import to a 'empty' fossology database and it 
appears to work (data only and no repository file info only the 
analysis results)  but it won't allow me to add another set of 
database data (most likely due to the primary key conflicts)

I may try again to get the product teams to submit their source for 
centralized scanning/analysis but they were reluctant to ship their code 
(various reasons of security and control...).   So I think a future 
feature to aggregate analysis data from several databases would be great.

I'll consider the short term suggestions and maybe we can develop a 
solution

thanks

tom





Bob Gobeille bob.gobei...@hp.com 
07/29/2009 11:13 AM

To
thomas.j.mur...@kodak.com
cc
fossology@fossology.org fossology@fossology.org
Subject
Re: [FOSSology] Combining databases from several servers







On Jul 27, 2009, at 11:40 AM, thomas.j.mur...@kodak.com wrote:

I  believe your 3rd suggestion is what I'm leaning toward ---  I have 
asked each product team to do a pg_dump to archive the database would 
be nice to then 'import/combine' them into a central database 

Hi Tom,
I was hoping to find a postgresql migration utility that allows me to do 
the import/combine.  So far, no luck.   The only suggestion that has come 
up is to change our primary/foreign keys to prefix a server ID.   There 
are different ways we could implement this, but code/schema changes are 
needed.

If you haven't added any licenses or agents, and you only want license 
data (not data from pkgmetagetta or specagent), and you throw away all the 
job queue history.  I think the only keys you need to update are for 
folders, pfile, upload, uploadtree, and users.

Some possibilities:
Is this something you (or any programmer) are interested in doing 
yourself?
You could hire HP Services, or a fossology developer, or  anyone to make 
the changes.
You could punt and simply reanalyze the code on your master server.
You could manually tweek the primary keys (need to set the top 3 bits to 
make unique keys for your 7 servers).  Then a normal import might work.
You could write a program to rewrite the primary keys.
You could wait and we can try and get this into v 1.3

You do realize that in addition to the db, you have to copy over your 
repository?  That's a simple rsync but maybe a lot of data.

Any other ideas?

Bob
___
fossology mailing list
fossology@fossology.org
http://fossology.org/mailman/listinfo/fossology