Re: [ZODB-Dev] Copying zodb's with relstorage

[Shane Hathaway]

On 02/10/2011 09:27 AM, Santi Camps wrote:
>
>
> On Thu, Feb 10, 2011 at 5:07 PM, Shane Hathaway  > wrote:
>
> On 02/10/2011 08:42 AM, Santi Camps wrote:
>
> The objective is to duplicate a storage using different mount
> points.
>   For instance, if we have  Database1 -> mount_point_1 , create
> Database2 and Database3 as copies of Database1 (using pg_dump &
> pg_restore), and then mount them as mount_point_2 and mount_point_3
>
>
> Yes, but why do you want to do that?  There might be a better way to
> accomplish what you're trying to do, or perhaps what you're doing is
> the right thing to do but there's some bug and you need to describe
> why that bug is important.
>
>
> Well, the original is a base site I want to replicate to be used for
> different clients.  I often have done that using mount points and zexp's
> export/import, but was trying to do it copying the database to avoid the
> long and heavy import process.

I see.  I don't know why the database seemed empty, but even after you 
fix that, assuming you're using Plone or similar, you're going to face a 
deeper problem: the catalog(s) in each site use absolute paths.  To fix 
that, you will have to re-catalog everything in each site, which is 
probably the most expensive part of the ZEXP import.  In other words, I 
suspect you won't actually save any time with this method of copying.

Another possible solution is to shrink your template so that it's not so 
expensive to copy.

If I have understood correctly, then this question is independent of the 
ZODB storage--you would have exactly the same problem with FileStorage 
or ZEO.

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Copying zodb's with relstorage

[Santi Camps]

On Thu, Feb 10, 2011 at 5:07 PM, Shane Hathaway wrote:

> On 02/10/2011 08:42 AM, Santi Camps wrote:
>
>> The objective is to duplicate a storage using different mount points.
>>  For instance, if we have  Database1 -> mount_point_1 , create
>> Database2 and Database3 as copies of Database1 (using pg_dump &
>> pg_restore), and then mount them as mount_point_2 and mount_point_3
>>
>
> Yes, but why do you want to do that?  There might be a better way to
> accomplish what you're trying to do, or perhaps what you're doing is the
> right thing to do but there's some bug and you need to describe why that bug
> is important.
>
>
Well, the original is a base site I want to replicate to be used for
different clients.  I often have done that using mount points and zexp's
export/import, but was trying to do it copying the database to avoid the
long and heavy import process.

Thanks

Santi Camps
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Copying zodb's with relstorage

[Shane Hathaway]

On 02/10/2011 08:42 AM, Santi Camps wrote:
> The objective is to duplicate a storage using different mount points.
>   For instance, if we have  Database1 -> mount_point_1 , create
> Database2 and Database3 as copies of Database1 (using pg_dump &
> pg_restore), and then mount them as mount_point_2 and mount_point_3

Yes, but why do you want to do that?  There might be a better way to 
accomplish what you're trying to do, or perhaps what you're doing is the 
right thing to do but there's some bug and you need to describe why that 
bug is important.

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Copying zodb's with relstorage

[Santi Camps]

On Thu, Feb 10, 2011 at 3:46 PM, Shane Hathaway wrote:

> On 02/10/2011 07:41 AM, Shane Hathaway wrote:
>
>> On 02/10/2011 06:30 AM, Santi Camps wrote:
>>
>>> I was trying to move a database copy a relstorage zodb and having some
>>> issues.  The original zodb is mounted using a mount point /original_path
>>>If I restore the backup of the database and mount it using exactly
>>> the same mount point /original_path in the destination zope, all goes
>>> right.   But what I want is to replicate the original database N times,
>>> so need to have /destination_pathN in the mount point.  When I do that,
>>> the database seems empty (no object is shown in the mounted point).
>>>
>>
>> The normal way to copy objects in ZODB is through ZEXP export and
>> import.  Have you tried that?  Also, what is your goal?
>>
>
> Um, I see you mentioned that you're trying to avoid ZEXP.  Ok. ;-)
>
> The question remains: What are you trying to accomplish?
>
>
The objective is to duplicate a storage using different mount points.  For
instance, if we have  Database1 -> mount_point_1 , create Database2 and
Database3 as copies of Database1 (using pg_dump & pg_restore), and then
mount them as mount_point_2 and mount_point_3

It works if I mount Database2 or Database3 with the same mount_point name
(mount_point_1), but when mount point name changes, objects inside are not
seen.  It seems like a problem with some reference to the root object in the
ZODB, but not able to fix it (don't now if it's possible, in fact)

Thanks for your answers

Santi Camps
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Copying zodb's with relstorage

[Shane Hathaway]

On 02/10/2011 07:41 AM, Shane Hathaway wrote:
> On 02/10/2011 06:30 AM, Santi Camps wrote:
>> I was trying to move a database copy a relstorage zodb and having some
>> issues.  The original zodb is mounted using a mount point /original_path
>> If I restore the backup of the database and mount it using exactly
>> the same mount point /original_path in the destination zope, all goes
>> right.   But what I want is to replicate the original database N times,
>> so need to have /destination_pathN in the mount point.  When I do that,
>> the database seems empty (no object is shown in the mounted point).
>
> The normal way to copy objects in ZODB is through ZEXP export and
> import.  Have you tried that?  Also, what is your goal?

Um, I see you mentioned that you're trying to avoid ZEXP.  Ok. ;-)

The question remains: What are you trying to accomplish?

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Copying zodb's with relstorage

[Shane Hathaway]

On 02/10/2011 06:30 AM, Santi Camps wrote:
> I was trying to move a database copy a relstorage zodb and having some
> issues.  The original zodb is mounted using a mount point /original_path
>If I restore the backup of the database and mount it using exactly
> the same mount point /original_path in the destination zope, all goes
> right.   But what I want is to replicate the original database N times,
> so need to have /destination_pathN in the mount point.  When I do that,
> the database seems empty (no object is shown in the mounted point).

The normal way to copy objects in ZODB is through ZEXP export and 
import.  Have you tried that?  Also, what is your goal?

> Is there any way to fix this updating registers in SQL ?   I know a way
> to solve it might be to export and import ZEXP, but the database is very
> big and I'm trying to avoid it  I've tried these 2 queries but no effect
> obtained, these fields seems just informative:
>
> update transaction set description=replace(description::text,
> 'helpdesk_src', 'redesistemas')::bytea;
> update transaction set username=replace(username::text, 'helpdesk_src',
> 'redesistemas')::bytea;

The username and description in the transaction table affect only what 
you see in the undo log.  I don't think they have any connection with 
the problem you are trying to solve.

> Where is stored the information about parent -> children objects ?  Is
> the prev_tid field of object_state table ?

No.  RelStorage stores all such information in pickles, just like 
FileStorage or any other ZODB storage.

If you're using RelStorage because you expect to be able to access and 
manipulate the object database using SQL, I'm afraid you will be 
disappointed.  All ZODB storages store pickles, and pickles are opaque 
to SQL.

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev