Re: [Zope] Re: Zeo as a Zeo Client

2006-05-04 Thread Chris Withers

Gerhard Schmidt wrote:
If I had the Time I whould do it. My ToDo List goes around up to the moon 
an back twice. About the funding I have to talk to my superior when he is 
back from his vacation.


Bear in mind that you will need serious amounts of cash to get this 
implemented. This is a massive massive change.


I'd suggest the money is better spent to purchasing yourself a ZRS 
license ;-)


cheers,

Chris

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-03 Thread Gerhard Schmidt
On Tue, May 02, 2006 at 10:23:40AM -0400, Tres Seaver wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 Gerhard Schmidt wrote:
 
  I try to reduce the load of the line between the backup Computing Center 
  and the Mainsite by having a zeo server as Proxy between the zope server 
  at the backup site an the ZEO at the main site. 
  
  Secound part is that the zeo at the backupsite can easily reconfigured in 
  a normal ZEO when the mainsite is offline. So I don't have to reconfigure 
  all zeoclients at the backupsite. The Data.fs is copied  every hour to the 
  backup site so that a have an up to one our backup of the data.fs in case 
  of a desaster at the main site. 
  
  A configuration like that is described in the Zope Book on page 230. 
  
  Besides I have found that with a growing number of zeo clients the Zeo 
  server gets slower but neither the CPU nor the Harddisk IO is at the limit. 
  We have a load of 0.2 to 0.3 and disk IO arrond 2-3 MB/sec. We have 12 
  zeoclients at the moment and 12 more are planed for the backup site. 
 
 I would look for a replication strategy to create your intermediate
 storage server:  the setup you are trying is not supported by the
 current ZEO setup.  Such strategies include:
 
   - Zope Corp's Zope Replication Services product, which keeps the
 secondary storage servers synchronized with the primary via
 the spread toolkit.
 
   - DirectoryStorage can be used to do replication via rsync.
 
   - Another possibility would be to use 'repozo' to create deltas
 on the primary, and then propagate them to the secondary via
 rsync, then apply them via 'repozo'.

I have never said that I want to have an realtime replaication. All I 
want is a ZEO at the backup site that forwards the request to the main 
site. Thats to provide a single point where i have to change the 
configuartion when a desaster at the main site happens. I simply don't 
want to change the configuration of all zope server at the backup site 
which are 12 at the first step an will grow as needed. 

As I said. The is a notable drop in perfomance with growing number auf 
connected clients. even if the clients aren't fetching objekts. I think 
this is becaus auf the growing invalidation overhead. We have a site 
with many write requests. So I hoped when i have a second zeo that the 
performance loss can be reduced. That's the second reason for the for 
this setup. 

As i figured that the cache is implemented in the zeoclient not in the 
zope server itself i thought ist might get me some proxy capabilities. 
But thats whould have been the sugar on the Top nothing realy needed as 
we have a 1 GBit connect between main and backup site. 

The question is why is it impossible to run a zeo as a zeoclient. I can 
setup a zeo with a zeoclient storage i See teh invalidations coming from 
the backend zeo. So this part works. The part with the client connect 
works also. Just the connect between both is missing. And as i said 
such a config is descibed in the Zope Book 2.5 version on page 230. 
Why has the support for such configurations droped in newer versions. 

We are planing to purchase ZRS later this year to setup an automatic 
failover. But as mentioned here. We can't use the ZRS backup servers 
when running in normal mode becaus they are read only. So I need an setup 
wer I can run a normal zeo in zeoclient mode when the mainsite is online. 
This zeo will be shutdown and replaced by the ZRS backup when the mainsite 
goes down. Otherwise i have to mess with IP takeover and other very messy 
strategies. I'm trying to get this as simple and as stable as possible. 

Bye 
Estartu


Gerhard Schmidt| Nick : estartu  IRC : Estartu  |
Fischbachweg 3 ||  PGP Public Key
86856 Hiltenfingen | EMail: [EMAIL PROTECTED]  |  on request 
Germany||  

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-03 Thread Andreas Jung



--On 3. Mai 2006 08:11:44 +0200 Gerhard Schmidt [EMAIL PROTECTED] wrote:


The question is why is it impossible to run a zeo as a zeoclient.


Because this usecase was/is never supported.


I can
setup a zeo with a zeoclient storage i See teh invalidations coming from
the backend zeo. So this part works. The part with the client connect
works also. Just the connect between both is missing. And as i said
such a config is descibed in the Zope Book 2.5 version on page 230.


We don't have a paper copy at hand. I can not find any description of your 
usecase in the 2.7 edition of the Zope Book...please verify it.



Why has the support for such configurations droped in newer versions.


This was never supported.  I think you are misreading something...

-aj


--
ZOPYX Ltd.  Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany
Web: www.zopyx.com - Email: [EMAIL PROTECTED] - Phone +49 - 7071 - 793376
E-Publishing, Python, Zope  Plone development, Consulting


pgpiEE4aA88pI.pgp
Description: PGP signature
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-03 Thread Gerhard Schmidt
On Wed, May 03, 2006 at 08:22:22AM +0200, Andreas Jung wrote:
 
 
 --On 3. Mai 2006 08:11:44 +0200 Gerhard Schmidt [EMAIL PROTECTED] wrote:
 
 The question is why is it impossible to run a zeo as a zeoclient.
 
 Because this usecase was/is never supported.
 
 I can
 setup a zeo with a zeoclient storage i See teh invalidations coming from
 the backend zeo. So this part works. The part with the client connect
 works also. Just the connect between both is missing. And as i said
 such a config is descibed in the Zope Book 2.5 version on page 230.
 
 We don't have a paper copy at hand. I can not find any description of your 
 usecase in the 2.7 edition of the Zope Book...please verify it.

It was lost in the 2.6 version of the Zope book i have scanned the Page 
with the Picture. See http://etustar.ze.tum.de/zopebook.jpg  
 
Bye
Estartu


Gerhard Schmidt| Nick : estartu  IRC : Estartu  |
Fischbachweg 3 ||  PGP Public Key
86856 Hiltenfingen | EMail: [EMAIL PROTECTED]  |  on request 
Germany||  

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-03 Thread Andreas Jung



--On 3. Mai 2006 08:35:48 +0200 Gerhard Schmidt [EMAIL PROTECTED] wrote:




It was lost in the 2.6 version of the Zope book i have scanned the Page
with the Picture. See http://etustar.ze.tum.de/zopebook.jpg



Scary...no idea why it is in the printed edition...at least the 2.6 and 2.7 
edition does not show such a setup. As mention earlier such a setup is not 
supported and not supposed to work. *At least* it is *extremely* uncommon.


-aj


--
ZOPYX Ltd.  Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany
Web: www.zopyx.com - Email: [EMAIL PROTECTED] - Phone +49 - 7071 - 793376
E-Publishing, Python, Zope  Plone development, Consulting


pgpjA3OZDcQGj.pgp
Description: PGP signature
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-03 Thread Gerhard Schmidt
On Wed, May 03, 2006 at 08:56:28AM +0200, Andreas Jung wrote:
 
 
 --On 3. Mai 2006 08:35:48 +0200 Gerhard Schmidt [EMAIL PROTECTED] wrote:
 
 
 
 It was lost in the 2.6 version of the Zope book i have scanned the Page
 with the Picture. See http://etustar.ze.tum.de/zopebook.jpg
 
 
 Scary...no idea why it is in the printed edition...at least the 2.6 and 2.7 
 edition does not show such a setup. As mention earlier such a setup is not 
 supported and not supposed to work. *At least* it is *extremely* uncommon.

Every major innovation was extremely uncommon bevor it was implemented. So 
thats not a reason not to do it. Is there a way to get this to work. I 
Think it whould be a very nice feature. Because it whould increase the 
scalability. When we can bring the cache to work it will improve 
perfomance for large sites as well. 

Bye
Estartu


Gerhard Schmidt| Nick : estartu  IRC : Estartu  |
Fischbachweg 3 ||  PGP Public Key
86856 Hiltenfingen | EMail: [EMAIL PROTECTED]  |  on request 
Germany||  

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-03 Thread Andreas Jung



--On 3. Mai 2006 09:15:45 +0200 Gerhard Schmidt [EMAIL PROTECTED] wrote:



Every major innovation was extremely uncommon bevor it was implemented.
So  thats not a reason not to do it. Is there a way to get this to work.
I  Think it whould be a very nice feature. Because it whould increase the
scalability. When we can bring the cache to work it will improve
perfomance for large sites as well.


There are lot of things that would be nice if they were implemented..
The best chance to get this feature into Zope is either to implement it 
yourself or by funding the development.


-aj

pgpjhKorwcsYL.pgp
Description: PGP signature
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-03 Thread Gerhard Schmidt
On Wed, May 03, 2006 at 09:19:31AM +0200, Andreas Jung wrote:
 
 
 --On 3. Mai 2006 09:15:45 +0200 Gerhard Schmidt [EMAIL PROTECTED] wrote:
 
 
 Every major innovation was extremely uncommon bevor it was implemented.
 So  thats not a reason not to do it. Is there a way to get this to work.
 I  Think it whould be a very nice feature. Because it whould increase the
 scalability. When we can bring the cache to work it will improve
 perfomance for large sites as well.
 
 There are lot of things that would be nice if they were implemented..
 The best chance to get this feature into Zope is either to implement it 
 yourself or by funding the development.

If I had the Time I whould do it. My ToDo List goes around up to the moon 
an back twice. About the funding I have to talk to my superior when he is 
back from his vacation.

Bye
Estartu


Gerhard Schmidt| Nick : estartu  IRC : Estartu  |
Fischbachweg 3 ||  PGP Public Key
86856 Hiltenfingen | Privat: [EMAIL PROTECTED] |   auf Anfrage/
Tel: 08232 77 36 4 | Dienst: [EMAIL PROTECTED]  |on request
Fax: 08232 77 36 3 ||

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-02 Thread Paul Winkler
On Tue, May 02, 2006 at 10:23:40AM -0400, Tres Seaver wrote:
 I would look for a replication strategy to create your intermediate
 storage server:  the setup you are trying is not supported by the
 current ZEO setup.  Such strategies include:
 
   - Zope Corp's Zope Replication Services product, which keeps the
 secondary storage servers synchronized with the primary via
 the spread toolkit.
 
   - DirectoryStorage can be used to do replication via rsync.

... but you can't use a DirectoryStorage replica as a hot or warm backup.

DirectoryStorage replicas are pulled from the master storage, and no
storage process can be using the replica storage while this pull is
happening - not even in read-only mode. This is documented at
http://dirstorage.sourceforge.net/replica.html

For more background discussion, see these threads - 
why hot failover won't work:
http://sourceforge.net/mailarchive/forum.php?thread_id=4906719forum_id=9987
some talk about cold failover:
http://sourceforge.net/mailarchive/forum.php?thread_id=7828511forum_id=9987

-- 

Paul Winkler
http://www.slinkp.com
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-02 Thread David Pratt

Tres Seaver wrote:


  - Another possibility would be to use 'repozo' to create deltas
on the primary, and then propagate them to the secondary via
rsync, then apply them via 'repozo'.


Hi Tres. I have also been investigating strategies but had not thought 
of this possibility. I am not sure how you would keep the master in sync 
if changes are being made on a local network and also on a remote server 
(master). Wouldn't this result in conflicts if the data were exchange 
back with the master?


Regards,
David
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-02 Thread Andreas Jung



--On 2. Mai 2006 12:38:28 -0300 David Pratt [EMAIL PROTECTED] wrote:


Tres Seaver wrote:


  - Another possibility would be to use 'repozo' to create deltas
on the primary, and then propagate them to the secondary via
rsync, then apply them via 'repozo'.


Hi Tres. I have also been investigating strategies but had not thought of
this possibility. I am not sure how you would keep the master in sync if
changes are being made on a local network and also on a remote server
(master).


repozo does not help you syncing multiple ZEO server..it just performs
an incremental backup of an existing Data.fs file. And I don't know of any 
solution have multiple master ZEO servers. There can be only one.


-aj


--
ZOPYX Ltd.  Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany
Web: www.zopyx.com - Email: [EMAIL PROTECTED] - Phone +49 - 7071 - 793376
E-Publishing, Python, Zope  Plone development, Consulting


pgpQbWhJP1Nl0.pgp
Description: PGP signature
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-02 Thread David Pratt

Andreas Jung wrote:


repozo does not help you syncing multiple ZEO server..it just performs
an incremental backup of an existing Data.fs file. And I don't know of 
any solution have multiple master ZEO servers. There can be only one.


-aj


For sure. I guess the best one could have would be replicated storage 
clients that when offline could operate on their own data. When they go 
online they would first attempt resolve their data with the (master) 
server (synchronizing), and then append each new transaction to the 
local replica as it was occuring (replicating) so that before you went 
offline again, your storage is up to date.


I think this is all possible with a sync server, each client having an 
independent asynchronous loop for synchronizing, and a synchronization 
protocol for two way syncing. Zope has most of this now but it was never 
designed to provide data to the local storage to keep it up to date 
(while the zeo client is being used). This would be quite handy and I 
think there are some possibilities to do this.


I guess in a general way, this is what the spread is doing for ZRS but 
the mechanism and use case is different. I don't think you'd want any 
client to be offline for long with this service with a replicated 
storage client. I think the idea is more of a hot backup in case of 
failure of the main storage but I know little of ZRS.


Regards
David
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-02 Thread Jonathan
If the objective is to have a second ZEO server available as a hot-backup, 
how about the idea of having ZEO clients with the ability to write to two 
separate ZEO servers (ideally running on separate hardware servers) with a 
single 'write' command?  We can accomplish this at the application level, 
but it would be a nice option to have ZEO do it automatically.


Is this a reasonable extension to ZEO?

Jonathan


- Original Message - 
From: David Pratt [EMAIL PROTECTED]

To: Andreas Jung [EMAIL PROTECTED]
Cc: Tres Seaver [EMAIL PROTECTED]; zope@zope.org
Sent: Tuesday, May 02, 2006 1:31 PM
Subject: Re: [Zope] Re: Zeo as a Zeo Client



Andreas Jung wrote:


repozo does not help you syncing multiple ZEO server..it just performs
an incremental backup of an existing Data.fs file. And I don't know of 
any solution have multiple master ZEO servers. There can be only one.


-aj


For sure. I guess the best one could have would be replicated storage 
clients that when offline could operate on their own data. When they go 
online they would first attempt resolve their data with the (master) 
server (synchronizing), and then append each new transaction to the local 
replica as it was occuring (replicating) so that before you went offline 
again, your storage is up to date.


I think this is all possible with a sync server, each client having an 
independent asynchronous loop for synchronizing, and a synchronization 
protocol for two way syncing. Zope has most of this now but it was never 
designed to provide data to the local storage to keep it up to date (while 
the zeo client is being used). This would be quite handy and I think there 
are some possibilities to do this.


I guess in a general way, this is what the spread is doing for ZRS but the 
mechanism and use case is different. I don't think you'd want any client 
to be offline for long with this service with a replicated storage client. 
I think the idea is more of a hot backup in case of failure of the main 
storage but I know little of ZRS.


Regards
David
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )



___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-02 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

David Pratt wrote:
 Andreas Jung wrote:
 

 repozo does not help you syncing multiple ZEO server..it just performs
 an incremental backup of an existing Data.fs file. And I don't know of
 any solution have multiple master ZEO servers. There can be only one.

 -aj
 
 
 For sure. I guess the best one could have would be replicated storage
 clients that when offline could operate on their own data. When they go
 online they would first attempt resolve their data with the (master)
 server (synchronizing), and then append each new transaction to the
 local replica as it was occuring (replicating) so that before you went
 offline again, your storage is up to date.
 
 I think this is all possible with a sync server, each client having an
 independent asynchronous loop for synchronizing, and a synchronization
 protocol for two way syncing. Zope has most of this now but it was never
 designed to provide data to the local storage to keep it up to date
 (while the zeo client is being used). This would be quite handy and I
 think there are some possibilities to do this.
 
 I guess in a general way, this is what the spread is doing for ZRS but
 the mechanism and use case is different. I don't think you'd want any
 client to be offline for long with this service with a replicated
 storage client. I think the idea is more of a hot backup in case of
 failure of the main storage but I know little of ZRS.

ZRS secondaries go into recovery mode whenever they lose connection
with the promary:  that recovery is sufficient to allow it to resync
(even to bring up a new, empty secondary).


Tres.
- --
===
Tres Seaver  +1 202-558-7113  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFEV7xH+gerLs4ltQ4RAnP3AJ9PGvYV1l4fqA9Sx1bnobr/ri4FYACdE1V9
ym8LhU8go+za7q15k7y5NSs=
=tTh+
-END PGP SIGNATURE-
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zeo as a Zeo Client

2006-05-02 Thread David Pratt

Tres Seaver wrote:


ZRS secondaries go into recovery mode whenever they lose connection
with the promary:  that recovery is sufficient to allow it to resync
(even to bring up a new, empty secondary).


Hi Tres. This is sort of how I envisioned ZRS (though I have never seen 
or used it). I thought that perhaps being offline for a good length of 
time (or bringing up a fresh new client) would put strain on the server 
and increase the risk of other replication clients being blocked by the 
process. I am not sure how many clients are typically configured with 
ZRS though or whether this would be an issue.


Regards,
David
___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )