Re: [ZODB-Dev] Reloading product in 2.8

2005-07-17 Thread Lennart Regebro
On 7/14/05, Dieter Maurer <[EMAIL PROTECTED]> wrote:
> I never used "auto refresh" but always use the manual refresh
> and I am very happy with it.
> 
> Recently, I implemented a "RefreshTool" that refreshes with
> a product also all the products that depend on it.
> With this, almost all refresh related problems disappear (exception:
> the persistency bugs of "auto refresh mode").

very interest ing...
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-16 Thread Dieter Maurer
Chris Withers wrote at 2005-7-15 20:38 +0100:
> ...
>If restarting a Zope client running of a storage server takes more than 
>one or two seconds on "normal" hardware, I reckon we should focus on 
>speeding that up rather than battling with the refresh stuff...

I can tell you that this is quite difficult:

  On my (3 years old) hardware, a restart costs about 5 s.

But, there are further reasons:

  *  colleagues of mine use a single Zope installation as
 a development platform for a group of developers.

 They do not want continued restarts to make product
 modifications visible.

 And they can not use Chris M's "bin/zopectl fg" approach,
 because only a single one could have the Zope process
 on its console (apart from the fact, that "zopectl fg"
 is inferior to detect errors because problem messages
 must be filtered out from other unrelated log messages).

 Refresh fits this bill very nicely...


  *  we use refresh even in a production environment:

 refresh is much less intrusive (as seen by our visitors)
 than a restart. Not only, it saves about 15s of
 restart time (restart is slower there because RAM is tight
 and the code needs to be loaded via NFS); it also
 keeps the session info intact (maintained in a RAM storage).

 As you see, we have high confidence in "refresh" :-)

-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-16 Thread Jim Fulton

Chris Withers wrote:

Dieter Maurer wrote:


  Let a bug occur in some component and then, instead (or in addition)
  of fixing the bug, we say often: rip the component off Zope.

  That was threadened for "Version"s, and "ZClass"es and now
  for "Refresh".



Well, they all don't work right, confuse people are aren't maintained, 
or a combination of all three. I'm sure if people who cared about them 
were willing to work on them (ie: you!) then they might have stayedin 
the core,


ZClasses are still in the core and versions are supported to some
level.  (I wasn't aware that the refresh thing was in the core. Is it?)

but your refusal to sign a contributor agreement means there 
isn't anyone.


Dieter has his reasons, which he has explained.  I'm hopeful that
he will sign the new Zope Foundation contrinutor's agreement.

> Unsupported code is dead code, and unless it needs zero
maintenance, which isn't true for any of the above, then it has no place 
being in the core...


Yup, OTOH, dropped features and backward incompatibility isn't so good.
This is why we should be very selective about what we put in.  (Of course,
we'll still get it wrong from time to time.)  Hopefully,
one of these days, we (including the Python community) will figure out
package management so that add-ons are easier to install and we don't have
to nail everything into the "core" (whatever that is ;) of either Zope
or Python.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-16 Thread Chris Withers

Chris McDonough wrote:

Refresh was never much of a timesaver for me.  Pressing "ctrl-C - up
arrow - return" on the console from which I run Zope is my answer.  I
know lots of people say that this is too slow, but on my most recent
project from the time I type "runzope" to the time I see "ready to
handle requests" is

real0m3.029s
user0m1.857s
sys 0m0.306s

... and I can live with that.


Indeed, this is where we should focus. It gives you a totally clean 
slate each time, and I've been convinced by too much pain and suffering 
over the last 5 or so years, that a clean slate is a very very good thing.


If restarting a Zope client running of a storage server takes more than 
one or two seconds on "normal" hardware, I reckon we should focus on 
speeding that up rather than battling with the refresh stuff...


cheers,

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-16 Thread Chris Withers

Dieter Maurer wrote:

  Let a bug occur in some component and then, instead (or in addition)
  of fixing the bug, we say often: rip the component off Zope.

  That was threadened for "Version"s, and "ZClass"es and now
  for "Refresh".


Well, they all don't work right, confuse people are aren't maintained, 
or a combination of all three. I'm sure if people who cared about them 
were willing to work on them (ie: you!) then they might have stayedin 
the core, but your refusal to sign a contributor agreement means there 
isn't anyone. Unsupported code is dead code, and unless it needs zero 
maintenance, which isn't true for any of the above, then it has no place 
being in the core...


cheers,

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-15 Thread Sidnei da Silva
On Fri, Jul 15, 2005 at 07:49:10PM +0200, Dieter Maurer wrote:
| But, it does not need to fail -- provided that with a product
| all its dependent products are refreshed as well.
| 
| At least, I never saw it failing when I had met this condition...
| 
| And I lost time only when I saw the not very helpful problem messages
| for the first few times (when I did not met this condition).
| 
| But otherwise, refresh saved me *lots* of time...

Same here. If you know the gotchas and you program properly it can
work flawleslly. Hell, I've even been able to refresh Archetypes
products, what can be more prone to fail than that?

-- 
Sidnei da Silva
Enfold Systems, LLC.
http://enfoldsystems.com
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-15 Thread Dieter Maurer
Jim Fulton wrote at 2005-7-14 18:01 -0400:
> ...
>> There is not much "nastiness" that "Refresh" does:
>> 
>>   (Manual) refresh has only a single problem:
>>   it does not refresh dependent products automatically.
>
>It also doesn't handle global data properly.
>
>It tries to do something that Python modules were never
>designed to support, which is to load them more than once.

It reloads the modules and Python has a builtin for this ;-)
(although "Refresh" does not use the builtin but reloads in a slightly
different way).

>When it fails, it does so in subtle ways that cause people
>to lose lots of time.

But, it does not need to fail -- provided that with a product
all its dependent products are refreshed as well.

At least, I never saw it failing when I had met this condition...

And I lost time only when I saw the not very helpful problem messages
for the first few times (when I did not met this condition).

But otherwise, refresh saved me *lots* of time...

-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-15 Thread Jim Fulton

Shane Hathaway wrote:

Jim Fulton wrote:


It also doesn't handle global data properly.

It tries to do something that Python modules were never
designed to support, which is to load them more than once.



However, given the existence of the reload() builtin, someone apparently
believed Python modules *were* designed to support reloading.  Because
reload() is a builtin, Python seems to promise that reload() is
reliable, but in fact it's rarely reliable.  There would be a lot less
confusion if reload() were moved somewhere like the "imp" module.


Yup.


When it fails, it does so in subtle ways that cause people
to lose lots of time.



I agree.  The right way to refresh is to detect code changes (ideally
using Linux's brand new inotify mechanism, or something similar when
inotify is not available), display a "please wait" message to the user,
and restart the process.


Doing so automatically could still be a problem.  If you are developing
you don't want the prpcess to restart automatically because a file
changed.  The change could be incomplete or the process could have
precious non-persistent state.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-15 Thread Jim Fulton

Sidnei da Silva wrote:

On Thu, Jul 14, 2005 at 06:01:26PM -0400, Jim Fulton wrote:
| When it fails, it does so in subtle ways that cause people
| to lose lots of time.

Usually when it fails I just restart. I wouldn't consider that losing
lots of time comparing restarting now and then to restarting
constantly.


The problem is *knowing* that it fails.  I've had a number of occasions
helping people debug deep and mysterious zope "bugs" that turned out
to be reload caused.  It works most of the time and people get lulled
into a false sense of security.  When it fails, they tend to blame
something else and waste lots of time trying to figure out what it is.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-15 Thread Shane Hathaway
Tim Peters wrote:
> [Jim Fulton]
>>I agree.  The right way to refresh is to detect code changes (ideally
>>using Linux's brand new inotify mechanism, or something similar when
>>inotify is not available), display a "please wait" message to the user,
>>and restart the process.
> 
> 
> If you're changing more than one file, how could that know when the source
> code is in a globally consistent state again?  (BTW, there are efficient
> ways to get notifications about modified files even on Win95, so that part
> isn't a hangup.)

That problem turns out to be easy for Zope.  The trick is to check for
source code changes at the beginning of web requests, but no more often
than once every two seconds or so.  Developers hack for a while,
producing a set of hopefully consistent changes, then test the new code
just by making a browser request.  Zope's auto-refresh feature works
this way, and I think that part turned out quite successful.

(I just realized there's an important use case: if the developer
introduces an error that prevents the process from restarting, Zope
needs to somehow bring the error to the developer's attention.  The
developer won't necessarily be staring at a terminal.)

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


RE: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Tim Peters
[Jim Fulton]
>> It [reload] also doesn't handle global data properly.
>>
>> It tries to do something that Python modules were never designed to
>> support, which is to load them more than once.

[Shane Hathaway]
> However, given the existence of the reload() builtin, someone apparently
> believed Python modules *were* designed to support reloading.

It was certainly hopeful -- other hopeful failures include the whole
rexec/bastion maze, and the "multiple interpreters" gimmick in the C API
that some people still spend their lives trying to out-guess <0.9 wink>.

One thing they have in common is that nobody volunteered ongoing work on
them, apart from Guido, and they would have required major work to become
praiseworthy.

> Because reload() is a builtin, Python seems to promise that reload() is
> reliable, but in fact it's rarely reliable.

There are more words about reload() in the manual than about any other
builtin, and they point out problem after problem.  It's very hard to remove
a builtin (has it ever been done?  I don't recall a case -- although apply()
is silently "deprecated"), and reload() was there from the start.  I thought
it was a bad sign when Python NEWS for version 1.0.2 remarked that there
were now "fewer chances of dumping core when trying to reload ..." .

> There would be a lot less confusion if reload() were moved somewhere like
> the "imp" module.

It's slated to disappear from Python 3000:

http://www.python.org/doc/essays/ppt/regrets/PythonRegrets.pdf

I doubt it will be removed from __builtin__ before then.

>> When it fails, it does so in subtle ways that cause people to lose lots
>> of time.

> I agree.  The right way to refresh is to detect code changes (ideally
> using Linux's brand new inotify mechanism, or something similar when
> inotify is not available), display a "please wait" message to the user,
> and restart the process.

If you're changing more than one file, how could that know when the source
code is in a globally consistent state again?  (BTW, there are efficient
ways to get notifications about modified files even on Win95, so that part
isn't a hangup.)

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Shane Hathaway
Jim Fulton wrote:
> It also doesn't handle global data properly.
> 
> It tries to do something that Python modules were never
> designed to support, which is to load them more than once.

However, given the existence of the reload() builtin, someone apparently
believed Python modules *were* designed to support reloading.  Because
reload() is a builtin, Python seems to promise that reload() is
reliable, but in fact it's rarely reliable.  There would be a lot less
confusion if reload() were moved somewhere like the "imp" module.

> When it fails, it does so in subtle ways that cause people
> to lose lots of time.

I agree.  The right way to refresh is to detect code changes (ideally
using Linux's brand new inotify mechanism, or something similar when
inotify is not available), display a "please wait" message to the user,
and restart the process.

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Chris McDonough
Refresh was never much of a timesaver for me.  Pressing "ctrl-C - up
arrow - return" on the console from which I run Zope is my answer.  I
know lots of people say that this is too slow, but on my most recent
project from the time I type "runzope" to the time I see "ready to
handle requests" is

real0m3.029s
user0m1.857s
sys 0m0.306s

... and I can live with that.

- C

On Thu, 2005-07-14 at 23:02 +0200, Dieter Maurer wrote:
> Lennart Regebro wrote at 2005-7-12 20:22 +0200:
> > ...
> >"well, if it happens to work for you, then
> >good luck, but it is actually a piece of unsupported crap that hasn't
> >worked for years and that nobody is bothered to fix bugs in".
> 
> I have the feeling that we have an easy tendency to say such things:
> 
>   Let a bug occur in some component and then, instead (or in addition)
>   of fixing the bug, we say often: rip the component off Zope.
> 
>   That was threadened for "Version"s, and "ZClass"es and now
>   for "Refresh".
> 
> For me, "Refresh" always was a big development time saver -- I knew
> that I had to refresh with a product also all dependent products.
> Recently, I automated this refreshing of dependent products.
> This made "Refresh" behave as expected (provided the product dependancies
> are described correctly) ;-)
> 

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Sidnei da Silva
On Thu, Jul 14, 2005 at 06:01:26PM -0400, Jim Fulton wrote:
| When it fails, it does so in subtle ways that cause people
| to lose lots of time.

Usually when it fails I just restart. I wouldn't consider that losing
lots of time comparing restarting now and then to restarting
constantly.

-- 
Sidnei da Silva
Enfold Systems, LLC.
http://enfoldsystems.com
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Jim Fulton

Dieter Maurer wrote:

Jens Vagelpohl wrote at 2005-7-12 18:46 +0100:


...
Due to the nastiness Refresh has to perform under the cover even  
Shane has to admit that it simply is not some "make development  
faster" panacea.



Shane might (perhaps) admit it but I do not :-)

There is not much "nastiness" that "Refresh" does:

  (Manual) refresh has only a single problem:
  it does not refresh dependent products automatically.


It also doesn't handle global data properly.

It tries to do something that Python modules were never
designed to support, which is to load them more than once.

When it fails, it does so in subtle ways that cause people
to lose lots of time.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Lennart Regebro wrote at 2005-7-12 20:22 +0200:
> ...
>"well, if it happens to work for you, then
>good luck, but it is actually a piece of unsupported crap that hasn't
>worked for years and that nobody is bothered to fix bugs in".

I have the feeling that we have an easy tendency to say such things:

  Let a bug occur in some component and then, instead (or in addition)
  of fixing the bug, we say often: rip the component off Zope.

  That was threadened for "Version"s, and "ZClass"es and now
  for "Refresh".

For me, "Refresh" always was a big development time saver -- I knew
that I had to refresh with a product also all dependent products.
Recently, I automated this refreshing of dependent products.
This made "Refresh" behave as expected (provided the product dependancies
are described correctly) ;-)

-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Lennart Regebro wrote at 2005-7-12 19:17 +0200:
> ...
>The refresh functionality has been unstable since it became a part of
>Zope core, and I have ended up not using it since it since then

I never used "auto refresh" but always use the manual refresh
and I am very happy with it.

Recently, I implemented a "RefreshTool" that refreshes with
a product also all the products that depend on it.
With this, almost all refresh related problems disappear (exception:
the persistency bugs of "auto refresh mode").

It greatly speeds up development and I would not want to miss it :-)

> ...
>I'm all for makeing a serious effort one day to fix this.

+1

-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Victor Safronovich wrote at 2005-7-12 18:17 +0600:
>My  product has 'Auto refresh mode' option checked.
>
>I  changed  some  source  code  of  my product, after that autorefresh have 
>been
>occured.
>And after autorefresh I got strange several errors:
> ...
>self._store_objects(ObjectWriter(obj), transaction)
>  File "C:\Program Files\Zope-2.8.0-final\lib\python\ZODB\Connection.py", line 
> 534, in _store_objects
>self._cache[oid] = obj
>ValueError: Cache values may only be in one cache.

I have seen this problem report for Zope 2.7 as well (in the mailing list).

Obviously, the "auto refresh mode" implementation contains a
persistency bug -- and not only since Zope 2.8...

-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Jens Vagelpohl wrote at 2005-7-12 18:46 +0100:
>...
>Due to the nastiness Refresh has to perform under the cover even  
>Shane has to admit that it simply is not some "make development  
>faster" panacea.

Shane might (perhaps) admit it but I do not :-)

There is not much "nastiness" that "Refresh" does:

  (Manual) refresh has only a single problem:
  it does not refresh dependent products automatically.

  But this can easily be fixed (provided product dependencies
  are configured correctly).

-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Jens Vagelpohl wrote at 2005-7-12 21:49 +0100:

[Lennart]
>> If it is the majoritys agreement that this answer is the best one, and
>> that the refresh functionality should be looked at as something that
>> may or may not work purely on luck, then I'll mark it as deprecated
>> and up for removal in 2.10.

[Jens]
>+1 from me

[Dieter]
-1 from me.

-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-12 Thread Lennart Regebro
On 7/12/05, Jens Vagelpohl <[EMAIL PROTECTED]> wrote:
> > If it is the majoritys agreement that this answer is the best one, and
> > that the refresh functionality should be looked at as something that
> > may or may not work purely on luck, then I'll mark it as deprecated
> > and up for removal in 2.10.
> 
> +1 from me
> 
> > Maybe we can start looking at the last non-integrated version of the
> > software, because it worked much better, and again make it a
> > standalone product?
> 
> Knock yourself out :)

Marking stuff as deprecated is well within my competence, but actually
making a working standalone product is probably not. :-) Ah well..

-- 
Lennart Regebro, Nuxeo http://www.nuxeo.com/
CPS Content Management http://www.cps-project.org/
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-12 Thread Jens Vagelpohl

Imagine if somebody tried to use ZPTs, and the answer to any question
where you get errors from it is "well, then don't use it?" It's just
not a viable answer.


Refresh is not a core functionality like ZPT. That comparison is a  
bit outlandish.




If it is the majoritys agreement that this answer is the best one, and
that the refresh functionality should be looked at as something that
may or may not work purely on luck, then I'll mark it as deprecated
and up for removal in 2.10.


+1 from me



Maybe we can start looking at the last non-integrated version of the
software, because it worked much better, and again make it a
standalone product?


Knock yourself out :)

jens

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


RE: [ZODB-Dev] Reloading product in 2.8

2005-07-12 Thread Tim Peters
[Victor Safronovich, having problems with "auto refresh"]

Victor (et alia), I think a discussion of auto-refresh would be more useful
on a Zope list -- it's not really (AFAIK) a ZODB issue.

This is what changed in ZODB, from ZODB 3.3c1's NEWS file (and no, it won't
be changed back to the way it was):

"""
- Transactions have new, backward-incompatible behavior in one respect:
  if a ``Transaction.commit()``, ``Transaction.commit(False)``, or
  ``Transaction.commit(True)`` raised an exception, prior behavior was that
  the transaction effectively aborted, and a new transaction began.
  A primary bad consequence was that, if in a sequence of subtransaction
  commits, one of the commits failed but the exception was suppressed,
  all changes up to and including the failing commit were lost, but
  later subtransaction commits in the sequence got no indication that
  something had gone wrong, nor did the final (top level) commit.  This
  could easily lead to inconsistent data being committed, from the
  application's point of view.

  The new behavior is that a failing commit "sticks" until explicitly
  cleared.  Now if an exception is raised by a ``commit()`` call (whether
  subtransaction or top level) on a Transaction object ``T``:

- Pending changes are aborted, exactly as they were for a failing
  commit before.

- But ``T`` remains the current transaction object (if ``tm`` is ``T``'s
  transaction manger, ``tm.get()`` continues to return ``T``).

- All subsequent attempts to do ``T.commit()``, ``T.join()``, or
  ``T.register()`` raise the new ``TransactionFailedError`` exception.
  Note that if you try to modify a persistent object, that object's
  resource manager (usually a ``Connection`` object) will attempt to
  ``join()`` the failed transaction, and ``TransactionFailedError``
  will be raised right away.

  So after a transaction or subtransaction commit fails, that must be
  explicitly cleared now, either by invoking ``abort()`` on the transaction
  object, or by invoking ``begin()`` on its transaction manager.
"""

That's mostly what Victor reported, instances of:

TransactionFailedError: An operation previously failed, with traceback:

followed by the traceback of the _original_ exception that caused ZODB to
mark the transaction as being fatally hosed.  The original exception may
have happened long in the past (but after the current transaction began).

Each instance of such a thing points to one or two application bugs:

1. The original exception reported.

and

2. That the original exception was suppressed to begin with (e.g, perhaps
   by a bare "except:" clause in the app, catching more than it really
   intended to catch), and without aborting the current transaction at
   the time in response.

The originally suppressed exceptions in Victor's report:

> ValueError: Cache values may only be in one cache.

and

> InvalidObjectReference: Attempt to store an object from a foreign database
connection

suggest that app code is sharing persistent objects across threads, e.g.
loading an object O from one connection in thread A, and then in thread B
trying to store a mutated O via B's connection.  ZODB doesn't support that
(it's fine if A and B have their own _copies_ of O, what they can't do is
share an in-memory persistent object).

The other exception reported is a sign that a module has been loaded into
sys.modules more than once, under different dotted path names; while these
can be nightmarish to untangle, they're not ZODB issues either:

> PicklingError: Can't pickle  'Products.CMFNauTools.Explorer.FolderExplorer'>: it's not the same object
> as Products.CMFNauTools.Explorer.FolderExplorer

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-12 Thread Lennart Regebro
On 7/12/05, Jens Vagelpohl <[EMAIL PROTECTED]> wrote:
> 
> On 12 Jul 2005, at 18:17, Lennart Regebro wrote:
> >>> previous Zope version, which I used, was 2.6.1, but it was very old
> >>> version and
> >>> I  decided to jump to 2.8.0. I took problems with autorefresh. What
> >>> can I do to solve
> >>> my problems?
> >>>
> >>
> >> If it creates problems then simply don't use it.
> >>
> >
> > That's not a very good answer.
> 
> I disagree.
> 
> Due to the nastiness Refresh has to perform under the cover even
> Shane has to admit that it simply is not some "make development
> faster" panacea. The only viable answer is "stop using if it gives
> you trouble" because it is probably impossible to make it work in
> 100% of all cases.

No, that's still not a viable answer, because it should be possible to
make some kind of list of cases where it is expected not to work, so
that you can try to see if this is one of the cases. The above answer
however basically means "well, if it happens to work for you, then
good luck, but it is actually a piece of unsupported crap that hasn't
worked for years and that nobody is bothered to fix bugs in".

Imagine if somebody tried to use ZPTs, and the answer to any question
where you get errors from it is "well, then don't use it?" It's just
not a viable answer.


If it is the majoritys agreement that this answer is the best one, and
that the refresh functionality should be looked at as something that
may or may not work purely on luck, then I'll mark it as deprecated
and up for removal in 2.10.

Maybe we can start looking at the last non-integrated version of the
software, because it worked much better, and again make it a
standalone product?
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-12 Thread Gary Poster


On Jul 12, 2005, at 1:46 PM, Jens Vagelpohl wrote:


On 12 Jul 2005, at 18:17, Lennart Regebro wrote:


previous Zope version, which I used, was 2.6.1, but it was very old
version and
I  decided to jump to 2.8.0. I took problems with autorefresh. What
can I do to solve
my problems?


If it creates problems then simply don't use it.


That's not a very good answer.


I disagree.


I agree with Jens that it is unlikely to ever work correctly 100% of  
the time.  I agree with Lennart that leaving it not working is not a  
good answer.  I think it should be removed as a feature.  Making  
every developer using ZODB eventually learn not to trust the feature  
seems unnecessarily painful.


Gary
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-12 Thread Jens Vagelpohl


On 12 Jul 2005, at 18:17, Lennart Regebro wrote:

previous Zope version, which I used, was 2.6.1, but it was very old
version and
I  decided to jump to 2.8.0. I took problems with autorefresh. What
can I do to solve
my problems?



If it creates problems then simply don't use it.



That's not a very good answer.


I disagree.

Due to the nastiness Refresh has to perform under the cover even  
Shane has to admit that it simply is not some "make development  
faster" panacea. The only viable answer is "stop using if it gives  
you trouble" because it is probably impossible to make it work in  
100% of all cases.


jens

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-12 Thread Lennart Regebro
On 7/12/05, Jens Vagelpohl <[EMAIL PROTECTED]> wrote:
> 
> On 12 Jul 2005, at 13:17, Victor Safronovich wrote:
> > previous Zope version, which I used, was 2.6.1, but it was very old
> > version and
> > I  decided to jump to 2.8.0. I took problems with autorefresh. What
> > can I do to solve
> > my problems?
> 
> If it creates problems then simply don't use it.

That's not a very good answer. 

The refresh functionality has been unstable since it became a part of
Zope core, and I have ended up not using it since it since then (So
that's what? Since Zope 2.3 or something). I've tried from time to
time, but I always end up going back to just restarting Zope every
time I make a code change, because refreshing and trying, and often
(but not always) getting incomprehensible errors and then restarting
Zope, is just much slower that simply restarting Zope every time.

That however, is a serious slowdown in the Zope development process,
and I would be very happy if this could be made to work stably. But
instead of getting better, it seems to deteriorate...

I'm all for makeing a serious effort one day to fix this. But I have
tried before, and I am way out of depth. Any suggestions of how to
solve this problem? Or should be just give up, and mark it as
deprecated in 2.8.1, so that people stop thinking it's actually
useful?
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Reloading product in 2.8

2005-07-12 Thread Jens Vagelpohl


On 12 Jul 2005, at 13:17, Victor Safronovich wrote:
previous Zope version, which I used, was 2.6.1, but it was very old  
version and
I  decided to jump to 2.8.0. I took problems with autorefresh. What  
can I do to solve

my problems?


If it creates problems then simply don't use it.

jens


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev