Re: [Launchpad-dev] Error in layer teardown: AttributeError: 'NoneType' object has no attribute 'read'
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Aaron Bentley wrote: I'm getting this error when layers are torn down. Anyone know why? I am still getting this. It's rather upsetting for me, because I like it when the test suite runs without error. It could be related to the fact that the machine I'm using for testing has only 2G of memory, and testing tends to gobble it, but I don't think it's exhausing swap. Aaron For example: Running canonical.testing.layers.AppServerLayer tests: Running in a subprocess. Set up canonical.testing.layers.BaseLayer in 0.004 seconds. Set up canonical.testing.layers.DatabaseLayer in 0.565 seconds. Set up canonical.testing.layers.LibrarianLayer in 9.472 seconds. Set up canonical.testing.layers.MemcachedLayer in 0.220 seconds. Set up canonical.testing.layers.LaunchpadLayer in 0.000 seconds. Set up canonical.testing.layers.FunctionalLayer in 7.586 seconds. Set up canonical.testing.layers.GoogleServiceLayer in 2.655 seconds. Set up canonical.testing.layers.LaunchpadFunctionalLayer in 0.000 seconds. Set up canonical.testing.layers.AppServerLayer in 12.713 seconds. Running: ..No handlers could be found for logger lazr.smtptest .. Ran 80 tests with 0 failures and 0 errors in 1 minutes 53.773 seconds. Tear down canonical.testing.layers.AppServerLayer in 0.508 seconds. Tear down canonical.testing.layers.LaunchpadFunctionalLayer in 0.000 seconds. Tear down canonical.testing.layers.LaunchpadLayer in 0.000 seconds. Tear down canonical.testing.layers.DatabaseLayer in 0.024 seconds. Tear down canonical.testing.layers.LibrarianLayer in 0.511 seconds. Tear down canonical.testing.layers.MemcachedLayer in 5.133 seconds. Tear down canonical.testing.layers.FunctionalLayer ... not supported 80 0 0 Tear down canonical.testing.layers.GoogleServiceLayer in 0.168 seconds. Tear down canonical.testing.layers.BaseLayer in 0.000 seconds. Error in sys.exitfunc: Exception in thread Thread-35: Traceback (most recent call last): File /usr/lib/python2.5/threading.py, line 486, in __bootstrap_inner self.run() File /usr/lib/python2.5/threading.py, line 446, in run self.__target(*self.__args, **self.__kwargs) File /home/abentley/launchpad/stable/eggs/zope.testing-3.8.1-py2.5.egg/zope/testing/testrunner/runner.py, line 439, in spawn_layer_in_subprocess erriter = iter(child.stderr.read().splitlines()) AttributeError: 'NoneType' object has no attribute 'read' ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAktDSzcACgkQ0F+nu1YWqI3H1wCfYpPbIniKcM4PBhCOxecLozSp fWAAn1H/jJ9Y9LR0kP3TPK4iVDglXR85 =YBQK -END PGP SIGNATURE- ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Error in layer teardown: AttributeError: 'NoneType' object has no attribute 'read'
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Aaron Bentley wrote: Aaron Bentley wrote: I'm getting this error when layers are torn down. Anyone know why? I am still getting this. It's rather upsetting for me, because I like it when the test suite runs without error. This appears to be self-inflicted. Please ignore. Aaron -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAktDZxwACgkQ0F+nu1YWqI3HPQCdElCD3e6XaSa22LS43nO9v1/u hggAnAkHrUTBL9J7IRrO05t++QbEAdP5 =09dV -END PGP SIGNATURE- ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Using a signal to switch to read-only mode
(CCing launchpad-dev as others might have ideas/suggestions) On Tue, 2010-01-05 at 08:13 +, Tom Haddon wrote: On Mon, 2010-01-04 at 18:16 -0200, Guilherme Salgado wrote: On Mon, 2009-12-21 at 09:21 +, Tom Haddon wrote: On Fri, 2009-12-18 at 10:37 -0500, Gary Poster wrote: I like the suggestions I've read. Thanks to all three of you. I'll summarize the proposals so far. - We will switch logrotation to use SIGHUP. - We will use SIGUSR2 as a flag for checking for the presence of a read-only.txt at the top of the tree. - At application start, or when SIGUSR2 fires, if read-only.txt is found at the top of the tree, the application will switch to (or stay in) read-only mode. If it is not found, the application will switch to (or stay in) normal read-write mode. - We will provide a key-value page to verify the read-only status of (each) application. Here are my thoughts: - I think the key-value page would be very valuable for LOSA peace of mind, so I like the idea. However, it is only pertinent for a given application instance. Going to this page through the load-balancer would not be valuable.LOSAs, would you immediately use this page if we offered it, going to each instance in the cluster? It'd be nice, but I don't want to block on it. If not, I'd like to push it out of the scope of this effort, until we can think about offering an aggregated view of information like this in a dashboard like the one Maris will hopefully be working on this cycle. - I think we should definitely log mode switches. Then LOSAs can at least trail the logs for a given instance to verify that the app noticed the signal and the presence or absence of the file. +1 - If the LOSAs don't want to rock the boat with changing logrotation to SIGHUP, we do have a swath of signals from SIGRTMIN to SIGRTMAX that we could use. I'm in favor of the SIGHUP switch if the LOSAs don't mind, though. This switch is okay. Today I started working on this, and following is my initial plan: Currently, the way we switch to read-only is by changing the read_only config to True *and* changing the main_master and main_slave configs to point to standalone databases. What we want is to get rid of the read_only config and collapse the extra config files we have for read-only mode (lpnet1-db-update) into the lpnet1 config. In order to do this we will use the presence of a file (read-only.txt) on the root of the tree to identify (upon startup or SIGUSR2) whether or not we're in read-only mode, and set the main_master and main_slave configs appropriately. As we'll be overwriting these config variables, we'll need to store all different values we might use for them in new variables (e.g. rw_main_master, rw_main_slave, ro_main_master and ro_main_slave). (we might even get rid of the main_master and main_slave config variables as they will be computed values, which can be moved somewhere else. although I'm not sure this is a good idea because all other db names live in config variables). The plan: • Change all places that use config.launchpad.read_only to use another helper, which tells whether or not we're in read-only mode by looking for a read-only.txt file. • switch logrotation to use SIGHUP. • Rename main_master and main_slave to rw_main_master and rw_main_slave, adding new (and empty) main_master and main_slave config variables, which get set upon startup/SIGUSR2 (with the values of rw_*). • log read-only/read-write switches However, after I started implementing it I realized that having two switches (the read-only.txt file and the SIGUSR2) to turn on read-only doesn't sound like a very good idea (as we may accidentally leave an app server in an inconsistent state), so we may want to use SIGUSR2 to create a read-only.txt file *and* trigger the code that sets the configs with the appropriate values. You don't need to worry about creating/deleting the read-only.txt file - we'll manage that through external means (initscripts or other helper scripts). I'd envisage you only need one signal which means check again whether we're in read-only or read-write mode. As we discussed on IRC, my concern was that having a read-only.txt file did not mean we were in read-only mode -- the SIGUSR2 is needed, and if forgotten the server would be in an inconsistent state. In that state, the python code thinks we're running in read only (because it relies on read-only.txt for that) but we're still connecting to the rw db
Re: [Launchpad-dev] Using a signal to switch to read-only mode
On Tue, 2010-01-05 at 18:54 -0200, Guilherme Salgado wrote: (CCing launchpad-dev as others might have ideas/suggestions) On Tue, 2010-01-05 at 08:13 +, Tom Haddon wrote: On Mon, 2010-01-04 at 18:16 -0200, Guilherme Salgado wrote: On Mon, 2009-12-21 at 09:21 +, Tom Haddon wrote: On Fri, 2009-12-18 at 10:37 -0500, Gary Poster wrote: I like the suggestions I've read. Thanks to all three of you. I'll summarize the proposals so far. - We will switch logrotation to use SIGHUP. - We will use SIGUSR2 as a flag for checking for the presence of a read-only.txt at the top of the tree. - At application start, or when SIGUSR2 fires, if read-only.txt is found at the top of the tree, the application will switch to (or stay in) read-only mode. If it is not found, the application will switch to (or stay in) normal read-write mode. - We will provide a key-value page to verify the read-only status of (each) application. Here are my thoughts: - I think the key-value page would be very valuable for LOSA peace of mind, so I like the idea. However, it is only pertinent for a given application instance. Going to this page through the load-balancer would not be valuable.LOSAs, would you immediately use this page if we offered it, going to each instance in the cluster? It'd be nice, but I don't want to block on it. If not, I'd like to push it out of the scope of this effort, until we can think about offering an aggregated view of information like this in a dashboard like the one Maris will hopefully be working on this cycle. - I think we should definitely log mode switches. Then LOSAs can at least trail the logs for a given instance to verify that the app noticed the signal and the presence or absence of the file. +1 - If the LOSAs don't want to rock the boat with changing logrotation to SIGHUP, we do have a swath of signals from SIGRTMIN to SIGRTMAX that we could use. I'm in favor of the SIGHUP switch if the LOSAs don't mind, though. This switch is okay. Today I started working on this, and following is my initial plan: Currently, the way we switch to read-only is by changing the read_only config to True *and* changing the main_master and main_slave configs to point to standalone databases. What we want is to get rid of the read_only config and collapse the extra config files we have for read-only mode (lpnet1-db-update) into the lpnet1 config. In order to do this we will use the presence of a file (read-only.txt) on the root of the tree to identify (upon startup or SIGUSR2) whether or not we're in read-only mode, and set the main_master and main_slave configs appropriately. As we'll be overwriting these config variables, we'll need to store all different values we might use for them in new variables (e.g. rw_main_master, rw_main_slave, ro_main_master and ro_main_slave). (we might even get rid of the main_master and main_slave config variables as they will be computed values, which can be moved somewhere else. although I'm not sure this is a good idea because all other db names live in config variables). The plan: • Change all places that use config.launchpad.read_only to use another helper, which tells whether or not we're in read-only mode by looking for a read-only.txt file. • switch logrotation to use SIGHUP. • Rename main_master and main_slave to rw_main_master and rw_main_slave, adding new (and empty) main_master and main_slave config variables, which get set upon startup/SIGUSR2 (with the values of rw_*). • log read-only/read-write switches However, after I started implementing it I realized that having two switches (the read-only.txt file and the SIGUSR2) to turn on read-only doesn't sound like a very good idea (as we may accidentally leave an app server in an inconsistent state), so we may want to use SIGUSR2 to create a read-only.txt file *and* trigger the code that sets the configs with the appropriate values. You don't need to worry about creating/deleting the read-only.txt file - we'll manage that through external means (initscripts or other helper scripts). I'd envisage you only need one signal which means check again whether we're in read-only or read-write mode. As we discussed on IRC, my concern was that having a read-only.txt file did not mean we were in read-only mode -- the SIGUSR2 is needed, and if forgotten the server would be in an inconsistent
Re: [Launchpad-dev] Build engineer report, Dec 2009
On Tue, Jan 5, 2010 at 4:22 AM, Barry Warsaw ba...@canonical.com wrote: On Jan 04, 2010, at 12:13 PM, Francis J. Lacoste wrote: On January 4, 2010, Gary Poster wrote: * I meant to do this myself, but it would be nice to have ctags generated so that vim could jump to the library method definition without instead of me grepping through the eggs. Please see ``bin/tags -v`` (or ``make tags``) and ``bin/tags -e`` (for you emacs users). Actually, this one is currently broken because of ctags. It creates symbol definition everytime the symbol is imported, not only where it is defined. Oh! I've been noticing this and thought it was caused by a bug in Emacs. It makes tags unusable for me. Thank Jono for bzr-tools-grep. My pleasure. Interestingly, I showed the recipe for bzr-tools-grep at UDS-L, and a git fan in the audience sniggered about Bazaar's vaunted usability because, bzr ls -VR --kind=file --null | xargs -0 grep -In %s becomes git grep %s Perhaps someone would like to add the grep command to bzr-core, or a plugin. jml ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Reminder: Launchpad reviewers meetings tomorrow
Hi, It's been a long time since we had a reviewers meeting so this is a reminder we'll be starting back up tomorrow. 1500 UTC for Europe/Americas 2100 UTC for Asia/Pacific Even if you're not a reviewer you're welcome to join us in #launchpad-meeting. Add items to the agenda if you have them: https://dev.launchpad.net/ReviewerMeetingAgenda See you then. --Brad ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build engineer report, Dec 2009
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Jan 06, 2010, at 09:42 AM, Jonathan Lange wrote: Interestingly, I showed the recipe for bzr-tools-grep at UDS-L, and a git fan in the audience sniggered about Bazaar's vaunted usability because, bzr ls -VR --kind=file --null | xargs -0 grep -In %s becomes git grep %s Perhaps someone would like to add the grep command to bzr-core, or a plugin. +1 - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iQIcBAEBCAAGBQJLQ8OsAAoJEBJutWOnSwa/k+QP/3ashD8giXYMr/Ulu1PCD+/5 boFFmZ79NgOZBYOQx48psMhXswk2nIjqG9G67aWaTXjmqWo5m5hPIr8AL7AAQu2i 9WqQ7oaiBiHopFftsCoFPdKi9xmBgytJyXgEkQ8JZrZLY+LaIuVh0gsPF1FHFJxm d9AUhzRlzGxTdSBI30GFEZuTY327h3JYDRwQ4y5eoFiNK2FTGuDoHe4QXkhC3peJ qCHNurJvKkkDSeklSpSO3N5ORB7DZOcb00FN80jNoZjGHnMuummaGovD9cusclvf t4RNQa42t7Y8CElaQ3aAZzcyISKvyi1DxO2dxwOz+IeATkkCKvyKNKdTHrWjP+4q 9KzmJVvUeY55ceBLsgn4bd+I3nc/FsEH4Z/PSMaIjIx/BrWoxV71XTlAiJ07Htmb zHfqaC6jFUKAB5f0vj0Xk1RNWdbBVQDpbp+zD7MIuaYg/JsPjRWvFQtbRzHgzbJs fXMW61fJ7VjuhKNQRGm+YJOylANFszCvQVMjkf2qI9BWbhKojTiAm11WrirDAWHI 2Me31c7gBY2l4S6AIQuVWe0NyJRy/C7hBR4ArKL4aiVqImJf3ddpoqWODWqXNNWD t8+O5wNVN9Ixzqln2hlURUu1BvyEByCYX1nammwhm4eHCsz68NFx87t5E2R/taHG 3Fdw67MlCi/YLmjMCdo6 =usp1 -END PGP SIGNATURE- ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build engineer report, Dec 2009
On Tue, Jan 5, 2010 at 3:54 AM, Gary Poster gary.pos...@canonical.com wrote: Thank you for the survey, Jono. My pleasure. ... * Making a new branch is new excrutiatingly slow. It took me 10 minutes to do one yesterday as I was short of real RAM and rf-branch seems RAM hungry. It's been like this ever since buildout and it drives me *bonkers*. Huh. My perceptions have been that making a branch is now much faster than when I started, which I have attributed to bzr improvements and shared eggs. People who feel the way this quote describes should maybe talk to me. My first guess is that you are working in some way that does not let you share eggs across branches. *If* this is the case, then my observation would be that rocketfuel-* is the blessed and supported way for building branches, and if you are doing something else, I certainly understand and sometimes do the same, but we need to either acknowledge that you are on an expert path and are responsible for your own safety, or that we need to have explicit support for a non rocketfuel-* approach. Either way, IMO the idea should be that there is a path (or two) that is supported, documented, and improved; if you stray off the path for your own reasons, that's your business: the maintenance of your path is your own responsibility (until we all decide to add it to the supported list). On the other hand, maybe people with this experience are in fact sharing eggs. In that case, we should see what the problem is. I believe and hope that this experience is unusual. Personally, I use: bzr cbranch stable foo cd foo ./utilities/link-external-sourcecode ~/src/launchpad/devel make build That is, I use sanctioned scripts and routines, and I share eggs, but I don't use the rocketfuel-* scripts. My reasons are: - as someone working on Launchpad / Bazaar integration, I wanted to be using Bazaar as much as possible - I use lightweight checkouts, and I don't know that rocketfuel-branch supports that - If ever someone makes a Bazaar GUI or Emacs mode that I prefer to the command line, I want to use that. * It seems silly to constantly consult the zcml for how urls, view classes, and templates are mapped. It would be nice if view names and template names were mapped by convention instead of by configuration, but that wouldn't help with urls. Maybe we just need a script to discover the mappings more quickly. FWIW, this is the kind of thing that Grok technology can provide. The discussion of the pros and cons of this particular idea within our own code base has not yet been run to ground, to my knowledge. I am open to this perspective. Where's the discussion taking place? * ./bin/lint has a lot of false positives mainly because it can't resolve a bunch of lazr modules. I agree that this is annoying. I thought I knew how to fix it but someone else had tried what I had in mind and it didn't work. I don't know what to do about this ATM, other than more digging. I don't see a bug about this on https://bugs.edge.launchpad.net/launchpad-foundations/+bugs?field.tag=build-infrastructure Personally, I think we should ditch pylint, or at least restrict it to checking formatting issues only. Pyflakes is much faster, catches almost everything relevant, and produces far fewer false positives. jml ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build engineer report, Dec 2009
On Wed, 6 Jan 2010 09:42:04 +1100, Jonathan Lange j...@canonical.com wrote: Interestingly, I showed the recipe for bzr-tools-grep at UDS-L, and a git fan in the audience sniggered about Bazaar's vaunted usability because, bzr ls -VR --kind=file --null | xargs -0 grep -In %s becomes git grep %s Perhaps someone would like to add the grep command to bzr-core, or a plugin. You mean like lp:~vila/bzr/grep (confusingly a plugin under the bzr project?) I agree that it would be great to have it in core. Thanks, James ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Using a signal to switch to read-only mode
Perhaps a similar method could be used in future for site-wide notifications. https://dev.launchpad.net/NotificationSystem. For example if there is a site-wide-notification.txt, it can contain configuration for a message to be displayed in a banner at the top of some or all html pages. I don't want to bloat the scope of the readonly flag, but perhaps it's useful to at least consider whether the same mechanism would work for both, or it may shed light on the implementation of the readonly switch. For example it suggests to me that just using SIGUSR2 other than meaning check now doesn't allow passing enough data. Perhaps you can check for the file from some high-level loop, but no more than every say ten seconds? -- Martin http://launchpad.net/~mbp/ ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Using a signal to switch to read-only mode
On Wed, Jan 6, 2010 at 10:30 AM, Martin Pool m...@canonical.com wrote: Perhaps a similar method could be used in future for site-wide notifications. https://dev.launchpad.net/NotificationSystem. For example if there is a site-wide-notification.txt, it can contain configuration for a message to be displayed in a banner at the top of some or all html pages. I'd rather pull this information from the database or even ajax load it from a static URL. Coordinating the .txt files over all the different appserver trees is a pain we should avoid when possible. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Build engineer report, Dec 2009
2010/1/6 James Westby jw+deb...@jameswestby.net: On Wed, 6 Jan 2010 09:42:04 +1100, Jonathan Lange j...@canonical.com wrote: Interestingly, I showed the recipe for bzr-tools-grep at UDS-L, and a git fan in the audience sniggered about Bazaar's vaunted usability because, bzr ls -VR --kind=file --null | xargs -0 grep -In %s becomes git grep %s Perhaps someone would like to add the grep command to bzr-core, or a plugin. You mean like lp:~vila/bzr/grep (confusingly a plugin under the bzr project?) I agree that it would be great to have it in core. https://bugs.edge.launchpad.net/bzr/+bug/503670 -- anyone want to try it? -- Martin http://launchpad.net/~mbp/ ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Broken image links on planet
Karl Fogel karl.fo...@canonical.com writes: Jonathan Lange j...@canonical.com writes: The per-blog images on http://planet.launchpad.net/ are all broken, e.g. http://planet.launchpad.net/feedjack/planetlaunchpad/img/faces/lp-blog.png I'm not sure how best to report this. How can I help fix this? This is mine -- mrevell let me know while I was on vacation, and I told him I'd look at it when I came back. I'm back. Consider it reported and assigned :-). I should have added: I'm off at a two-day Lean Training thang in Boston right now, hence the delay. I haven't forgotten about this! ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
