Re: [Launchpad-dev] Help please - launchpadlib collections
Hi Ian,
Firstly I wanted to note that lp.projects is a heterogenous collection
isn't it? You can access both projects and project groups with it, and
don't they have a different interface?
Secondly I'm wondering why this is trying to be a collection at all. You
wouldn't even want to iterate them, and if they are heterogenous then
putting them in a collection seems odd. Would it be better for them to
be accessed as top level objects (lp.myservice)?
Thanks,
James
On Wed, 11 Apr 2012 10:27:39 +1000, Ian Booth ian.bo...@canonical.com wrote:
Hi
I'd like some input with an issue concerning collections in launchpadlib.
The background is that I want to have launchpadlib recognise a new top
level collection called 'services'. Services are looked up by name. The
collection is heterogeneous in that each named service provides
different capabilities via different exported methods. All services
extend an IService interface and this is how the collection is currently
defined:
class IServiceFactory(Interface):
export_as_webservice_collection(IService)
@operation_parameters(name=TextLine(required=True))
@operation_returns_entry(IService)
@export_read_operation()
@operation_for_version(beta)
def getByName(name):
Lookup a service by name.
I want to allow launchpadlib to be used like so:
lp = Launchpad()
service = lp.services['myservice']
service.foo()
(As an aside, the current syntax required to access a named service is:
# XXX 2012-02-23 wallyworld bug 681767
# Launchpadlib can't do relative url's
service = launchpad.load(
'%s/+services/sharing' % self.launchpad._root_uri)
Even if bug 681767 were fixed, the client would still need to know the
URL to traverse whereas having a named 'services' collection hides this
implementation detail. Perhaps this is ok though?)
Because the services collection is heterogeneous, there needs to be some
way which allows launchpadlib to know what each service instance in the
collection provides. My initial solution was to define a new ServiceSet
in launchpadlib (a solution already used for projects, bugs,
distributions etc):
class ServiceSet(CollectionWithKeyBasedLookup):
A custom subclass capable of service lookup by service name.
def _get_url_from_id(self, key):
Transform a service name into the URL to a service.
return str(
self._root._root_uri.ensureSlash()) +
'+services/' + str(key)
# All services are different so we need to ensure each service
# representation is retrieved when needed.
collection_of = None
This works well and correctly deals with the fact that each named
service instance has different capabilities. However, concerns were
raised that it adds to LOC count and could be done generically without
the helper class.
My understanding is that an alternative solution would require changes
and/or enhancements to the WADL generation rules, and for additional
information to be added to the WADL. At the moment, lazr.restful (which
is used by launchpadlib) doesn't really know what constitutes a top
level collection and uses heuristics to determine this.
eg from ServiceRootResource
# XXX sinzui 2008-09-29 bug=276079:
# Top-level collections need a marker interface
# so that so top-level utilities are explicit.
if (provided.isOrExtends(ICollection)
and ICollection.implementedBy(registration.factory)):
This issue would need to be addressed along with the ability to define
more specifically what's in each collection.
I would love to get input from the smart folks on this list as to what
the best way forward is. I think Gary, Francis, (and Leonard) have
worked on this specific stuff in the past. Do we want to attempt to
tackle the higher level issues highlighted above? Apart from the bug
concerning the definition of top level collections (276079), I couldn't
find a bug specifically related to heterogeneous collections. The LOC
count for an alternative solution would be (far?) greater than what my
solution has, but may allow launchpadlib to in the future handle new
heterogeneous collections without additional code changes. I am wary of
diverting too much time away from the disclosure project for a nice to
have but non-core disclosure item. I see the options as:
1. Adopt my current solution which defines a ServiceSet in launchpadlib
and allows the syntax lp.services['myservice']
2. Fix bug 681767 (not sure of the effort required) and use syntax
lp.load('/+services/myservice')
3. Address the WADL/lazr.restful issues to support syntax
lp.services['myservice'] and make launchpadlib 'futureproof'
Thoughts?
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help :
[Launchpad-dev] [Bug 188564] [NEW] Build also packages for Debian in PPA's
You have been subscribed to a public bug by Rafael Belmonte (eaglescreen): This could be nice way how to repay Debian for all packages we've just synced from them - build PPA packages also for Debian branches that meet package's dependencies and provide repositories with these packages. ** Affects: launchpad Importance: Low Status: Triaged ** Tags: feature lp-soyuz ppa soyuz-core -- Build also packages for Debian in PPA's https://bugs.launchpad.net/bugs/188564 You received this bug notification because you are a member of Launchpad Development mailing list, which is subscribed to the bug report. ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] Parallel testings checkpoint notes - 2012-04-04
= Overview = We've made some notable accomplishments since our last report, but it doesn't feel as much like progress. We had to do a fair amount of work just to keep the testing framework working, thanks to challenges with lxc, our zope.testing fork, and ec2 precise images, among others. We have fixed all of the test failures that we had identified by the time of the last meeting, but, thanks perhaps in part to the increased rigor of adding --shuffle to our tests to ferret out more isolation bugs, we have more to fix. We have made our buildbot process a bit better, by introducing better reporting, better testr support, and better handling of exceptional error cases. We are still waiting on IS for the testing machine, and will have much more to do with them before the project is done. We expect, therefore, to have time to work on additional, related projects, possibly including work such as lpsetup and a parallel version of ec2 test. = Progress towards biweekly action items = * [yellow] Bug fixes for test failures discovered in parallel test runs. * We addressed all items identified in the last meeting and several more. We have new test failures to fix. * [yellow] [carry over from past four weeks] Deliver scaling assessment based on experimental results, using ec2 * Still blocked by IS * [yellow] Get data center box running tests, and have a single comparison run with ec2. * Still blocked by IS * /dev/random exhaustion solution approved and installed * IS requested that we first determine whether it is a problem on the data center machines, or if it is specific to the ec2 environment. If we determine that it is a problem, we will address then . = Other accomplishments = * [carry over from two weeks] blocked on Clint Byrum releasing python-shelltoolbox and Python charm helpers * some lpsetup progress = Progress on tracked items = * No progress on testr bugs * 914166 * 949950 * 957145 * 961103 * 974622 [new] [high priority for us] * LXC * 968371 [new] [resolved] * 959352 [kernel] [no progress] = Goals for next meeting = * Bug fixes for test failures discovered in parallel test runs. * 974584 (celery tests; reliable, at least on 3 runs on 8-core ec2) * 974585 (another UncleanReactorError failure) * 974586 (bug import tests; intermittent) * 974608 (bug test; intermittent) * 974617 (test_operational_view_integration; intermittent) * 974632 (memcache disconnection; also lpbuildbot; intermittent) * Provide fixes for some of the testr bugs (esp. 974622) * Deliver scaling assessment based on experimental results, using ec2 (carry over from previous four weeks) * (carry over from previous two weeks) Get data center box running tests, and have a single comparison run with ec2. -- Francis J. Lacoste francis.laco...@canonical.com signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
[Launchpad-dev] cheap trunk LoC status check
Just doing bzr diff -r date:2012-03-12..-1 | diffstat will get you one months worth of LoC changes to trunk (as I write this it is ... 586 files changed, 13951 insertions(+), 5928 deletions(-) (but you'd need to step over several months to see if thats a trend, or just a big branch right after a big cleanup...) -Rob ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Help please - launchpadlib collections
Hi James Firstly I wanted to note that lp.projects is a heterogenous collection isn't it? You can access both projects and project groups with it, and don't they have a different interface? AFAIK, project groups are handled separately, using lp.project_groups Secondly I'm wondering why this is trying to be a collection at all. You wouldn't even want to iterate them, and if they are heterogenous then putting them in a collection seems odd. Would it be better for them to be accessed as top level objects (lp.myservice)? You and Francis both asked whether exposing the services as a collection is the right thing to do. The reason I like that approach is that it provides a level of discoverability and keeps the launchpadlib interface manageable for want of a better term. If we access each individual service as a separate top level object, we have lp.service1 lp.service2 lp.service3 ... etc Compare the above with lp.services['service1'] lp.services['service2'] ... etc So when we add a new service, the launchpadlib interface doesn't change (acquire a new top level object), and I view this stability as a good thing. Plus, it is possible to iterate over the named services and see what's there, what service interfaces are implemented, what methods are available etc. ie a classic discoverability pattern. Even though the services are heterogeneous, they all extend IService which provides common behaviour such as service lifecycle management hooks etc - well not at the moment, but the idea is there. So it can be argued that one would need to be able to iterate over all services in the collection and start/stop them etc, or ask for stats or whatever. So that's my take on the question of putting the services in a collection. ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Help please - launchpadlib collections
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi Francis
One thing that I'm wondering though is if your ServiceSet is a real
collection. Can you iterate the services? Is there a use case for that?
In other word, is there any advantage to providing the collection API to it?
In a way, you might be better to export this as a ITopLevelEntry, and
still provide custom class that does the key-mapping in launchpadlib.
See my answer (sent to the list) to James who asked a similar question.
snip
WADL issues
snip
1. Adopt my current solution which defines a ServiceSet in launchpadlib
and allows the syntax lp.services['myservice']
I think that might be the best way forward - modulo my concerns with the
fact that you are not really exporting a collection.
Hopefully my explanation in the other email provides the reasons why I
chose to export the services as a collection.
2. Fix bug 681767 (not sure of the effort required) and use syntax
lp.load('/+services/myservice')
That one looks shallow. It's worth fixing, but the API wouldn't be as
nice as the key lookup.
Agreed. I was going to look at fixing that bug but really also prefer
the key lookup approach.
3. Address the WADL/lazr.restful issues to support syntax
lp.services['myservice'] and make launchpadlib 'futureproof'
Adding support for representing URLs in the WADL would be nice, but
would require some effort (and several branches across launchpad,
lazr.restful, lazr.restfulclient and launchpadlib). It would be a nice
addition, but probably a distraction at this point.
That was my thinking which is why I chose the easy path of
implementing option 1 even though it adds (a little) to LOC count.
Thanks for the input on this topic.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iJwEAQECAAYFAk+GL0AACgkQCJ79BCOJFca9DgQAgdKhnZje77HRXGEHys31bDP5
SIsH5q0XHFgVbFPez7xw0H0fdT7j6AltxtTM8q5+kRw9hLkRjysw7c9gjD8GrYp0
ve6HSebeS3rRE+4r1iZKm09jZT6HE+s9AcKnviSf0CSAv9hEDfMW9M8gpZUPIvqu
tIPB7KizDmx99Ow9xyg=
=XJra
-END PGP SIGNATURE-
___
Mailing list: https://launchpad.net/~launchpad-dev
Post to : launchpad-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Help please - launchpadlib collections
On 12-04-11 09:26 PM, Ian Booth wrote: 1. Adopt my current solution which defines a ServiceSet in launchpadlib and allows the syntax lp.services['myservice'] I think that might be the best way forward - modulo my concerns with the fact that you are not really exporting a collection. Hopefully my explanation in the other email provides the reasons why I chose to export the services as a collection. Yes, it does. If you think that iterating over the services is part of the ServiceSet API, then it makes sense to export as a collection. -- Francis J. Lacoste francis.laco...@canonical.com signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Help please - launchpadlib collections
On 12/04/12 11:17, Ian Booth wrote: Hi James Firstly I wanted to note that lp.projects is a heterogenous collection isn't it? You can access both projects and project groups with it, and don't they have a different interface? AFAIK, project groups are handled separately, using lp.project_groups lp.projects['fooprojectgroup'] happens to work because launchpadlib's ProjectSet generates a URL of /fooprojectgroup, and that URL namespace is shared between projects, distributions and project groups. The returned interfaces are heterogeneous because of exactly the hack that Ian proposes as the easy solution -- indexing a client-defined collection like ProjectSet just generates a URL with a known structure and requests that resource, respecting whatever resource type is returned. signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Help please - launchpadlib collections
On Thu, Apr 12, 2012 at 2:11 PM, William Grant william.gr...@canonical.com wrote: On 12/04/12 11:17, Ian Booth wrote: Hi James Firstly I wanted to note that lp.projects is a heterogenous collection isn't it? You can access both projects and project groups with it, and don't they have a different interface? AFAIK, project groups are handled separately, using lp.project_groups lp.projects['fooprojectgroup'] happens to work because launchpadlib's ProjectSet generates a URL of /fooprojectgroup, and that URL namespace is shared between projects, distributions and project groups. The returned interfaces are heterogeneous because of exactly the hack that Ian proposes as the easy solution -- indexing a client-defined collection like ProjectSet just generates a URL with a known structure and requests that resource, respecting whatever resource type is returned. Right, and thats why I'm saying the previously sketched thing of respecting such information consistently will reduce the special casing code, close of a couple of long standing bugs, be easier to maintain and work with. So I rejected your MP and pointed you at the root cause. It still lets you expose a collection of services but you don't need the special casing. I don't think it is a distraction in any way. -Rob ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Help please - launchpadlib collections
On Thu, Apr 12, 2012 at 2:32 PM, Robert Collins robert.coll...@canonical.com wrote: On Thu, Apr 12, 2012 at 2:11 PM, William Grant william.gr...@canonical.com wrote: On 12/04/12 11:17, Ian Booth wrote: Hi James Firstly I wanted to note that lp.projects is a heterogenous collection isn't it? You can access both projects and project groups with it, and don't they have a different interface? AFAIK, project groups are handled separately, using lp.project_groups lp.projects['fooprojectgroup'] happens to work because launchpadlib's ProjectSet generates a URL of /fooprojectgroup, and that URL namespace is shared between projects, distributions and project groups. The returned interfaces are heterogeneous because of exactly the hack that Ian proposes as the easy solution -- indexing a client-defined collection like ProjectSet just generates a URL with a known structure and requests that resource, respecting whatever resource type is returned. Right, and thats why I'm saying the previously sketched thing of respecting such information consistently will reduce the special casing code, close of a couple of long standing bugs, be easier to maintain and work with. So I rejected your MP and pointed you at the root cause. It still lets you expose a collection of services but you don't need the special casing. I don't think it is a distraction in any way. (thats bug #340935 - thanks wgrant) -Rob ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-dev] Help please - launchpadlib collections
lp.projects['fooprojectgroup'] happens to work because launchpadlib's ProjectSet generates a URL of /fooprojectgroup, and that URL namespace is shared between projects, distributions and project groups. The returned interfaces are heterogeneous because of exactly the hack that Ian proposes as the easy solution -- indexing a client-defined collection like ProjectSet just generates a URL with a known structure and requests that resource, respecting whatever resource type is returned. Right, and thats why I'm saying the previously sketched thing of respecting such information consistently will reduce the special casing code, close of a couple of long standing bugs, be easier to maintain and work with. So I rejected your MP and pointed you at the root cause. It still lets you expose a collection of services but you don't need the special casing. I don't think it is a distraction in any way. (thats bug #340935 - thanks wgrant) Oh, so that's why I couldn't find the bug - I was looking in bugs for the launchpadlib project. I had no idea the wadllib project even existed. And why I kept saying to myself what bug?. Various conversations around fixing the problem without introducing another client collection seemed to lead to having to modify launchpadlib and the wadl generation amongst other things, and seemed much more involved/complicated than the original proposal. Given that advice and my lack of previous exposure to the vagaries of wadl generation et al, it is a distraction since my main aim is to get the disclosure work finished and this stuff is at this stage more of a nice to have from that perspective. ___ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
