Tom Mueller (pkg-discuss) wrote: [snip removing arguments that are not going anywhere] >> Everything you've described can be done currently with the API as it's >> written. All a client should need to know (at least this is my opinon >> until I'm given a convincing counterpoint) is which packages are >> changing, what way they're changing, and enough information about >> those packages to answer questions the user may have. I believe the >> API provides exactly that, and, if it's deficient, can be expanded. >> > This is definitely a question of aesthetics. Just because it is possible > with the API doesn't mean that it is simple and easy to understand for > users and easy to extent when needed. > I believe the current API meets these demands. >>> [snip] >>> >>> >>>> If so, I'd happily take suggestions. I'm mildly against moving >>>> things into __init__.py as I think it's a fairly python specific >>>> construction which provides a needless barrier to entry to others >>>> coming to use our code, something I'd suggest is something an API >>>> should avoid where possible. >>>> >>>> >>> Moving it into __init__.py makes it so that the only interface the >>> users need to know about is "import pkg.client". >>> Seems pretty simple. I don't know why this would be a barrier to >>> entry, assuming that there is appropriate documentation with the API. >>> >>> >> It's a barrier to entry because __init__.py exactly obviously a class >> module. It's certainly not the first place anyone would go looking for >> an api if they downloaded the gate. >> > Why would anyone wanting to use the API download the gate? When you use > any other python module, do you download the source code for that module? > > One of the primary parts of having an API is that it should be > documented. We need to have python manual-like documentation for the > pkg(5) API. > As I've said, repeatedly before, documentation is coming. I can either document the API, or try to make it actually functional or complete. >> If I'm a newbie to python, I'd go look for the client.py file in pkg >> directory, and immediately hit a brick wall, cause, wait, there's no >> file named that there. This something that python does that many >> languages don't (C, C++, Java, OCaml, I feel fairly certain don't have >> a similar, commonly used organization. With less certainly, I'd add >> scheme, ML, Perl and Prolog to that list.). It's the same reason I'd >> avoid having the API return generator functions for example. I believe >> we'd like the API to be accessible to programmers from all >> backgrounds, not just Python wizards. >> > The solution to that is documentation. > See above. No, ONE solution is documentation. The other is to actually use commonly known practices.
[snip see earlier comments about arguments not going anywhere] >>>>> - the PkgClient class requires a ProgressTracker object. So is >>>>> this API making ProgressTracker a public interface? Also, if one >>>>> wants to use different progress trackers for different operations, >>>>> how does one do that? (for example, different progress trackers >>>>> for loading the catalogs vs. making the install plan in the >>>>> plan_install method.) >>>>> >>>>> >>>> Yes, as I've said previously in email here, the ProgressTracker is >>>> part of the API. >>>> >>>> >>> Is there a document that lists what classes and methods are all part >>> of the API, similar to what we see in the Python manual for each module? >>> >>> >> Nope, I've been more focused on actually making the thing work. >> But, the list, as I know it today, is everything in api.py, >> api_errors.py, progress.py, and bits and pieces of misc.py (which I >> hope to get removed eventually, but I lost that battle recently, and >> I'll focus on other things before returning to that issue again). >> > and fmri.py and parts of version.py (DotSequence). I recommend making > it a high priority to document this list. > You're welcome to document it if you believe it's such an urgent need. I believe other things are higher priority. [snip] >>>>> What is the intent of the __activity_lock functionality? We >>>>> eventually need to have locking that prevents multiple clients (in >>>>> separate processes) from modifying an image concurrently. Once we >>>>> have that, the same functionality will also work for different >>>>> threads in a single process. So thread-based locking is not really >>>>> useful here. It's also not clear to me that this API layer should >>>>> be responsible for locking, as that may not provide sufficient >>>>> granularity. >>>>> >>>>> >>>> The point of locking is so that, for example, multiple GUI threads >>>> can't start planning an install, an uninstall, and an image_update >>>> all at the same time on the same PkgClient object, for example. Can >>>> this become more finely grained in the future? Probably. But for >>>> now, they help make the code thread safe, something that's important >>>> for the GUI. When we have the more general mechanism, perhaps, and >>>> only perhaps, it can replace the mechanisms here. >>>> >>>> >>> IMHO, cross process concurrency is a bigger issue. And solving that >>> problem would solve the thread one too. >>> >>> >> I agree, but, that's not THIS problem. If and when we solve that >> issue, the locking code in api can poof (probably). This API is not >> going to fix all problems with our code. It's supposed to fix a select >> few. Cross-process concurrency is not in that set. Of course, if >> someone has a ready solution, by all means give me the patch, either >> to the gate or on top of my API, and I'll integrate it into my putback >> happily. >> > My point is that I don't see anything in issue 3443 that says the API > has to fix the cross-thread problem either (except maybe for the word > "safe" in the summary, but the description doesn't say anything about > it). The introduction of locking into the API seems to be feature creep. > Ok, so your objection is that I saw a potential problem and put in a simple fix before it became an issue a user filed a bug about it? >>>>> At what point would you expect the imports of the other >>>>> pkg.client.* and pkg.* modules to be removed from client.py? >>>>> >>>>> >>>> When the API is complete, and not in development. >>>> >>>> >>>>> (before or after the first checkin of this API?) >>>>> >>>>> >>>> After, since I foresee at least 2 stages of commits, and probably 3 >>>> >>>> >>>>> I'd like to at least see a plan for how complicated this api >>>>> module will get in order to be able to remove those imports, so >>>>> that the true expense of this option will be understood before the >>>>> team has to live with it. >>>>> >>>>> >>>> So, what you'd like is for me to do all the work needed to >>>> completely analyze what the final API would look like (which >>>> basically means creating and implementing the final API since, at >>>> least for me, that's the only way to be certain I've gotten things >>>> correct). That's fine, but there's little chance that can be >>>> finished in time for the GUI team to move to it for the november >>>> release. Until I receive firm direction that we've changed our >>>> schedule on this, I'm going to proceed as I've planned. >>>> >>>> I'll make one other point, if this option, once it's fully in place, >>>> is truly so terrible that we as a team "can't live with it", then we >>>> can blow it up and someone else can try again. All that we'll have >>>> lost is the time I've spent on it (and the amount of time we've >>>> spent kvetching about it). >>>> >>>> >>> In a previous message[1], Stephen outlined three general options for >>> introducing a stable API. It seems like you've picked option 1 >>> without even considering option 2 or 3. In fact, you've started to >>> write the code for option 1 without having looked at what a design >>> for options 2 or 3 would even look like (at least I haven't seen such >>> a design sent to the pkg-discuss alias, maybe options 2 and 3 were >>> discarded having never made it that far). Since this started, I've >>> just been asking that there be a reasonable evaluation of these >>> various options. This isn't a question of trying one thing and then >>> having someone else try another thing. If the various major design >>> options are considered and then compared side by side, then an >>> appropriate design can be selected. And if option 1 is selected, I >>> can support that (as I've tried to do by providing these comments). >>> >>> >> No, I've considered 2 and 3, just because I haven't put them out to >> the group doesn't mean I didn't think about them. (I'll group them >> together as I believe they're essentially isomorphic transformations >> of the code. While the details might differ, the overall structure >> would be similar.) I viewed my job as to look at the various options, >> and provide a solution to the problem. Had there been two options >> which seemed to be reasonable alternatives, then I would've turned to >> the group for guidance. Since, in my opinion, the choice was clear, I >> moved ahead. I'd suggest that 2 or 3 is what you and some others have >> proposed previously, so it's certainly been considered. I feel like >> I've done the best I can to explain why I don't believe those are >> adequate solutions to our problems. >> > Did I miss that explanation? I'm sorry if I passed that by. I'd like to > take another look at the explanation for why #3 would not solve the > problems. The very first message I saw on the API is here: > http://mail.opensolaris.org/pipermail/pkg-discuss/2008-September/006168.html > . And this doesn't say anything about why the #3 approach wouldn't solve > the problems. > Our entire discussion, essentially has been about my preference for option #1 over #'s 2 and 3, and yours for #2 or #3 over #1. I'm not really interested in rehashing it again, but the short summary is that #1 provides opportunities for refactoring the interfaces to make things simpler and to more cleanly separate implementation from interface (not letting the client know about a plan beyond, essentially what packages will change and whether any cautions need to be presented to the user.) Brock > Tom > >> I get that you disagree. If you'd like, you can put together an API >> which follows options 2 or 3, and we as a group can choose between >> them. I'm not going to spend my time doing that because I've done it >> internally, and found them lacking for our problems. You can also >> continue to try to convince me that you're right, but please >> understand that until I am convinced, I'm going to plow ahead because >> we have deadlines to meet and other things to do. >> >>> Tom >>> >>> [1] >>> http://mail.opensolaris.org/pipermail/pkg-discuss/2008-September/006464.html >>> >>> >>> >>> >>>>> Thanks. >>>>> Tom >>>>> >>>>> >>>>> >>>>> Brock Pytlik wrote: >>>>> >>>>> >>>>> >>>>>> Here's the link: >>>>>> http://cr.opensolaris.org/~bpytlik/ips-api-v1/ >>>>>> >>>>>> I haven't done as much cleanup as I usually try to do, but since >>>>>> I'm ooto tomorrow, I thought it would be better to get this out >>>>>> now, rather than next Monday. >>>>>> >>>>>> Other things that I know need to be improved, but that I think can >>>>>> wait for another round of revisions: >>>>>> API Exception organization >>>>>> More complete functionality >>>>>> Automated testing of cancellation (I hope to work with the GUI >>>>>> team to test this by hand in the interim) >>>>>> Versioning of the Progress tracker (perhaps) >>>>>> More Documentation >>>>>> Figure out the future approach to testing we'll take >>>>>> >>>>>> Thanks, >>>>>> Brock >>>>>> _______________________________________________ >>>>>> pkg-discuss mailing list >>>>>> [email protected] >>>>>> http://mail.opensolaris.org/mailman/listinfo/pkg-discuss >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> pkg-discuss mailing list >>>>> [email protected] >>>>> http://mail.opensolaris.org/mailman/listinfo/pkg-discuss >>>>> >>>>> >>> _______________________________________________ >>> pkg-discuss mailing list >>> [email protected] >>> http://mail.opensolaris.org/mailman/listinfo/pkg-discuss >>> >>> > > _______________________________________________ > pkg-discuss mailing list > [email protected] > http://mail.opensolaris.org/mailman/listinfo/pkg-discuss > _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
