On Wed, Jul 22, 2009 at 10:03:07PM -0500, Shawn Walker wrote: > >Hm. Do we really want to bloat a catalog with dozens of languages? I > >assume that almost all clients will want just one language, and however > >fast the serialization and deserialization is, multiplying the data in > >there is going to slow things down. > > The issue comes in with signing. I'm willing to re-work this to > split the catalogs by locale again, but when I asked Bart about > this, he had mentioned that, at the moment, there were no plans to > split manifests by locale. > > The reasoning, by extension, was that the same would have to be > applied to catalogs.
I don't see how that follows. The server can (and should) keep catalogs by locale, too. I really don't want to go down the road that the GNOME .desktop files took, with all locales in a single file. It's a horror. > >> and all data is assumed to be encodable using UTF-8. > > > >We'll want an escape for this, I'm pretty sure. > > This may or may not put a damper on using JSON, but JSON only allows > Unicode: > > "The character encoding of JSON text is always Unicode. UTF-8 is the > only encoding that makes sense on the wire, but UTF-16 and UTF-32 > are also permitted." [1] > > If we must support non-Unicode encoding, then I will have to abandon > JSON in favour of the manifest-style format. Keep it as it is for now. I think we can support other encodings later, if necessary. > >> 'created': '2005-06-14T08:00:00.686485', > >> 'last-modified': '2009-05-08T16:10:25.686485', > > > >These should be in UTC, right? Also, is there a reason you chose the > >extended format, rather than the basic format used in the FMRIs? > > Yes, those should have a 'Z' on the end. And yes, I used the > ISO-8601 format instead of the basic format used in FMRIs since > that's a standard format that should be easily parsable and requires > no explanation or special logic The basic format doesn't fit those requirements? They're both part of the 8601 standard. > (ignoring the fact that python 2.4 can't parse microseconds in ISO-8601 > dates for datetime objects despite the fact that the standard allows > them). Do future versions? We'll be running 2.6 soon enough, I hope. > >I wonder if we couldn't push the actions fully into JSON, too: > > There are two reasons I didn't go that route: > > 1) serialization overhead (even on x86) > > 2) file-size difference (much larger) > > 3) Seemed pointless, since the client api consumes actions, and the > fastest way to transform the data back into an action object is > through the action string. As Bart pointed out to me the other day, > the Catalog is a glorified action pipeline. Okay, just checking. > >We've been talking about providing the install folks with a quick way of > >determining the size of packages to be installed; it would make sense for > >this information to go here as well. There would be multiple entries, for > >all the combinations of variants and facets. It would also be synthetic, > >not pulled directly from the manifest. > > > >I would also expect package rename and obsoletion set actions to go here as > >well. > > I'll assume the above are FYIs only and not a request for a change > to the proposed format or entries. Correct. I just figured as long as we were listing consumers, we might as well list all currently known ones. As long as having this data doesn't change the spec (and I don't think it does), then you can add that information or not, however you see fit. > The point of /var/pkg/publisher is that metadata, other than catalog > data, will eventually be stored on a per-publisher basis. So, instead of > having /var/pkg/catalog /var/pkg/<metatype1> /var/pkg/<metatype2>, I'm > just organising this a bit more. > > To be clear, the intent is really: > > /var/pkg/publisher/<prefix>/ > > But, it probably should be: > > /var/pkg/publisher/<prefix>/catalogs/ What other non-catalog metadata do you see going there? > >> - For performance reasons, the client api will also store > >> versions of each of the catalogs proposed that only > >> contain entries for installed FMRIs to accelerate common > >> client functions such as info, list, uninstall, etc. > > > >Does/could this eliminate the need for /var/pkg/state/install, as well? > >And the "installed" files. > > It does. And by that, will it be part of the initial wad, or come some time later? > >> It was discovered that the likely reason for poor serialization on > >> some SPARC systems is that simplejson uses a recursive function- > >> based iterative encoder that does not perform well on SPARC systems > >> (due to register windows?). > > > >We have a workaround now, but we should probably file a bug on this and > >possibly work on an implementation that isn't recursive. > > Where should that bug be filed? On the Python bug tracking system. Danek _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
