Shawn Walker wrote:
Greetings,
So, as part of the catalog work that's happening, it has become clear
that the repository and client metadata needs to be reorganised (at a
relatively high-level; this doesn't conflict with Brock's fanout work).
In particular, we're open to namespace collisions when storing package
manifests because publisher is not part of the scheme used to store
files (e.g. pkg/foo/1.0 manifest from publisher A and pkg/foo/1.0
manifest from publisher B).
The current structure also makes certain repository operations or cache
management policies difficult or expensive to perform.
The below proposal is an attempt to resolve this, while unifying the
client and server storage schemes.
...
__cert/
So, the open question here is whether all certs will be 'ssl' certs? If
they are, this should be 'ssl'.
__history/
__index/
__pm_cache/
__state/
<publisher>/
file/ (formerly download)
pkg/
<stem>/
<manifest-named-after-uri-encoded-version>
pkg_cache/
<stem>/
<uri-encoded-version>.<cache_name> (manifest cache file)
Danek suggested that dropping the '__' prefix and moving the <publisher>
directories down one level (under a directory named 'publisher') might
be preferable. This would fit nicely with the existing scheme used by
the client which already has a /var/pkg/publisher directory (in the gate
currently).
Initial testing and research indicates that each additional level of
depth increases the time listdir operations take by about 3-5% when
comparing timings of 1,000,000 recursive os.listdir()s of the three
following structures starting at 'file/':
<ROOT>/
file/
00/
04b0bf/
0004b0bf77d1e13658849868b80dcd581d83cc0b
<ROOT>/
opensolaris.org/
file/
00/
04b0bf/
0004b0bf77d1e13658849868b80dcd581d83cc0b
<ROOT>/
publisher/
opensolaris.org/
file/
00/
04b0bf/
0004b0bf77d1e13658849868b80dcd581d83cc0b
As such, are there any objections to the layout below, or should I
retain the original proposed scheme?
==============================
Proposed Server Storage Scheme
==============================
<REPO_ROOT>/
catalog/
index/
publisher/
<publisher>/
file/
pkg/
<stem>/
<manifest-named-after-uri-encoded-version>
trans/
==============================
Proposed Client Storage Scheme
==============================
<IMG_ROOT>/
cert/
history/
index/
pm_cache/
state/
publisher/
<publisher>/
file/ (formerly download)
pkg/
<stem>/
<manifest-named-after-uri-encoded-version>
trans/
pkg_cache/
<stem>/
<uri-encoded-version>.<cache_name> (manifest cache file)
Cheers,
--
Shawn Walker
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
