This is an automated email from the ASF dual-hosted git repository.
sbp pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-trusted-release.git
The following commit(s) were added to refs/heads/main by this push:
new 77551b5 Add documentation about the storage interface
77551b5 is described below
commit 77551b5961b6fd13e5442cb8b6315f31f95997fa
Author: Sean B. Palmer <s...@miscoranda.com>
AuthorDate: Mon Jul 21 11:24:13 2025 +0100
Add documentation about the storage interface
---
docs/storage-interface.html | 60 +++++++++++++++++++++++++++++++
docs/storage-interface.md | 86 +++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 146 insertions(+)
diff --git a/docs/storage-interface.html b/docs/storage-interface.html
new file mode 100644
index 0000000..9c9165d
--- /dev/null
+++ b/docs/storage-interface.html
@@ -0,0 +1,60 @@
+<h1>Storage interface</h1>
+<p>All writes to the database and filesystem are to be mediated through the
storage interface in <code>atr.storage</code>. The storage interface
<strong>enforces permissions</strong>, <strong>centralises audit
logging</strong>, and <strong>exposes misuse resistant methods</strong>.</p>
+<h2>How do we use the storage interface?</h2>
+<p>Open a storage interface session with a context manager. Then:</p>
+<ol>
+<li>Request permissions from the session depending on the role of the
user.</li>
+<li>Use the exposed functionality.</li>
+<li>Handle the outcome or outcomes.</li>
+</ol>
+<p>Here is an actual example from our API code:</p>
+<pre><code class="language-python">async with storage.write(asf_uid) as write:
+ wafm = write.as_foundation_member().writer_or_raise()
+ ocr: types.Outcome[types.Key] = await wafm.keys.ensure_stored_one(data.key)
+ key = ocr.result_or_raise()
+
+ for selected_committee_name in selected_committee_names:
+ wacm =
write.as_committee_member(selected_committee_name).writer_or_raise()
+ outcome: types.Outcome[types.LinkedCommittee] = await
wacm.keys.associate_fingerprint(
+ key.key_model.fingerprint
+ )
+ outcome.result_or_raise()
+</code></pre>
+<p>The <code>wafm</code> (<strong>w</strong>rite <strong>a</strong>s
<strong>f</strong>oundation <strong>m</strong>ember) object exposes
functionality which is only available to foundation members. The
<code>wafm.keys.ensure_stored_one</code> method is an example of such
functionality. The <code>wacm</code> object goes further and exposes
functionality only available to committee members.</p>
+<p>In this case we decide to raise as soon as there is any error. We could
also choose instead to display a warning, ignore the error, etc.</p>
+<p>The first few lines in the context session show the classic three step
approach. Here they are again with comments:</p>
+<pre><code> # 1. Request permissions
+ wafm = write.as_foundation_member().writer_or_raise()
+
+ # 2. Use the exposed functionality
+ ocr: types.Outcome[types.Key] = await wafm.keys.ensure_stored_one(data.key)
+
+ # 3. Handle the outcome
+ key = ocr.result_or_raise()
+</code></pre>
+<h2>How do we add functionality to the storage interface?</h2>
+<p>Add all the functionality to classes in modules in the
<code>atr/storage/writers</code> directory. Code to write public keys to
storage, for example, goes in <code>atr/storage/writers/keys.py</code>.</p>
+<p>Classes in modules in the <code>atr/storage/writers</code> directory must
be named as follows:</p>
+<pre><code>class FoundationParticipant:
+ ...
+
+class FoundationMember(FoundationParticipant):
+ ...
+
+class CommitteeParticipant(FoundationMember):
+ ...
+
+class CommitteeMember(CommitteeParticipant):
+ ...
+</code></pre>
+<p>This creates a hierarchy, <code>FoundationParticipant</code> →
<code>FoundationMember</code> → <code>CommitteeParticipant</code> →
<code>CommitteeMember</code>. We can add other permissions levels if
necessary.</p>
+<p>Use <code>__private_methods</code> for code specific to one permission
level which is not exposed in the interface, e.g. helpers. Use
<code>public_methods</code> for code appropriate to expose when users meet the
appropriate permission level. Consider returning outcomes, as explained in the
next section.</p>
+<h2>Returning outcomes</h2>
+<p>Consider using the <strong>outcome types</strong> in
<code>atr.storage.types</code> when returning results from writer module
methods. The outcome types <em>solve many problems</em>, but here is an
example:</p>
+<p>Imagine the user is submitting a <code>KEYS</code> file containing several
keys. Some of the keys are already in the database, some are not in the
database, and some are broken keys that do not parse. After processing, each
key is associated with a different state: the key was parsed but not added, the
key was parsed and added, or the key wasn't even parsed. We consider some of
these success states, some warning states, and others error states.</p>
+<p>How do we represent this?</p>
+<p>Outcomes are one possibility. For each key we can return
<code>OutcomeResult</code> for a success, and <code>OutcomeException</code>
when there was a Python error. The caller can then decide what to do with this
information. It might ignore the exception, raise it, or print an error message
to the user. Better yet, we can aggregate these into an <code>Outcomes</code>
list, which provides many useful methods for processing all of the outcomes
together. It can count how many exceptions [...]
+<p>We do not have to return outcomes from public storage interface methods,
but these classes were designed to make the storage interface easy to use.</p>
+<h2>What makes this safe?</h2>
+<p>We can always open a database session or write to the filesystem, so there
is no way to make storage access truly safe. But abstracting these operations
to a well known interface makes it more likely that we use only this way of
doing things, which we can then concentrate on getting right. This is in
contrast to writing storage access in <em>ad hoc</em> ways, some of which may
be correct and some of which may not.</p>
+<p>Code relative to a permissions level is only ever exposed in the storage
interface when it is proven, at the type level and during runtime, that the
user has credentials for those permissions. Helper code remains private due to
the use of <code>__private_methods</code>, which undergo name mangling in
Python. As mentioned in the introduction, the storage interface is also the
suitable place to add audit logging, currently planned and not yet
implemented.</p>
diff --git a/docs/storage-interface.md b/docs/storage-interface.md
new file mode 100644
index 0000000..ef7d04c
--- /dev/null
+++ b/docs/storage-interface.md
@@ -0,0 +1,86 @@
+# Storage interface
+
+All writes to the database and filesystem are to be mediated through the
storage interface in `atr.storage`. The storage interface **enforces
permissions**, **centralises audit logging**, and **exposes misuse resistant
methods**.
+
+## How do we use the storage interface?
+
+Open a storage interface session with a context manager. Then:
+
+1. Request permissions from the session depending on the role of the user.
+2. Use the exposed functionality.
+3. Handle the outcome or outcomes.
+
+Here is an actual example from our API code:
+
+```python
+async with storage.write(asf_uid) as write:
+ wafm = write.as_foundation_member().writer_or_raise()
+ ocr: types.Outcome[types.Key] = await wafm.keys.ensure_stored_one(data.key)
+ key = ocr.result_or_raise()
+
+ for selected_committee_name in selected_committee_names:
+ wacm =
write.as_committee_member(selected_committee_name).writer_or_raise()
+ outcome: types.Outcome[types.LinkedCommittee] = await
wacm.keys.associate_fingerprint(
+ key.key_model.fingerprint
+ )
+ outcome.result_or_raise()
+```
+
+The `wafm` (**w**rite **a**s **f**oundation **m**ember) object exposes
functionality which is only available to foundation members. The
`wafm.keys.ensure_stored_one` method is an example of such functionality. The
`wacm` object goes further and exposes functionality only available to
committee members.
+
+In this case we decide to raise as soon as there is any error. We could also
choose instead to display a warning, ignore the error, etc.
+
+The first few lines in the context session show the classic three step
approach. Here they are again with comments:
+
+```
+ # 1. Request permissions
+ wafm = write.as_foundation_member().writer_or_raise()
+
+ # 2. Use the exposed functionality
+ ocr: types.Outcome[types.Key] = await wafm.keys.ensure_stored_one(data.key)
+
+ # 3. Handle the outcome
+ key = ocr.result_or_raise()
+```
+
+## How do we add functionality to the storage interface?
+
+Add all the functionality to classes in modules in the `atr/storage/writers`
directory. Code to write public keys to storage, for example, goes in
`atr/storage/writers/keys.py`.
+
+Classes in modules in the `atr/storage/writers` directory must be named as
follows:
+
+```
+class FoundationParticipant:
+ ...
+
+class FoundationMember(FoundationParticipant):
+ ...
+
+class CommitteeParticipant(FoundationMember):
+ ...
+
+class CommitteeMember(CommitteeParticipant):
+ ...
+```
+
+This creates a hierarchy, `FoundationParticipant` → `FoundationMember` →
`CommitteeParticipant` → `CommitteeMember`. We can add other permissions levels
if necessary.
+
+Use `__private_methods` for code specific to one permission level which is not
exposed in the interface, e.g. helpers. Use `public_methods` for code
appropriate to expose when users meet the appropriate permission level.
Consider returning outcomes, as explained in the next section.
+
+## Returning outcomes
+
+Consider using the **outcome types** in `atr.storage.types` when returning
results from writer module methods. The outcome types _solve many problems_,
but here is an example:
+
+Imagine the user is submitting a `KEYS` file containing several keys. Some of
the keys are already in the database, some are not in the database, and some
are broken keys that do not parse. After processing, each key is associated
with a different state: the key was parsed but not added, the key was parsed
and added, or the key wasn't even parsed. We consider some of these success
states, some warning states, and others error states.
+
+How do we represent this?
+
+Outcomes are one possibility. For each key we can return `OutcomeResult` for a
success, and `OutcomeException` when there was a Python error. The caller can
then decide what to do with this information. It might ignore the exception,
raise it, or print an error message to the user. Better yet, we can aggregate
these into an `Outcomes` list, which provides many useful methods for
processing all of the outcomes together. It can count how many exceptions there
were, for example, or apply a [...]
+
+We do not have to return outcomes from public storage interface methods, but
these classes were designed to make the storage interface easy to use.
+
+## What makes this safe?
+
+We can always open a database session or write to the filesystem, so there is
no way to make storage access truly safe. But abstracting these operations to a
well known interface makes it more likely that we use only this way of doing
things, which we can then concentrate on getting right. This is in contrast to
writing storage access in _ad hoc_ ways, some of which may be correct and some
of which may not.
+
+Code relative to a permissions level is only ever exposed in the storage
interface when it is proven, at the type level and during runtime, that the
user has credentials for those permissions. Helper code remains private due to
the use of `__private_methods`, which undergo name mangling in Python. As
mentioned in the introduction, the storage interface is also the suitable place
to add audit logging, currently planned and not yet implemented.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@tooling.apache.org
For additional commands, e-mail: commits-h...@tooling.apache.org
