asf-tooling commented on issue #1233:
URL:
https://github.com/apache/tooling-trusted-releases/issues/1233#issuecomment-4409630736
<!-- gofannon-issue-triage-bot v2 -->
**Automated triage** — analyzed at `main@2da7807a`
**Type:** `bug_fix` • **Classification:** `actionable` • **Confidence:**
`medium`
**Application domain(s):** `artifact_upload`
### Summary
The issue reports that an SVN import of files for project 'otava' version
'0.8.0' fails with 'Path cannot contain dotfiles', but the path likely contains
a version string like '0.8.0' which has dots but is NOT a dotfile (a dotfile is
a file/directory whose name starts with '.'). The validation logic in the
`safe.RelPath` or related path-handling code appears to incorrectly flag path
components containing dots as 'dotfiles' rather than checking only whether a
path component *begins* with a dot. The task `svn_import_files` runs as a
background worker, and the error is only visible in the task log — not surfaced
to the user ('invisible and unreported').
### Where this lives in the code today
#### `atr/shared/upload.py` — `SvnImportForm` (lines 38-57)
_currently does this_
The SVN import form uses safe.RelPath for path validation; the form
submission succeeded (task was queued), so the initial path validated, but
later processing in the task fails.
```python
class SvnImportForm(form.Form):
variant: SVN_IMPORT = form.value(SVN_IMPORT)
# svn_area: form.Enum[SvnArea] = form.label(
# "svn:dist area",
# "Select whether to import from dev or release.",
# widget=form.Widget.RADIO,
# )
svn_path: safe.RelPath = form.label(
"SVN path",
"Path within the committee's svn:dist directory, e.g.
'java-library/4_0_4' or '3.1.5rc1'.",
)
revision: str = form.label(
"Revision",
"Specify an SVN revision number or leave as HEAD for the latest.",
default="HEAD",
)
target_subdirectory: safe.OptionalRelPath = form.label(
"Target subdirectory",
"Optional: Subdirectory to place imported files, defaulting to the
root.",
)
```
#### `atr/post/upload.py` — `_construct_svn_url` (lines 137-142)
_currently does this_
Constructs the SVN URL by prepending area/committee to the user-supplied
path; output is safe.RelPath which implies the prepend result also passes
validation.
```python
def _construct_svn_url(
committee_key: str, area: shared.upload.SvnArea, path: safe.RelPath, *,
is_podling: bool
) -> safe.RelPath:
if is_podling:
return path.prepend(f"{area.value}/incubator/{committee_key}")
return path.prepend(f"{area.value}/{committee_key}")
```
#### `atr/post/upload.py` — `_svn_import` (lines 152-198)
_currently does this_
The _svn_import function queues the task via storage; the actual
'svn_import_files' task executes asynchronously in a worker, where the dotfile
check fails.
```python
async def _svn_import(
session: web.Committer,
svn_form: shared.upload.SvnImportForm,
project_key: safe.ProjectKey,
version_key: safe.VersionKey,
) -> web.WerkzeugResponse:
# audit_guidance any file uploads are from known and managed
repositories so file size is not an issue
try:
target_subdirectory = svn_form.target_subdirectory if
svn_form.target_subdirectory else None
svn_area = shared.upload.SvnArea.DEV
svn_path = svn_form.svn_path
async with db.session() as data:
release = await session.release(project_key, version_key,
data=data)
is_podling = (release.project.committee is not None) and
release.project.committee.is_podling
committee_key = release.project.committee_key or str(project_key)
svn_url = _construct_svn_url(
committee_key,
svn_area, # pyright: ignore[reportArgumentType]
svn_path,
is_podling=is_podling,
)
async with storage.write(session) as write:
wacp = await write.as_project_committee_participant(project_key)
await wacp.release.import_from_svn(
project_key,
version_key,
svn_url,
svn_form.revision,
target_subdirectory,
)
return await session.redirect(
get.compose.selected,
success="SVN import task queued successfully",
project_key=project_key,
version_key=version_key,
)
except Exception:
log.exception("Error queueing SVN import task:")
return await session.redirect(
get.upload.selected,
error="Error queueing SVN import task",
project_key=project_key,
version_key=version_key,
)
```
### Where new code would go
- `atr/models/safe.py` — RelPath class or validation method
The 'Path cannot contain dotfiles' error message almost certainly
originates from validation logic in the safe.RelPath type (or a related path
validation helper). The dotfile check likely needs to be fixed to only reject
components that START with a dot, not components that merely contain a dot.
- `atr/tasks/svn.py or atr/storage/writers/release.py` — svn_import_files
task implementation
The actual background task that performs the SVN import and triggers the
validation error. The dotfile check may be applied to filenames downloaded from
SVN rather than just the user-supplied path.
### Proposed approach
The root cause is almost certainly in `atr/models/safe.py` where path
validation for dotfiles is implemented. The check likely uses something like
`'.' in component` rather than `component.startswith('.')` when inspecting path
segments. For a directory name like `0.8.0` or `apache-otava-0.8.0`, the
component contains dots but doesn't start with one — it's not a dotfile.
The fix should modify the dotfile validation to only reject path components
that begin with a dot (e.g., `.git`, `.svn`, `.DS_Store`) while allowing
components that contain dots elsewhere (e.g., `0.8.0`, `file.tar.gz`). I need
to see `atr/models/safe.py` to confirm the exact validation logic and produce a
precise diff. Additionally, the error reporting should surface this failure to
the user more visibly rather than only showing it in the task log.
### Open questions
- What is the exact dotfile validation logic in atr/models/safe.py? Need to
see the RelPath class implementation to confirm whether it checks `'.' in
component` vs `component.startswith('.')`.
- Where is the `svn_import_files` task implemented? The background task that
actually downloads from SVN and processes files — need to see where it
validates individual file paths from the SVN checkout.
- Is the dotfile check applied to the SVN source path, the filenames within
the SVN directory, or the target path where files are placed?
- Should the error be surfaced to the user via a flash message or
notification system, addressing the 'invisible and unreported' aspect of the
issue?
### Files examined
- `atr/post/upload.py`
- `atr/shared/upload.py`
- `atr/tasks/quarantine.py`
---
*Draft from a triage agent. A human reviewer should validate before merging
any change. The agent did not run tests or verify diffs apply.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
