lahirujayathilake opened a new issue, #466: URL: https://github.com/apache/airavata-custos/issues/466
## Task ACCESS-CI delivers allocation events to resource providers via the AMIE protocol. A request/reply messaging convention covering project creates, account creates, supplements, renewals, deactivations, and merges. This issue is the public entry point for how Custos integrates with that protocol. The integration is a dedicated connector at [`connectors/ACCESS/AMIE-Processor/`](https://github.com/apache/airavata-custos/tree/master/connectors/ACCESS/AMIE-Processor) that polls the ACCESS AMIE API, decodes each packet, dispatches it to a typed handler, applies the resulting state changes to the Custos domain model atomically, and replies back to ACCESS. The connector writes into a source-agnostic identity model in [`core/domain/`](https://github.com/apache/airavata-custos/tree/master/core/domain), allowing future allocation sources (NAIRR, internal, etc.) to live alongside without schema changes. ## How it should work - A **poller** pulls pending AMIE packets every 30 seconds (configurable) and persists them to `amie_packets` with corresponding `amie_processing_events`. - A **processor** drains pending events every 5 seconds, claims each inside a database transaction, and dispatches to a typed handler by packet type. - Each **handler** does a specific task. Find or create Person, provision ClusterAccount, create Project / ProjectMembership, write Award + Allocation + GRANT CreditTransfer where applicable and sends the corresponding `notify_*` or `data_*` reply back to ACCESS. - An **audit log** in core records every action tagged with `source_module='access_amie'` and packet/event correlation in JSON metadata. - Retries are exponential (30s → 60s → 600s, max 3 attempts); failed packets become rows in `amie_processing_errors` for inspection. The domain model should splits in two. * **Core entities**: `persons`, `external_identities`, `person_dns`, `audit_log`, live in `core/domain/` and are source-agnostic, any future allocation connector writes into them tagged with its own source name. * **Allocation entities**: `projects`, `cluster_accounts`, `project_memberships`, `awards`, `allocations`, `credit_transfers`, live in the AMIE connector for the time being. `request_project_create` is the only allocation holding packet. It carries an `AllocationType` field differentiating initial allocations from supplements and renewals, the handler should branches on a case-insensitive match (new / supplement / renewal) and writes the corresponding Award + Allocation + GRANT ledger entry atomically. Credit transfers are append-only, and the ledger's integrity should be preserved. The AMIE pipeline is already in the master branch supporting the workflows; poller, processor, packet decoding, typed handlers, replies, retries, metrics, Grafana dashboard, k6 load tests, local mock server. [PR](https://github.com/apache/airavata-custos/pull/452) With this issue need to support the allocation hierarchy; awards, allocations, credit_transfers; allocation service wired into `request_project_create` with new / supplement / renewal branching. ## References - AMIE protocol (ACCESS-CI Confluence): https://access-ci.atlassian.net/wiki/spaces/ACP/pages/589496333/AMIE+Documentation - SLURM mapping (downstream): [#462](https://github.com/apache/airavata-custos/issues/462) - Source: [`connectors/ACCESS/AMIE-Processor/`](https://github.com/apache/airavata-custos/tree/master/connectors/ACCESS/AMIE-Processor) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
