This is an automated email from the ASF dual-hosted git repository.
lahirujayathilake pushed a change to branch cybershuttle-dev
in repository https://gitbox.apache.org/repos/asf/airavata.git
from 5a1ce93b2e revise jupyterhub spawned notebooks, add makefiles
new 14badc1e98 initial experiment sdk implementation
new 1dcb49a8e7 improve auth flow, add python code invocation, add plan
crud apis and file ul/dl apis, reduce settings.ini deps, fix bugs
new cd2d2a9d7e venv support for agent, update notebooks, show tables as
pandas df, update dir refs, fix bugs.
new 919bfd1d9d make settings.ini lean, add runtime picker {by cluster,
category}. change agent base dir.
new 88541dede6 add pre-submission validator, update storage dir, take
CONNECTION_SVC_URL and FILEMGR_SVC_URL from settings.ini, cleanup runtime.py,
add fallback apis to airavata.py, rearrange files, setup file ul/dl apis,
improve ux, update notebooks, add agent scripts, refine code, fix bugs
new c09d8235e3 fix bug in code execution, remove jupyter kernel start from
agent add support to cold-start analysis agents reset changes to
sftp_file_handling_client update notebooks remove verbose log from remote code
execution separate python env creation and code execution steps. always return
stdout+err for command/script executions lock agent to python 3.12 and update
pyproject.toml update file listing and python execution cmds
new 276a64ec1d bump version to 0.0.16 add AlphaFold2 and Gaussian
applications update realm from 10000000 -> default add GROMACS and AMBER to MD
SDK
new 5deda3fa7a choose group/project from API, silent plan launch,
add_replica() -> create_task(), display task state, planid and taskid, improve
messaging
new 84e5ebf7ed add back kernel launcher to agent
new 36694d02b4 refine Makefiles, move app-scripts to agent-framework/, fix
bugs in agent.go and sample Dockerfile
new f58c097ab5 regenerate thrift specs
new 7147066445 cleanup airavata_sdk wrappers, error-handling in
airavata_experiments
new b5015274a9 add type-hinting imports to __init__ files
new b7f8a2b3a2 cleanup imports, fix bugs found by type checker
The 14 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../airavata-python-sdk/.gitignore | 8 +
.../airavata-python-sdk/airavata/__init__.py | 4 +
.../airavata/api/Airavata-remote | 4 +-
.../airavata-python-sdk/airavata/api/Airavata.py | 6252 +++++++++--------
.../airavata-python-sdk/airavata/api/__init__.py | 5 +
.../airavata-python-sdk/airavata/api/constants.py | 8 +-
.../airavata/api/credential/__init__.py | 4 +
.../credential/store/CredentialStoreService-remote | 4 +-
.../api/credential/store/CredentialStoreService.py | 349 +-
.../airavata/api/credential/store/__init__.py | 5 +
.../airavata/api/credential/store/constants.py | 8 +-
.../api/credential/store/error/__init__.py | 5 +
.../api/credential/store/error/constants.py | 8 +-
.../airavata/api/credential/store/error/ttypes.py | 12 +-
.../airavata/api/credential/store/ttypes.py | 8 +-
.../airavata/api/error/__init__.py | 5 +
.../airavata/api/error/constants.py | 8 +-
.../airavata/api/error/ttypes.py | 97 +-
.../api/sharing/SharingRegistryService-remote | 4 +-
.../airavata/api/sharing/SharingRegistryService.py | 1424 ++--
.../airavata/api/sharing/__init__.py | 5 +
.../airavata/api/sharing/constants.py | 8 +-
.../airavata/api/sharing/ttypes.py | 8 +-
.../airavata-python-sdk/airavata/api/ttypes.py | 8 +-
.../airavata-python-sdk/airavata/base/__init__.py | 4 +
.../airavata/base/api/BaseAPI-remote | 4 +-
.../airavata/base/api/BaseAPI.py | 22 +-
.../airavata/base/api/__init__.py | 5 +
.../airavata/base/api/constants.py | 8 +-
.../airavata/base/api/ttypes.py | 8 +-
.../airavata-python-sdk/airavata/model/__init__.py | 5 +
.../airavata/model/appcatalog/__init__.py | 4 +
.../appcatalog/accountprovisioning/__init__.py | 5 +
.../appcatalog/accountprovisioning/constants.py | 8 +-
.../model/appcatalog/accountprovisioning/ttypes.py | 56 +-
.../model/appcatalog/appdeployment/__init__.py | 5 +
.../model/appcatalog/appdeployment/constants.py | 8 +-
.../model/appcatalog/appdeployment/ttypes.py | 91 +-
.../model/appcatalog/appinterface/__init__.py | 5 +
.../model/appcatalog/appinterface/constants.py | 8 +-
.../model/appcatalog/appinterface/ttypes.py | 28 +-
.../model/appcatalog/computeresource/__init__.py | 5 +
.../model/appcatalog/computeresource/constants.py | 8 +-
.../model/appcatalog/computeresource/ttypes.py | 426 +-
.../model/appcatalog/gatewaygroups/__init__.py | 5 +
.../model/appcatalog/gatewaygroups/constants.py | 8 +-
.../model/appcatalog/gatewaygroups/ttypes.py | 20 +-
.../model/appcatalog/gatewayprofile/__init__.py | 5 +
.../model/appcatalog/gatewayprofile/constants.py | 8 +-
.../model/appcatalog/gatewayprofile/ttypes.py | 94 +-
.../appcatalog/groupresourceprofile/__init__.py | 5 +
.../appcatalog/groupresourceprofile/constants.py | 8 +-
.../appcatalog/groupresourceprofile/ttypes.py | 142 +-
.../model/appcatalog/parallelism/__init__.py | 5 +
.../model/appcatalog/parallelism/constants.py | 8 +-
.../model/appcatalog/parallelism/ttypes.py | 27 +-
.../airavata/model/appcatalog/parser/__init__.py | 5 +
.../airavata/model/appcatalog/parser/constants.py | 8 +-
.../airavata/model/appcatalog/parser/ttypes.py | 145 +-
.../model/appcatalog/storageresource/__init__.py | 5 +
.../model/appcatalog/storageresource/constants.py | 8 +-
.../model/appcatalog/storageresource/ttypes.py | 26 +-
.../appcatalog/userresourceprofile/__init__.py | 5 +
.../appcatalog/userresourceprofile/constants.py | 8 +-
.../model/appcatalog/userresourceprofile/ttypes.py | 68 +-
.../airavata/model/application/__init__.py | 4 +
.../airavata/model/application/io/__init__.py | 5 +
.../airavata/model/application/io/constants.py | 8 +-
.../airavata/model/application/io/ttypes.py | 115 +-
.../airavata/model/commons/__init__.py | 5 +
.../airavata/model/commons/constants.py | 8 +-
.../airavata/model/commons/ttypes.py | 40 +-
.../airavata/model/constants.py | 8 +-
.../airavata/model/credential/__init__.py | 4 +
.../airavata/model/credential/store/__init__.py | 5 +
.../airavata/model/credential/store/constants.py | 8 +-
.../airavata/model/credential/store/ttypes.py | 118 +-
.../airavata/model/data/__init__.py | 4 +
.../airavata/model/data/movement/__init__.py | 5 +
.../airavata/model/data/movement/constants.py | 8 +-
.../airavata/model/data/movement/ttypes.py | 157 +-
.../airavata/model/data/replica/__init__.py | 5 +
.../airavata/model/data/replica/constants.py | 8 +-
.../airavata/model/data/replica/ttypes.py | 134 +-
.../airavata/model/dbevent/__init__.py | 5 +
.../airavata/model/dbevent/constants.py | 8 +-
.../airavata/model/dbevent/ttypes.py | 130 +-
.../airavata/model/experiment/__init__.py | 5 +
.../airavata/model/experiment/constants.py | 8 +-
.../airavata/model/experiment/ttypes.py | 212 +-
.../airavata/model/group/__init__.py | 5 +
.../airavata/model/group/constants.py | 8 +-
.../airavata/model/group/ttypes.py | 64 +-
.../airavata/model/job/__init__.py | 5 +
.../airavata/model/job/constants.py | 8 +-
.../airavata/model/job/ttypes.py | 36 +-
.../airavata/model/messaging/__init__.py | 4 +
.../airavata/model/messaging/event/__init__.py | 5 +
.../airavata/model/messaging/event/constants.py | 8 +-
.../airavata/model/messaging/event/ttypes.py | 301 +-
.../airavata/model/process/__init__.py | 5 +
.../airavata/model/process/constants.py | 8 +-
.../airavata/model/process/ttypes.py | 76 +-
.../airavata/model/scheduling/__init__.py | 5 +
.../airavata/model/scheduling/constants.py | 8 +-
.../airavata/model/scheduling/ttypes.py | 42 +-
.../airavata/model/security/__init__.py | 5 +
.../airavata/model/security/constants.py | 8 +-
.../airavata/model/security/ttypes.py | 16 +-
.../airavata/model/sharing/__init__.py | 5 +
.../airavata/model/sharing/constants.py | 8 +-
.../airavata/model/sharing/ttypes.py | 362 +-
.../airavata/model/status/__init__.py | 5 +
.../airavata/model/status/constants.py | 8 +-
.../airavata/model/status/ttypes.py | 234 +-
.../airavata/model/task/__init__.py | 5 +
.../airavata/model/task/constants.py | 8 +-
.../airavata/model/task/ttypes.py | 182 +-
.../airavata/model/tenant/__init__.py | 5 +
.../airavata/model/tenant/constants.py | 8 +-
.../airavata/model/tenant/ttypes.py | 94 +-
.../airavata-python-sdk/airavata/model/ttypes.py | 8 +-
.../airavata/model/user/__init__.py | 5 +
.../airavata/model/user/constants.py | 8 +-
.../airavata/model/user/ttypes.py | 269 +-
.../airavata/model/workflow/__init__.py | 5 +
.../airavata/model/workflow/constants.py | 8 +-
.../airavata/model/workflow/ttypes.py | 348 +-
.../airavata/model/workspace/__init__.py | 5 +
.../airavata/model/workspace/constants.py | 8 +-
.../airavata/model/workspace/ttypes.py | 192 +-
.../airavata/service/__init__.py | 4 +
.../airavata/service/profile/__init__.py | 5 +
.../airavata/service/profile/constants.py | 8 +-
.../service/profile/groupmanager/__init__.py | 4 +
.../groupmanager/cpi/GroupManagerService-remote | 4 +-
.../groupmanager/cpi/GroupManagerService.py | 360 +-
.../service/profile/groupmanager/cpi/__init__.py | 5 +
.../service/profile/groupmanager/cpi/constants.py | 8 +-
.../profile/groupmanager/cpi/error/__init__.py | 5 +
.../profile/groupmanager/cpi/error/constants.py | 8 +-
.../profile/groupmanager/cpi/error/ttypes.py | 12 +-
.../service/profile/groupmanager/cpi/ttypes.py | 8 +-
.../airavata/service/profile/iam/__init__.py | 4 +
.../airavata/service/profile/iam/admin/__init__.py | 4 +
.../service/profile/iam/admin/services/__init__.py | 4 +
.../iam/admin/services/cpi/IamAdminServices-remote | 4 +-
.../iam/admin/services/cpi/IamAdminServices.py | 416 +-
.../profile/iam/admin/services/cpi/__init__.py | 5 +
.../profile/iam/admin/services/cpi/constants.py | 8 +-
.../iam/admin/services/cpi/error/__init__.py | 5 +
.../iam/admin/services/cpi/error/constants.py | 8 +-
.../profile/iam/admin/services/cpi/error/ttypes.py | 12 +-
.../profile/iam/admin/services/cpi/ttypes.py | 8 +-
.../airavata/service/profile/tenant/__init__.py | 4 +
.../profile/tenant/cpi/TenantProfileService-remote | 4 +-
.../profile/tenant/cpi/TenantProfileService.py | 190 +-
.../service/profile/tenant/cpi/__init__.py | 5 +
.../service/profile/tenant/cpi/constants.py | 8 +-
.../service/profile/tenant/cpi/error/__init__.py | 5 +
.../service/profile/tenant/cpi/error/constants.py | 8 +-
.../service/profile/tenant/cpi/error/ttypes.py | 12 +-
.../airavata/service/profile/tenant/cpi/ttypes.py | 8 +-
.../airavata/service/profile/ttypes.py | 8 +-
.../airavata/service/profile/user/__init__.py | 4 +
.../profile/user/cpi/UserProfileService-remote | 4 +-
.../service/profile/user/cpi/UserProfileService.py | 198 +-
.../airavata/service/profile/user/cpi/__init__.py | 5 +
.../airavata/service/profile/user/cpi/constants.py | 8 +-
.../service/profile/user/cpi/error/__init__.py | 5 +
.../service/profile/user/cpi/error/constants.py | 8 +-
.../service/profile/user/cpi/error/ttypes.py | 12 +-
.../airavata/service/profile/user/cpi/ttypes.py | 8 +-
.../airavata_experiments/__init__.py | 180 +
.../airavata_experiments/airavata.py | 800 +++
.../auth}/__init__.py | 15 +
.../airavata_experiments/auth/device_auth.py | 186 +
.../airavata_experiments/base.py | 133 +
.../utils => airavata_experiments/md}/__init__.py | 10 +
.../airavata_experiments/md/applications.py | 258 +
.../airavata_experiments/plan.py | 199 +
.../airavata_experiments/runtime.py | 274 +
.../airavata_experiments/scripter.py | 144 +
.../airavata_experiments/sftp.py | 122 +
.../airavata_experiments/task.py | 106 +
.../airavata_sdk/clients/api_server_client.py | 7231 +-------------------
.../clients/credential_store_client.py | 47 +-
.../airavata_sdk/clients/file_handling_client.py | 18 +-
.../airavata_sdk/clients/group_manager_client.py | 216 +-
.../airavata_sdk/clients/iam_admin_client.py | 239 +-
.../airavata_sdk/clients/keycloak_token_fetcher.py | 100 +-
.../clients/sftp_file_handling_client.py | 23 +-
.../clients/sharing_registry_client.py | 916 +--
.../airavata_sdk/clients/tenant_profile_client.py | 126 +-
.../airavata_sdk/clients/user_profile_client.py | 137 +-
.../clients/utils/api_server_client_util.py | 41 +-
.../clients/utils/data_model_creation_util.py | 93 +-
.../clients/utils/experiment_handler_util.py | 79 +-
.../samples/api_server_client_samples.py | 17 +-
.../samples/create_launch_echo_experiment.py | 43 +-
.../samples/create_launch_gaussian_experiment.py | 50 +-
.../airavata_sdk/samples/file_utils.py | 1 +
.../samples/group_manager_client_samples.py | 8 +-
.../samples/iam_admin_client_samples.py | 5 +-
.../airavata_sdk/samples/metadata_fetcher.py | 16 +-
.../samples/sharing_registry_client_samples.py | 6 +-
.../samples/tenant_profile_client_samples.py | 5 +-
.../samples/user_profile_client_samples.py | 8 +-
.../airavata_sdk/transport/settings.py | 56 +-
.../airavata_sdk/transport/utils.py | 317 +-
.../airavata-python-sdk/pyproject.toml | 13 +-
.../airavata-python-sdk/requirements.txt | 11 -
.../airavata-python-sdk/setup.cfg | 26 -
.../airavata-python-sdk/setup.py | 22 -
modules/agent-framework/airavata-agent/Makefile | 4 +-
modules/agent-framework/airavata-agent/agent.go | 90 +-
.../deployments/agent-cerebrum/Makefile | 2 +-
.../deployments/agent-cyberfaces/Makefile | 2 +-
.../deployments/agent-i-guide/Makefile | 2 +-
.../agent-framework/deployments/agent-md/Makefile | 2 +-
.../deployments/agent-sample/Dockerfile | 2 -
.../deployments/agent-sample/Makefile | 2 +-
.../app-scripts/anvil_agent.sh} | 0
.../deployments/app-scripts/expanse_af2.sh | 133 +
.../app-scripts/expanse_agent.sh} | 11 +-
.../deployments/app-scripts/expanse_namd.sh | 231 +
.../jupyterhub/data/1_experiment_sdk.ipynb | 38 +-
.../data/{1_experiment_sdk.ipynb => smd_cpu.ipynb} | 24 +-
.../data/{1_experiment_sdk.ipynb => smd_gpu.ipynb} | 34 +-
.../component-cpis/generate-cpi-stubs.sh | 2 +-
.../generate-thrift-stubs.sh | 10 +-
.../service-cpis/generate-service-cpi-stubs.sh | 2 +-
232 files changed, 11890 insertions(+), 15402 deletions(-)
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/__init__.py
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/airavata.py
copy
airavata-api/airavata-client-sdks/airavata-python-sdk/{airavata_sdk/clients/utils
=> airavata_experiments/auth}/__init__.py (75%)
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/auth/device_auth.py
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/base.py
copy
airavata-api/airavata-client-sdks/airavata-python-sdk/{airavata_sdk/clients/utils
=> airavata_experiments/md}/__init__.py (82%)
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/md/applications.py
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/plan.py
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/runtime.py
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/scripter.py
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/sftp.py
create mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/airavata_experiments/task.py
delete mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/requirements.txt
delete mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/setup.cfg
delete mode 100644
airavata-api/airavata-client-sdks/airavata-python-sdk/setup.py
copy modules/agent-framework/{airavata-agent/agent.sh =>
deployments/app-scripts/anvil_agent.sh} (100%)
create mode 100755
modules/agent-framework/deployments/app-scripts/expanse_af2.sh
rename modules/agent-framework/{airavata-agent/agent.sh =>
deployments/app-scripts/expanse_agent.sh} (77%)
create mode 100755
modules/agent-framework/deployments/app-scripts/expanse_namd.sh
copy
modules/agent-framework/deployments/jupyterhub/data/{1_experiment_sdk.ipynb =>
smd_cpu.ipynb} (94%)
copy
modules/agent-framework/deployments/jupyterhub/data/{1_experiment_sdk.ipynb =>
smd_gpu.ipynb} (93%)
