[GitHub] [airavata] DImuthuUpe commented on pull request #276: Fetching intermediate outputs for an running experiment
DImuthuUpe commented on pull request #276: URL: https://github.com/apache/airavata/pull/276#issuecomment-1005938283 @machristie good catch. I will update the code. Thanks for this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@airavata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[Airavata] Overriding and removing SLURM script parameters
Hi All, With the introduction of GPU queues in various clusters, traditional SLURM scripts need several changes and those changes are not consistent with each cluster. Because of that, we need to override SLURM script parameters depending on the cluster configuration. This PR [1] provides the flexibility for users to either override existing SLURM parameters or remove them from the scripts. For example, GPU queues on Expanse do not accept scripts with #SBATCH -n parameter and you can remove it by adding a pre-job command RM #SBATCH -n. I have rebased and merged this as it was extensively tested on Ultrascan environment. [1] https://github.com/apache/airavata/pull/277 Thanks Dimuthu
[GitHub] [airavata] DImuthuUpe merged pull request #277: Bringing ultrascan job script optimization logic to master
DImuthuUpe merged pull request #277: URL: https://github.com/apache/airavata/pull/277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@airavata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airavata] machristie commented on pull request #276: Fetching intermediate outputs for an running experiment
machristie commented on pull request #276: URL: https://github.com/apache/airavata/pull/276#issuecomment-1005753766 @DImuthuUpe I think I've figure it out. Gaussian-Application-Output has type URI_COLLECTION (which might be a misconfiguration, but we'll leave that aside for now). So when the output objects are saved, AiravataTask is calling saveExperimentOutputCollection https://github.com/apache/airavata/blob/AIRAVATA-3549/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/task/AiravataTask.java#L306-L329. I think this method also needs to be updated like the way saveExperimentOutput was updated to save outputs to either the experiment or process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@airavata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airavata] machristie commented on pull request #276: Fetching intermediate outputs for an running experiment
machristie commented on pull request #276:
URL: https://github.com/apache/airavata/pull/276#issuecomment-1005746880
Yeah you're right @DImuthuUpe , it should be setting a data product URI. It
doesn't seem to be and I can't figure out why. I'm looking at your test run
yesterday, experiment id
`Clone_of_Clone_of_Clone_of_Clone_of_Clone_of_Gauss_5052a8ec-0b1f-4829-ae33-af81f35e1d32`.
It has two processes the job submission one
`PROCESS_59024a0d-4066-41b5-953a-caa2c8a59fc9` and the intermediate output
fetching one `PROCESS_7ff674cb-d9ba-4805-be06-4bf6f5dfbef6`. In the latter,
there is just the one process output, for `Gaussian-Application-Output` and
it's value is `*.log`.
```
...
"processOutputs": [
{
"name": "Gaussian-Application-Output",
"value": "*.log",
"type": 4,
"applicationArgument": "",
"isRequired": false,
"requiredToAddedToCommandLine": false,
"dataMovement": true,
"location": null,
"searchQuery": null,
"outputStreaming": false,
"storageResourceId": null,
"metaData": null
}
],
...
```
For reference, I'm seeing this through the django.seagrid.org REST API:
https://django.seagrid.org/api/experiments/Clone_of_Clone_of_Clone_of_Clone_of_Clone_of_Gauss_5052a8ec-0b1f-4829-ae33-af81f35e1d32/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@airavata.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
