[airavata-django-portal-commons] branch main updated: Adding clarifying note about django-webpack-loader versions
This is an automated email from the ASF dual-hosted git repository. machristie pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airavata-django-portal-commons.git The following commit(s) were added to refs/heads/main by this push: new 1f9e796 Adding clarifying note about django-webpack-loader versions 1f9e796 is described below commit 1f9e796ba08ce614475c4302e2a05501f42f0993 Author: Marcus Christie AuthorDate: Thu Feb 2 16:56:29 2023 -0500 Adding clarifying note about django-webpack-loader versions --- README.md | 7 +++ 1 file changed, 7 insertions(+) diff --git a/README.md b/README.md index 4493800..282c112 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,13 @@ settings_module = sys.modules[__name__] dynamic_apps.merge_settings(settings_module) ``` +- Note: if the dynamic Django app uses WEBPACK_LOADER, keep in mind that it is + important that the version of + [django-webpack-loader](https://github.com/django-webpack/django-webpack-loader) + and the version of webpack-bundle-tracker be compatible. If you're using + django-webpack-loader prior to version 1.0 then a known good pair of versions + is django-webpack-loader==0.6.0 and webpack-bundle-tracker==0.4.3. + 2. Also add `'airavata_django_portal_commons.dynamic_apps.context_processors.custom_app_registry'` to the context_processors list:
[airavata-mft] 04/05: Adding missing dependencies to Agent
This is an automated email from the ASF dual-hosted git repository.
dimuthuupe pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airavata-mft.git
commit f4db19e06b6e221de47170d425dd3c685aebee09
Author: Dimuthu Wannipurage
AuthorDate: Thu Feb 2 13:23:33 2023 -0500
Adding missing dependencies to Agent
---
agent/pom.xml | 5 +
agent/service/pom.xml | 9 +++--
2 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/agent/pom.xml b/agent/pom.xml
index 676936a..8eff88b 100644
--- a/agent/pom.xml
+++ b/agent/pom.xml
@@ -43,6 +43,11 @@
protobuf-java
${protobuf.java}
+
+com.google.protobuf
+protobuf-java-util
+${protobuf.java}
+
diff --git a/agent/service/pom.xml b/agent/service/pom.xml
index 0d1a5cb..c4bc494 100644
--- a/agent/service/pom.xml
+++ b/agent/service/pom.xml
@@ -49,6 +49,11 @@
netty-tcnative-boringssl-static
2.0.54.Final
+
+com.google.guava
+guava
+${guava.version}
+
org.apache.airavata
mft-scp-transport
@@ -163,7 +168,7 @@
-
+
11
[airavata-mft] 03/05: Avoiding database reset when services restart
This is an automated email from the ASF dual-hosted git repository. dimuthuupe pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airavata-mft.git commit ea9a03e34a991c3abc308e91f3182aebe9cdd46f Author: Dimuthu Wannipurage AuthorDate: Thu Feb 2 13:22:23 2023 -0500 Avoiding database reset when services restart --- standalone-service/src/main/resources/application.properties | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/standalone-service/src/main/resources/application.properties b/standalone-service/src/main/resources/application.properties index 95a221f..e22d45d 100644 --- a/standalone-service/src/main/resources/application.properties +++ b/standalone-service/src/main/resources/application.properties @@ -16,11 +16,13 @@ # spring.main.allow-bean-definition-overriding=true -spring.datasource.url=jdbc:h2:~/mft_db;DB_CLOSE_ON_EXIT=FALSE +spring.datasource.url=jdbc:h2:~/mft_db;DB_CLOSE_ON_EXIT=FALSE;IFEXISTS=TRUE;DB_CLOSE_DELAY=-1; +spring.jpa.hibernate.ddl-auto=update consul.host=localhost consul.port=8500 resource.service.host=localhost resource.service.port=7003 secret.service.host=localhost -secret.service.port=7003 \ No newline at end of file +secret.service.port=7003 +agent.id=local-agent \ No newline at end of file
[airavata-mft] 01/05: Prioritizing local agent if no agent is specified in the request
This is an automated email from the ASF dual-hosted git repository.
dimuthuupe pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airavata-mft.git
commit 5161d90f2d2800dc10cbb71a45139de3187eccae
Author: Dimuthu Wannipurage
AuthorDate: Thu Feb 2 13:16:58 2023 -0500
Prioritizing local agent if no agent is specified in the request
---
.../java/org/apache/airavata/mft/api/handler/MFTApiHandler.java | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git
a/api/service/src/main/java/org/apache/airavata/mft/api/handler/MFTApiHandler.java
b/api/service/src/main/java/org/apache/airavata/mft/api/handler/MFTApiHandler.java
index 69c7262..371bab3 100644
---
a/api/service/src/main/java/org/apache/airavata/mft/api/handler/MFTApiHandler.java
+++
b/api/service/src/main/java/org/apache/airavata/mft/api/handler/MFTApiHandler.java
@@ -512,7 +512,11 @@ public class MFTApiHandler extends
MFTTransferServiceGrpc.MFTTransferServiceImpl
if (liveAgentIds.isEmpty()) {
throw new Exception("No agent is available to perform the
operation");
}
-targetAgent = liveAgentIds.get(0);
+if (liveAgentIds.stream().anyMatch(id ->
id.equals("local-agent"))) {
+targetAgent = "local-agent";
+} else {
+targetAgent = liveAgentIds.get(0);
+}
logger.info("Using agent {} for processing the operation",
targetAgent);
} else {
Optional agentInfo =
mftConsulClient.getAgentInfo(targetAgent);
[airavata-mft] 02/05: Ignoring failed Agent instances when reusing
This is an automated email from the ASF dual-hosted git repository.
dimuthuupe pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airavata-mft.git
commit 1bc3594d1c63b07fbcd4f50f9bb297e9aa2b2d5d
Author: Dimuthu Wannipurage
AuthorDate: Thu Feb 2 13:21:45 2023 -0500
Ignoring failed Agent instances when reusing
---
.../mft/controller/spawner/AgentOrchestrator.java | 8 +--
.../mft/controller/spawner/AgentSpawner.java | 2 +-
.../mft/controller/spawner/EC2AgentSpawner.java| 63 +++---
3 files changed, 49 insertions(+), 24 deletions(-)
diff --git
a/controller/src/main/java/org/apache/airavata/mft/controller/spawner/AgentOrchestrator.java
b/controller/src/main/java/org/apache/airavata/mft/controller/spawner/AgentOrchestrator.java
index cb58002..ceceab6 100644
---
a/controller/src/main/java/org/apache/airavata/mft/controller/spawner/AgentOrchestrator.java
+++
b/controller/src/main/java/org/apache/airavata/mft/controller/spawner/AgentOrchestrator.java
@@ -32,7 +32,7 @@ public class AgentOrchestrator {
private static final Logger logger =
LoggerFactory.getLogger(AgentOrchestrator.class);
-private final int SPAWNER_MAX_IDLE_SECONDS = 30;
+private final int SPAWNER_MAX_IDLE_SECONDS = 30; // TODO Externalize this
private class TransferInfo {
private final String transferId;
@@ -145,7 +145,7 @@ public class AgentOrchestrator {
logger.info("Removing consul key {}",
transferInfo.consulKey);
transferDispatcher.getMftConsulClient().getKvClient().deleteKey(transferInfo.consulKey);
logger.info("Terminating the spawner");
-metadata.spawner.terminate();
+metadata.spawner.terminate(true);
launchedSpawnersMap.remove(key);
} finally {
metadata.transferInfos.remove(agentTransferId);
@@ -169,7 +169,7 @@ public class AgentOrchestrator {
} catch (Exception e) {
logger.info("Killing spawner with key {} as
the agent is not responding and inactive for {} seconds",
key, SPAWNER_MAX_IDLE_SECONDS);
-metadata.spawner.terminate();
+metadata.spawner.terminate(false);
launchedSpawnersMap.remove(key);
return;
}
@@ -181,7 +181,7 @@ public class AgentOrchestrator {
logger.info("Killing spawner with key {}
as all files were transferred and the agent" +
" is inactive for {}
seconds",
key, SPAWNER_MAX_IDLE_SECONDS);
-metadata.spawner.terminate();
+metadata.spawner.terminate(false);
launchedSpawnersMap.remove(key);
}
}
diff --git
a/controller/src/main/java/org/apache/airavata/mft/controller/spawner/AgentSpawner.java
b/controller/src/main/java/org/apache/airavata/mft/controller/spawner/AgentSpawner.java
index a8a733a..3dadf4d 100644
---
a/controller/src/main/java/org/apache/airavata/mft/controller/spawner/AgentSpawner.java
+++
b/controller/src/main/java/org/apache/airavata/mft/controller/spawner/AgentSpawner.java
@@ -33,7 +33,7 @@ public abstract class AgentSpawner {
public abstract void launch();
public abstract Future getLaunchState();
-public abstract void terminate();
+public abstract void terminate(boolean failed);
public abstract Future getTerminateState();
}
diff --git
a/controller/src/main/java/org/apache/airavata/mft/controller/spawner/EC2AgentSpawner.java
b/controller/src/main/java/org/apache/airavata/mft/controller/spawner/EC2AgentSpawner.java
index 0ac8094..30cf81e 100644
---
a/controller/src/main/java/org/apache/airavata/mft/controller/spawner/EC2AgentSpawner.java
+++
b/controller/src/main/java/org/apache/airavata/mft/controller/spawner/EC2AgentSpawner.java
@@ -23,6 +23,7 @@ import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.ec2.AmazonEC2;
import com.amazonaws.services.ec2.AmazonEC2ClientBuilder;
import com.amazonaws.services.ec2.model.*;
+import com.amazonaws.services.lightsail.model.CreateInstanceSnapshotRequest;
import org.apache.airavata.mft.agent.stub.SecretWrapper;
import org.apache.airavata.mft.agent.stub.StorageWrapper;
import org.slf4j.Logger;
@@ -109,6 +110,20 @@ public class EC2AgentSpawner extends AgentSpawner {
.withCredentials(new
AWSStaticCredentialsProvider(awsCreds))
.build();
[airavata-mft] 05/05: Refactoring Swift transport to support auth V3
This is an automated email from the ASF dual-hosted git repository.
dimuthuupe pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airavata-mft.git
commit 8aa069fc99cf66999e2a79d7cedfb7cc5ffe7bf0
Author: Dimuthu Wannipurage
AuthorDate: Thu Feb 2 13:25:46 2023 -0500
Refactoring Swift transport to support auth V3
---
python-cli/README.md | 2 +-
.../mft_cli/airavata_mft_cli/storage/__init__.py | 3 +
.../mft_cli/airavata_mft_cli/storage/swift.py | 82 +
python-cli/mft_cli/pyproject.toml | 3 +-
python-sdk/setup.cfg | 8 +-
python-sdk/src/airavata_mft_sdk/mft_client.py | 4 +
.../airavata_mft_sdk/swift/SwiftCredential_pb2.py | 58 -
.../src/airavata_mft_sdk/swift/SwiftStorage_pb2.py | 36 +++---
.../backend/sql/entity/SwiftStorageEntity.java | 21
.../stub/src/main/proto/swift/SwiftStorage.proto | 6 -
.../server/backend/sql/SQLSecretBackend.java | 79 ++--
.../sql/entity/swift/SwiftSecretEntity.java| 14 ++-
...retEntity.java => SwiftV2AuthSecretEntity.java} | 46 +++
...retEntity.java => SwiftV3AuthSecretEntity.java} | 60 --
...itory.java => SwiftV2AuthSecretRepository.java} | 6 +-
...itory.java => SwiftV3AuthSecretRepository.java} | 6 +-
.../src/main/proto/swift/SwiftCredential.proto | 38 +++---
.../transport/swift/SwiftIncomingConnector.java| 32 +
.../transport/swift/SwiftMetadataCollector.java| 133 ++---
.../transport/swift/SwiftOutgoingConnector.java| 29 +
.../airavata/mft/transport/swift/SwiftUtil.java| 68 +++
21 files changed, 423 insertions(+), 311 deletions(-)
diff --git a/python-cli/README.md b/python-cli/README.md
index b941fe5..b076417 100644
--- a/python-cli/README.md
+++ b/python-cli/README.md
@@ -37,7 +37,7 @@ Install dependencies
```
pip install grpcio==1.46.3
pip install grpcio-tools==1.46.3
-pip install airavata_mft_sdk==0.0.1-alpha21
+pip install airavata_mft_sdk==0.0.1-alpha24
```
Build the binary
diff --git a/python-cli/mft_cli/airavata_mft_cli/storage/__init__.py
b/python-cli/mft_cli/airavata_mft_cli/storage/__init__.py
index e7ba957..c125b96 100644
--- a/python-cli/mft_cli/airavata_mft_cli/storage/__init__.py
+++ b/python-cli/mft_cli/airavata_mft_cli/storage/__init__.py
@@ -21,6 +21,7 @@ from pick import pick
import airavata_mft_cli.storage.s3 as s3
import airavata_mft_cli.storage.azure as azure
import airavata_mft_cli.storage.gcs as gcs
+import airavata_mft_cli.storage.swift as swift
from airavata_mft_sdk import mft_client
from airavata_mft_sdk.common import StorageCommon_pb2
from rich.console import Console
@@ -39,6 +40,8 @@ def add_storage():
azure.handle_add_storage()
elif option == "Google Cloud Storage (GCS)":
gcs.handle_add_storage()
+elif option == "Openstack SWIFT":
+swift.handle_add_storage()
@app.command("list")
def list_storage():
diff --git a/python-cli/mft_cli/airavata_mft_cli/storage/swift.py
b/python-cli/mft_cli/airavata_mft_cli/storage/swift.py
new file mode 100644
index 000..0bdf5fc
--- /dev/null
+++ b/python-cli/mft_cli/airavata_mft_cli/storage/swift.py
@@ -0,0 +1,82 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+from rich import print
+from pick import pick
+import typer
+from airavata_mft_sdk import mft_client
+from airavata_mft_sdk.swift import SwiftCredential_pb2
+from airavata_mft_sdk.swift import SwiftStorage_pb2
+from airavata_mft_sdk.common import StorageCommon_pb2
+
+def handle_add_storage():
+
+options = ["v3", "v2" ]
+option, index = pick(options, "Select Keystone Auth Version",
indicator="=>")
+
+secret_create_req= SwiftCredential_pb2.SwiftSecretCreateRequest()
+if (option == "v3"):
+
+user_name = typer.prompt("User Name")
+password = typer.prompt("Password")
+tenant_name = typer.prompt("Tenant Name")
+project_domain = typer.prompt("Project Domain Name", "Default")
+user_domain = typer.prompt("User Domain Name", "Default")
+v3_sec =
SwiftCredential_pb2.SwiftV3AuthSecret(userDomainName=user_domain,
[airavata-mft] branch master updated (d677a59 -> 8aa069f)
This is an automated email from the ASF dual-hosted git repository.
dimuthuupe pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/airavata-mft.git
from d677a59 Removing samples as they are not maintained anymore
new 5161d90 Prioritizing local agent if no agent is specified in the
request
new 1bc3594 Ignoring failed Agent instances when reusing
new ea9a03e Avoiding database reset when services restart
new f4db19e Adding missing dependencies to Agent
new 8aa069f Refactoring Swift transport to support auth V3
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
agent/pom.xml | 5 +
agent/service/pom.xml | 9 +-
.../airavata/mft/api/handler/MFTApiHandler.java| 6 +-
.../mft/controller/spawner/AgentOrchestrator.java | 8 +-
.../mft/controller/spawner/AgentSpawner.java | 2 +-
.../mft/controller/spawner/EC2AgentSpawner.java| 63 +++---
python-cli/README.md | 2 +-
.../mft_cli/airavata_mft_cli/storage/__init__.py | 3 +
.../mft_cli/airavata_mft_cli/storage/swift.py | 82 +
python-cli/mft_cli/pyproject.toml | 3 +-
python-sdk/setup.cfg | 8 +-
python-sdk/src/airavata_mft_sdk/mft_client.py | 4 +
.../airavata_mft_sdk/swift/SwiftCredential_pb2.py | 58 -
.../src/airavata_mft_sdk/swift/SwiftStorage_pb2.py | 36 +++---
.../backend/sql/entity/SwiftStorageEntity.java | 21
.../stub/src/main/proto/swift/SwiftStorage.proto | 6 -
.../server/backend/sql/SQLSecretBackend.java | 79 ++--
.../swift/SwiftAuthCredentialSecretEntity.java | 65 --
.../sql/entity/swift/SwiftSecretEntity.java| 14 ++-
...retEntity.java => SwiftV2AuthSecretEntity.java} | 46 +++
.../SwiftV3AuthSecretEntity.java} | 40 ++-
...itory.java => SwiftV2AuthSecretRepository.java} | 6 +-
...itory.java => SwiftV3AuthSecretRepository.java} | 6 +-
.../src/main/proto/swift/SwiftCredential.proto | 38 +++---
.../src/main/resources/application.properties | 6 +-
.../transport/swift/SwiftIncomingConnector.java| 32 +
.../transport/swift/SwiftMetadataCollector.java| 133 ++---
.../transport/swift/SwiftOutgoingConnector.java| 29 +
.../airavata/mft/transport/swift/SwiftUtil.java| 68 +++
29 files changed, 483 insertions(+), 395 deletions(-)
create mode 100644 python-cli/mft_cli/airavata_mft_cli/storage/swift.py
delete mode 100644
services/secret-service/server/src/main/java/org/apache/airavata/mft/secret/server/backend/sql/entity/swift/SwiftAuthCredentialSecretEntity.java
rename
services/secret-service/server/src/main/java/org/apache/airavata/mft/secret/server/backend/sql/entity/swift/{SwiftPasswordSecretEntity.java
=> SwiftV2AuthSecretEntity.java} (74%)
copy
services/secret-service/server/src/main/java/org/apache/airavata/mft/secret/server/backend/sql/entity/{ODataSecretEntity.java
=> swift/SwiftV3AuthSecretEntity.java} (65%)
rename
services/secret-service/server/src/main/java/org/apache/airavata/mft/secret/server/backend/sql/repository/swift/{SwiftAuthCredentialSecretRepository.java
=> SwiftV2AuthSecretRepository.java} (81%)
rename
services/secret-service/server/src/main/java/org/apache/airavata/mft/secret/server/backend/sql/repository/swift/{SwiftPasswordSecretRepository.java
=> SwiftV3AuthSecretRepository.java} (82%)
create mode 100644
transport/swift-transport/src/main/java/org/apache/airavata/mft/transport/swift/SwiftUtil.java
[airavata-mft] branch master updated: Removing samples as they are not maintained anymore
This is an automated email from the ASF dual-hosted git repository. dimuthuupe pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airavata-mft.git The following commit(s) were added to refs/heads/master by this push: new d677a59 Removing samples as they are not maintained anymore d677a59 is described below commit d677a591260da99bfbb20cbd31632d3758340e3b Author: Dimuthu Wannipurage AuthorDate: Thu Feb 2 13:15:54 2023 -0500 Removing samples as they are not maintained anymore --- python-sdk/samples/README.md | 29 - python-sdk/samples/sample.py | 42 -- 2 files changed, 71 deletions(-) diff --git a/python-sdk/samples/README.md b/python-sdk/samples/README.md deleted file mode 100644 index 9065b23..000 --- a/python-sdk/samples/README.md +++ /dev/null @@ -1,29 +0,0 @@ - - -### Instructions to run samples - -* Start MFT Components -* Run following commands -``` -virtualenv -p python3 env -source env/bin/activate -pip install airavata-mft-sdk -python3 sample.py -``` \ No newline at end of file diff --git a/python-sdk/samples/sample.py b/python-sdk/samples/sample.py deleted file mode 100644 index 6c75454..000 --- a/python-sdk/samples/sample.py +++ /dev/null @@ -1,42 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. -# -from airavata_mft_sdk import mft_client -from airavata_mft_sdk import MFTTransferApi_pb2 -from airavata_mft_sdk.s3 import S3Storage_pb2 - -client = mft_client.MFTClient() - - - -create_request = S3Storage_pb2.S3StorageCreateRequest(bucketName = "bucket", - region = "us-east", - storageId = "some_id", - endpoint = "https://endpoint.url;, - name = "s3-storage") - -print(client.s3_storage_api.createS3Storage(create_request)) - -transfer_request = MFTTransferApi_pb2.TransferApiRequest(sourceResourceId="source_id", - sourceType = "S3", - sourceToken = "source_token", - destinationResourceId = "dest_id", - destinationType = "S3", - destinationToken = "dest_token") -print(client.transfer_api.submitTransfer(transfer_request)) -
[airavata-mft] branch master updated: Removing java cli as python cli replaces it
This is an automated email from the ASF dual-hosted git repository.
dimuthuupe pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airavata-mft.git
The following commit(s) were added to refs/heads/master by this push:
new 65d721a Removing java cli as python cli replaces it
65d721a is described below
commit 65d721a2a8bd5a9d8134f4113106590fba251422
Author: Dimuthu Wannipurage
AuthorDate: Thu Feb 2 13:10:27 2023 -0500
Removing java cli as python cli replaces it
---
command-line/pom.xml | 91 --
.../airavata/mft/command/line/CommandLineUtil.java | 37 ---
.../airavata/mft/command/line/MainRunner.java | 38
.../mft/command/line/sub/gcs/GCSAddSubCommand.java | 93 --
.../command/line/sub/gcs/GCSRemoteSubCommand.java | 106 -
.../mft/command/line/sub/gcs/GCSSubCommand.java| 24 -
.../line/sub/odata/ODataRemoteAddSubCommand.java | 76 ---
.../line/sub/odata/ODataRemoteSubCommand.java | 57 ---
.../command/line/sub/odata/ODataSubCommand.java| 26 -
.../mft/command/line/sub/s3/S3SubCommand.java | 28 --
.../sub/s3/storage/S3StorageAddSubCommand.java | 89 -
.../line/sub/s3/storage/S3StorageSubCommand.java | 69 --
.../command/line/sub/swift/SwiftAddSubCommand.java | 99 ---
.../line/sub/swift/SwiftRemoteSubCommand.java | 24 -
.../command/line/sub/swift/SwiftSubCommand.java| 25 -
.../sub/transfer/SubmitTransferSubCommand.java | 83
.../line/sub/transfer/TransferStateSubCommand.java | 91 --
.../line/sub/transfer/TransferSubCommand.java | 25 -
pom.xml| 8 +-
19 files changed, 7 insertions(+), 1082 deletions(-)
diff --git a/command-line/pom.xml b/command-line/pom.xml
deleted file mode 100644
index 520d5ec..000
--- a/command-line/pom.xml
+++ /dev/null
@@ -1,91 +0,0 @@
-
-
-http://maven.apache.org/POM/4.0.0;
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
-
-airavata-mft
-org.apache.airavata
-0.01-SNAPSHOT
-
-4.0.0
-
-mft-command-line
-
-
-11
-11
-
-
-
-
-org.apache.airavata
-mft-api-client
-0.01-SNAPSHOT
-
-
-org.springframework.boot
-spring-boot-starter
-
-
-
-
-info.picocli
-picocli
-4.6.3
-
-
-
-
-
-
-org.apache.maven.plugins
-maven-assembly-plugin
-3.1.1
-
-
-
-jar-with-dependencies
-
-
-
-
org.apache.airavata.mft.command.line.MainRunner
-
-
-
-
-
-
-make-assembly
-package
-
-single
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git
a/command-line/src/main/java/org/apache/airavata/mft/command/line/CommandLineUtil.java
b/command-line/src/main/java/org/apache/airavata/mft/command/line/CommandLineUtil.java
deleted file mode 100644
index 9a8446c..000
---
a/command-line/src/main/java/org/apache/airavata/mft/command/line/CommandLineUtil.java
+++ /dev/null
@@ -1,37 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.airavata.mft.command.line;
-
-public final class CommandLineUtil {
-
-public static void printTable(int[] columnWidths, String[][] content) {
-for (String[] row : content) {
-for (int i = 0; i < columnWidths.length; i++) {
-
[airavata] branch develop updated: bug fix in data analyzer
This is an automated email from the ASF dual-hosted git repository.
isjarana pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/airavata.git
The following commit(s) were added to refs/heads/develop by this push:
new 8eef0e1433 bug fix in data analyzer
new 2dc66ff72e Merge pull request #384 from isururanawaka/metaschedular
8eef0e1433 is described below
commit 8eef0e1433af29fa1afd90ad0c03c93e0a298397
Author: Isuru Ranawaka
AuthorDate: Thu Feb 2 12:33:07 2023 -0500
bug fix in data analyzer
---
.../metascheduler/metadata/analyzer/impl/DataAnalyzerImpl.java | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git
a/modules/airavata-metascheduler/metadata-analyzer/src/main/java/org/apache/airavata/metascheduler/metadata/analyzer/impl/DataAnalyzerImpl.java
b/modules/airavata-metascheduler/metadata-analyzer/src/main/java/org/apache/airavata/metascheduler/metadata/analyzer/impl/DataAnalyzerImpl.java
index 294c385f4d..53c2948051 100644
---
a/modules/airavata-metascheduler/metadata-analyzer/src/main/java/org/apache/airavata/metascheduler/metadata/analyzer/impl/DataAnalyzerImpl.java
+++
b/modules/airavata-metascheduler/metadata-analyzer/src/main/java/org/apache/airavata/metascheduler/metadata/analyzer/impl/DataAnalyzerImpl.java
@@ -56,10 +56,11 @@ public class DataAnalyzerImpl implements DataAnalyzer {
Map timeDistribution =
client.getAVGTimeDistribution(gateway,15);
-if(!timeDistribution.isEmpty() && timeDistribution.size()==3) {
-LOGGER.info("orch Time: " + timeDistribution.get("orchTime")
+ " queued Time "
-+ timeDistribution.get("queuedTime") + " helix Time "
+ timeDistribution.get("helix"));
+String msg ="";
+for(Map.Entry entry: timeDistribution.entrySet()){
+msg = " avg time "+entry.getKey()+" : "+entry.getValue();
}
+LOGGER.info(msg);
} catch (Exception ex) {
String msg = "Error occurred while executing data analyzer" +
ex.getMessage();
[airavata-data-catalog] branch main updated: Adding the mandatory LICENSE and NOTICE files
This is an automated email from the ASF dual-hosted git repository. smarru pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airavata-data-catalog.git The following commit(s) were added to refs/heads/main by this push: new 0265694 Adding the mandatory LICENSE and NOTICE files 0265694 is described below commit 0265694ad0eaa3098c358e12ac534e20bd6f031d Author: Suresh Marru AuthorDate: Thu Feb 2 11:47:15 2023 -0500 Adding the mandatory LICENSE and NOTICE files --- LICENSE | 202 NOTICE | 5 ++ 2 files changed, 207 insertions(+) diff --git a/LICENSE b/LICENSE new file mode 100644 index 000..7a4a3ea --- /dev/null +++ b/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated
[airavata-mft] branch master updated: Adding NOTICE file
This is an automated email from the ASF dual-hosted git repository. smarru pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airavata-mft.git The following commit(s) were added to refs/heads/master by this push: new 899f3ad Adding NOTICE file 899f3ad is described below commit 899f3adb31c1fa89106da74d2a59ac95f6f73ddc Author: Suresh Marru AuthorDate: Thu Feb 2 11:40:51 2023 -0500 Adding NOTICE file --- NOTICE | 5 + 1 file changed, 5 insertions(+) diff --git a/NOTICE b/NOTICE new file mode 100644 index 000..1204a42 --- /dev/null +++ b/NOTICE @@ -0,0 +1,5 @@ +Apache Airavata MFT +Copyright 2023 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). \ No newline at end of file
[airavata] branch develop updated: Add process
This is an automated email from the ASF dual-hosted git repository.
isjarana pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/airavata.git
The following commit(s) were added to refs/heads/develop by this push:
new 241ad912c1 Add process
new 68f4d3a9d0 Merge pull request #383 from isururanawaka/metaschedular
241ad912c1 is described below
commit 241ad912c1f0e2c4d489219b104d55774c5a8cbc
Author: Isuru Ranawaka
AuthorDate: Thu Feb 2 09:32:19 2023 -0500
Add process
---
.../registry/core/repositories/expcatalog/ProcessRepository.java | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git
a/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/ProcessRepository.java
b/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/ProcessRepository.java
index d8c6c37876..f40cf18201 100644
---
a/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/ProcessRepository.java
+++
b/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/ProcessRepository.java
@@ -35,6 +35,7 @@ import org.dozer.Mapper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import java.math.BigDecimal;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
@@ -205,13 +206,13 @@ public class ProcessRepository extends
ExpCatAbstractRepository helixTimeList =
processRepository.selectWithNativeQuery(QueryConstants.FIND_AVG_TIME_HELIX_NATIVE_QUERY,
gatewayId,String.valueOf(searchTime));
if(orchTimeList.size()>0 && orchTimeList.get(0) != null){
-
timeDistributions.put(DBConstants.MetaData.ORCH_TIME,Double.parseDouble((String)
orchTimeList.get(0)));
+timeDistributions.put(DBConstants.MetaData.ORCH_TIME,
((BigDecimal)orchTimeList.get(0)).doubleValue());
}
if(queueingTimeList.size()>0 && queueingTimeList.get(0) != null){
-
timeDistributions.put(DBConstants.MetaData.QUEUED_TIME,Double.parseDouble((String)
queueingTimeList.get(0)));
+
timeDistributions.put(DBConstants.MetaData.QUEUED_TIME,((BigDecimal)queueingTimeList.get(0)).doubleValue());
}
if(helixTimeList.size()>0 && helixTimeList.get(0) != null){
-
timeDistributions.put(DBConstants.MetaData.HELIX,Double.parseDouble((String)
helixTimeList.get(0)));
+
timeDistributions.put(DBConstants.MetaData.HELIX,((BigDecimal)helixTimeList.get(0)).doubleValue());
}
return timeDistributions;
}
[airavata] branch develop updated: Add process
This is an automated email from the ASF dual-hosted git repository.
isjarana pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/airavata.git
The following commit(s) were added to refs/heads/develop by this push:
new d11eafb182 Add process
new 3a7585548c Merge pull request #382 from isururanawaka/metaschedular
d11eafb182 is described below
commit d11eafb1828ef13437e8e9f5db466f138e754a59
Author: Isuru Ranawaka
AuthorDate: Thu Feb 2 08:47:55 2023 -0500
Add process
---
.../registry/core/repositories/expcatalog/ProcessRepository.java | 7 +++
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git
a/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/ProcessRepository.java
b/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/ProcessRepository.java
index 3116212b7a..d8c6c37876 100644
---
a/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/ProcessRepository.java
+++
b/modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/ProcessRepository.java
@@ -204,17 +204,16 @@ public class ProcessRepository extends
ExpCatAbstractRepository helixTimeList =
processRepository.selectWithNativeQuery(QueryConstants.FIND_AVG_TIME_HELIX_NATIVE_QUERY,
gatewayId,String.valueOf(searchTime));
-if(orchTimeList.size()>0){
+if(orchTimeList.size()>0 && orchTimeList.get(0) != null){
timeDistributions.put(DBConstants.MetaData.ORCH_TIME,Double.parseDouble((String)
orchTimeList.get(0)));
}
-if(queueingTimeList.size()>0){
+if(queueingTimeList.size()>0 && queueingTimeList.get(0) != null){
timeDistributions.put(DBConstants.MetaData.QUEUED_TIME,Double.parseDouble((String)
queueingTimeList.get(0)));
}
-if(helixTimeList.size()>0){
+if(helixTimeList.size()>0 && helixTimeList.get(0) != null){
timeDistributions.put(DBConstants.MetaData.HELIX,Double.parseDouble((String)
helixTimeList.get(0)));
}
return timeDistributions;
}
-
}
