I found more culprits in EXPERIMENT_SUMMARY, but:
MariaDB [Airavata_production_expcatalog]> select * from EXPERIMENT_SUMMARY
where EXECUTION_ID like '%snake%';
+---------------------------------------------------+-----------------------------------------------------+--------------------------+------------------+---------------------------------------------------+-----------------+---------------------+-------------+-----------+----------------------------------------------------------------+----------------------+
| EXPERIMENT_ID | PROJECT_ID
| GATEWAY_ID | USER_NAME |
EXECUTION_ID | EXPERIMENT_NAME |
CREATION_TIME | DESCRIPTION | STATE | RESOURCE_HOST_ID
| TIME_OF_STATE_CHANGE |
+---------------------------------------------------+-----------------------------------------------------+--------------------------+------------------+---------------------------------------------------+-----------------+---------------------+-------------+-----------+----------------------------------------------------------------+----------------------+
| 12345_f870dd35-e6d4-4053-a92b-302820dd129e |
Project1_3c40da06-4bef-4326-8097-ccf44859285a | usda_ars_science_gateway
| jarett.deangelis | ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 | 12345
| 2016-11-02 10:22:14 | | COMPLETED |
login.scinet.ars.usda.gov_b43d44c7-6d73-4321-b88d-62f3e7768d0c | 2016-11-02
10:22:52 |
| 2345_aa3b31ea-6f2c-48e4-8b15-6a39ef23330c |
Project1_3c40da06-4bef-4326-8097-ccf44859285a | usda_ars_science_gateway
| jarett.deangelis | ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 | 2345
| 2016-11-02 10:24:46 | | COMPLETED |
login.scinet.ars.usda.gov_b43d44c7-6d73-4321-b88d-62f3e7768d0c | 2016-11-02
10:25:24 |
| asdfasdfasdf_6727643c-1749-45df-86c4-1a9ecb2fd1a3 |
Project1_3c40da06-4bef-4326-8097-ccf44859285a | usda_ars_science_gateway
| jarett.deangelis | ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 |
asdfasdfasdf | 2016-10-21 16:08:33 | | LAUNCHED |
login.scinet.ars.usda.gov_b43d44c7-6d73-4321-b88d-62f3e7768d0c | 2016-10-21
16:08:44 |
| qwerty_68f5e215-d998-47c1-b702-c66346889689 |
DefaultProject_dfd1a414-dbdf-41c8-ad9f-6092af16cf3d | usda_ars_science_gateway
| jarett.deangelis | ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 | qwerty
| 2016-10-31 10:57:09 | asdf | LAUNCHED |
login.scinet.ars.usda.gov_b43d44c7-6d73-4321-b88d-62f3e7768d0c | 2016-10-31
10:57:09 |
| qwerty_68f5e215-d998-47c1-b702-c66346889689 |
DefaultProject_dfd1a414-dbdf-41c8-ad9f-6092af16cf3d | usda_ars_science_gateway
| jarett.deangelis | ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 | qwerty
| 2016-10-31 10:57:09 | asdf | CREATED |
login.scinet.ars.usda.gov_b43d44c7-6d73-4321-b88d-62f3e7768d0c | 2016-10-31
10:57:09 |
+---------------------------------------------------+-----------------------------------------------------+--------------------------+------------------+---------------------------------------------------+-----------------+---------------------+-------------+-----------+----------------------------------------------------------------+----------------------+
5 rows in set (0.01 sec)
MariaDB [Airavata_production_expcatalog]> delete from EXPERIMENT_SUMMARY where
EXECUTION_ID like '%snake%';
ERROR 1288 (HY000): The target table EXPERIMENT_SUMMARY of the DELETE is not
updatable
Not sure what I can do with that.
Jarett
> On Dec 28, 2016, at 5:24 PM, Suresh Marru <[email protected]> wrote:
>
> Hi Jarett,
>
> Can you remove the entries of ls_snakemake in application catalog database as
> well? I believe Eroma was able to reproduce the issue and we will work on a
> fix. Sorry for the troubles.
>
> Suresh
>
>> On Dec 28, 2016, at 5:13 PM, Jarett DeAngelis <[email protected]> wrote:
>>
>> So, I keep getting this popping up in my log, and whenever it does, the
>> middleware grinds to a halt and will not process any more jobs through:
>>
>> 2016-12-28 14:56:03,058 [pool-3-thread-49] ERROR
>> org.apache.airavata.api.server.handler.AiravataServerHandler -
>> ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03
>> org.apache.airavata.registry.cpi.AppCatalogException:
>> javax.persistence.NoResultException: Query "SELECT p FROM
>> ApplicationInterface p WHERE p.interfaceID =:param0" selected no result, but
>> expected unique result.
>>
>> This “ls_snakemake” job is months old and is associated with an interface,
>> module and deployment which no longer exist, which is why this error is
>> getting thrown. What I don’t understand is what is “reminding” Airavata of
>> this job and causing it to get hung up. Last week, I found references to it
>> in the experiment catalog database in the EXPERIMENT table, and deleting
>> these allowed Airavata to continue. Now it is happening again and the
>> references are gone. How do I eliminate all references to this job from all
>> of Airavata such that it stops getting hung up on this phantom job?
>>
>> Thanks,
>> Jarett
>
