Yeah, I poked through everything that had any references to “param” (which was the prefix I used for parameters in my Snakemake exploration) and deleted all of them in those tables. As of today the gateway is still functioning normally, so I think I’m okay there.
Thanks, Jarett > On Dec 29, 2016, at 10:05 AM, Christie, Marcus Aaron <[email protected]> wrote: > > > Jarett, > > I’m a little out of the loop on the issue you’ve run into, but I’ll chime in > in the hope that I can help. > > EXPERIMENT_SUMMARY is a DB View that joins together the EXPERIMENT and > USER_CONFIGURATION_DATA tables with the LATEST_EXPERIMENT_STATUS view (which > in turn is a self-joined view of the EXPERIMENT_STATUS table). > > So you might want to check also the EXPERIMENT, USER_CONFIGURATION_DATA and > EXPERIMENT_STATUS tables. > > > Marcus > > >> On Dec 28, 2016, at 6:07 PM, Jarett DeAngelis <[email protected] >> <mailto:[email protected]>> wrote: >> >> I found more culprits in EXPERIMENT_SUMMARY, but: >> >> MariaDB [Airavata_production_expcatalog]> select * from EXPERIMENT_SUMMARY >> where EXECUTION_ID like '%snake%'; >> +---------------------------------------------------+-----------------------------------------------------+--------------------------+------------------+---------------------------------------------------+-----------------+---------------------+-------------+-----------+----------------------------------------------------------------+----------------------+ >> | EXPERIMENT_ID | PROJECT_ID >> | GATEWAY_ID | USER_NAME | >> EXECUTION_ID | EXPERIMENT_NAME | >> CREATION_TIME | DESCRIPTION | STATE | RESOURCE_HOST_ID >> | TIME_OF_STATE_CHANGE | >> +---------------------------------------------------+-----------------------------------------------------+--------------------------+------------------+---------------------------------------------------+-----------------+---------------------+-------------+-----------+----------------------------------------------------------------+----------------------+ >> | 12345_f870dd35-e6d4-4053-a92b-302820dd129e | >> Project1_3c40da06-4bef-4326-8097-ccf44859285a | >> usda_ars_science_gateway | jarett.deangelis | >> ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 | 12345 | >> 2016-11-02 10:22:14 | | COMPLETED | login.scinet.ars.usda.gov >> <http://login.scinet.ars.usda.gov/>_b43d44c7-6d73-4321-b88d-62f3e7768d0c | >> 2016-11-02 10:22:52 | >> | 2345_aa3b31ea-6f2c-48e4-8b15-6a39ef23330c | >> Project1_3c40da06-4bef-4326-8097-ccf44859285a | >> usda_ars_science_gateway | jarett.deangelis | >> ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 | 2345 | >> 2016-11-02 10:24:46 | | COMPLETED | login.scinet.ars.usda.gov >> <http://login.scinet.ars.usda.gov/>_b43d44c7-6d73-4321-b88d-62f3e7768d0c | >> 2016-11-02 10:25:24 | >> | asdfasdfasdf_6727643c-1749-45df-86c4-1a9ecb2fd1a3 | >> Project1_3c40da06-4bef-4326-8097-ccf44859285a | >> usda_ars_science_gateway | jarett.deangelis | >> ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 | asdfasdfasdf | >> 2016-10-21 16:08:33 | | LAUNCHED | login.scinet.ars.usda.gov >> <http://login.scinet.ars.usda.gov/>_b43d44c7-6d73-4321-b88d-62f3e7768d0c | >> 2016-10-21 16:08:44 | >> | qwerty_68f5e215-d998-47c1-b702-c66346889689 | >> DefaultProject_dfd1a414-dbdf-41c8-ad9f-6092af16cf3d | >> usda_ars_science_gateway | jarett.deangelis | >> ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 | qwerty | >> 2016-10-31 10:57:09 | asdf | LAUNCHED | login.scinet.ars.usda.gov >> <http://login.scinet.ars.usda.gov/>_b43d44c7-6d73-4321-b88d-62f3e7768d0c | >> 2016-10-31 10:57:09 | >> | qwerty_68f5e215-d998-47c1-b702-c66346889689 | >> DefaultProject_dfd1a414-dbdf-41c8-ad9f-6092af16cf3d | >> usda_ars_science_gateway | jarett.deangelis | >> ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 | qwerty | >> 2016-10-31 10:57:09 | asdf | CREATED | login.scinet.ars.usda.gov >> <http://login.scinet.ars.usda.gov/>_b43d44c7-6d73-4321-b88d-62f3e7768d0c | >> 2016-10-31 10:57:09 | >> +---------------------------------------------------+-----------------------------------------------------+--------------------------+------------------+---------------------------------------------------+-----------------+---------------------+-------------+-----------+----------------------------------------------------------------+----------------------+ >> 5 rows in set (0.01 sec) >> >> MariaDB [Airavata_production_expcatalog]> delete from EXPERIMENT_SUMMARY >> where EXECUTION_ID like '%snake%'; >> ERROR 1288 (HY000): The target table EXPERIMENT_SUMMARY of the DELETE is not >> updatable >> >> >> Not sure what I can do with that. >> >> Jarett >> >>> On Dec 28, 2016, at 5:24 PM, Suresh Marru <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hi Jarett, >>> >>> Can you remove the entries of ls_snakemake in application catalog database >>> as well? I believe Eroma was able to reproduce the issue and we will work >>> on a fix. Sorry for the troubles. >>> >>> Suresh >>> >>>> On Dec 28, 2016, at 5:13 PM, Jarett DeAngelis <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> So, I keep getting this popping up in my log, and whenever it does, the >>>> middleware grinds to a halt and will not process any more jobs through: >>>> >>>> 2016-12-28 14:56:03,058 [pool-3-thread-49] ERROR >>>> org.apache.airavata.api.server.handler.AiravataServerHandler - >>>> ls_snakemake_003a47d0-0d2c-474a-b734-9f4bf2903a03 >>>> org.apache.airavata.registry.cpi.AppCatalogException: >>>> javax.persistence.NoResultException: Query "SELECT p FROM >>>> ApplicationInterface p WHERE p.interfaceID =:param0" selected no result, >>>> but expected unique result. >>>> >>>> This “ls_snakemake” job is months old and is associated with an interface, >>>> module and deployment which no longer exist, which is why this error is >>>> getting thrown. What I don’t understand is what is “reminding” Airavata of >>>> this job and causing it to get hung up. Last week, I found references to >>>> it in the experiment catalog database in the EXPERIMENT table, and >>>> deleting these allowed Airavata to continue. Now it is happening again and >>>> the references are gone. How do I eliminate all references to this job >>>> from all of Airavata such that it stops getting hung up on this phantom >>>> job? >>>> >>>> Thanks, >>>> Jarett >>> >> >
