[jira] [Commented] (AIRAVATA-2376) Duplicating experiment records in Experiment statistics and Experiment browse
[ https://issues.apache.org/jira/browse/AIRAVATA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118805#comment-16118805 ] ASF subversion and git services commented on AIRAVATA-2376: --- Commit e9b42768b1b64138ba5883772f1d6dd37303a11b in airavata's branch refs/heads/develop from [~marcuschristie] [ https://git-wip-us.apache.org/repos/asf?p=airavata.git;h=e9b4276 ] Merge branch 'AIRAVATA-2376' > Duplicating experiment records in Experiment statistics and Experiment browse > - > > Key: AIRAVATA-2376 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2376 > Project: Airavata > Issue Type: Bug > Components: PGA PHP Web Gateway, Registry API >Affects Versions: 0.17 > Environment: https://seagrid.org and all other gateways as well >Reporter: Eroma >Assignee: Marcus Christie >Priority: Critical > Fix For: 0.17 > > > When experiments are launched there are two records in EXPERIMENT SUMMARY > table for LAUNCHED and EXECUTING state which has the exact same state change > time. In PGA however two records show up for the same experiment and with > same status. e.g.: two exact records for a single experiment. This only > appears until the experiment is FAILED or COMPLETED. But both users and > gateway admins notices this and this needs fixing. > h3. Resources > * MySQL JDBC driver configuration: > ** > https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html > ** uses sendFractionalSeconds > ** added in [MySQL Connector/J > 5.1.37|https://docs.oracle.com/cd/E17952_01/connector-j-relnotes-en/news-5-1-37.html] > * https://issues.apache.org/jira/browse/OPENJPA-2159 > * MariaDB JDBC driver configuration: > ** https://mariadb.com/kb/en/mariadb/about-mariadb-connector-j/ > ** uses useFractionalSeconds > * MariaDB JDBC driver defaults to useFractionalSeconds as true since 1.1.8 > ** https://mariadb.com/kb/en/mariadb/mariadb-java-client-118-changelog/ > * [MariaDB TIMESTAMP|https://mariadb.com/kb/en/mariadb/timestamp/] > * [Microseconds in > MariaDB|https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/] > * How to specify TIMESTAMP precision in JPA annotations? > ** [Confused why they seem to use scale instead of precision in > OpenJPA|https://issues.apache.org/jira/browse/OPENJPA-2555] > * [Supporting evidence that we need to use MariaDB driver when using MariaDB > server|https://issues.alfresco.com/jira/browse/MNT-17613] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AIRAVATA-2376) Duplicating experiment records in Experiment statistics and Experiment browse
[
https://issues.apache.org/jira/browse/AIRAVATA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068337#comment-16068337
]
Marcus Christie commented on AIRAVATA-2376:
---
I found an example of an experiment with two statuses in the same second but
only one status in LATEST_EXPERIMENT_STATUS:
In EXPERIMENT_STATUS table:
| CREATED |
LT_Cu2Se_HangChi_order_13_8A_grad_3_restart_f231677b-141c-4eb1-a477-dbc75ca7a42b
|
2017-06-28 18:56:50.938000 |
| LAUNCHED |
LT_Cu2Se_HangChi_order_13_8A_grad_3_restart_f231677b-141c-4eb1-a477-dbc75ca7a42b
|
2017-06-28 18:56:52.167000 |
| EXECUTING |
LT_Cu2Se_HangChi_order_13_8A_grad_3_restart_f231677b-141c-4eb1-a477-dbc75ca7a42b
|
2017-06-28 18:56:52.485000 |
In LATEST_EXPERIMENT_STATUS:
{code:sql}
select * from LATEST_EXPERIMENT_STATUS where EXPERIMENT_ID =
'LT_Cu2Se_HangChi_order_13_8A_grad_3_restart_f231677b-141c-4eb1-a477-dbc75ca7a42b';
{code}
|
LT_Cu2Se_HangChi_order_13_8A_grad_3_restart_f231677b-141c-4eb1-a477-dbc75ca7a42b
| EXECUTING | 2017-06-28 18:56:52.485000 |
> Duplicating experiment records in Experiment statistics and Experiment browse
> -
>
> Key: AIRAVATA-2376
> URL: https://issues.apache.org/jira/browse/AIRAVATA-2376
> Project: Airavata
> Issue Type: Bug
> Components: PGA PHP Web Gateway, Registry API
>Affects Versions: 0.17
> Environment: https://seagrid.org and all other gateways as well
>Reporter: Eroma
>Assignee: Marcus Christie
>Priority: Critical
> Fix For: 0.17
>
>
> When experiments are launched there are two records in EXPERIMENT SUMMARY
> table for LAUNCHED and EXECUTING state which has the exact same state change
> time. In PGA however two records show up for the same experiment and with
> same status. e.g.: two exact records for a single experiment. This only
> appears until the experiment is FAILED or COMPLETED. But both users and
> gateway admins notices this and this needs fixing.
> h3. Resources
> * MySQL JDBC driver configuration:
> **
> https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html
> ** uses sendFractionalSeconds
> ** added in [MySQL Connector/J
> 5.1.37|https://docs.oracle.com/cd/E17952_01/connector-j-relnotes-en/news-5-1-37.html]
> * https://issues.apache.org/jira/browse/OPENJPA-2159
> * MariaDB JDBC driver configuration:
> ** https://mariadb.com/kb/en/mariadb/about-mariadb-connector-j/
> ** uses useFractionalSeconds
> * MariaDB JDBC driver defaults to useFractionalSeconds as true since 1.1.8
> ** https://mariadb.com/kb/en/mariadb/mariadb-java-client-118-changelog/
> * [MariaDB TIMESTAMP|https://mariadb.com/kb/en/mariadb/timestamp/]
> * [Microseconds in
> MariaDB|https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/]
> * How to specify TIMESTAMP precision in JPA annotations?
> ** [Confused why they seem to use scale instead of precision in
> OpenJPA|https://issues.apache.org/jira/browse/OPENJPA-2555]
> * [Supporting evidence that we need to use MariaDB driver when using MariaDB
> server|https://issues.alfresco.com/jira/browse/MNT-17613]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AIRAVATA-2376) Duplicating experiment records in Experiment statistics and Experiment browse
[
https://issues.apache.org/jira/browse/AIRAVATA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052244#comment-16052244
]
Marcus Christie commented on AIRAVATA-2376:
---
Ran
{code:sql}
ALTER TABLE EXPERIMENT_STATUS MODIFY COLUMN TIME_OF_STATE_CHANGE TIMESTAMP(6)
NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6);
{code}
on the SciGaP develop environment on Jetstream.
> Duplicating experiment records in Experiment statistics and Experiment browse
> -
>
> Key: AIRAVATA-2376
> URL: https://issues.apache.org/jira/browse/AIRAVATA-2376
> Project: Airavata
> Issue Type: Bug
> Components: PGA PHP Web Gateway, Registry API
>Affects Versions: 0.17
> Environment: https://seagrid.org and all other gateways as well
>Reporter: Eroma
>Assignee: Marcus Christie
>Priority: Critical
> Fix For: 0.17
>
>
> When experiments are launched there are two records in EXPERIMENT SUMMARY
> table for LAUNCHED and EXECUTING state which has the exact same state change
> time. In PGA however two records show up for the same experiment and with
> same status. e.g.: two exact records for a single experiment. This only
> appears until the experiment is FAILED or COMPLETED. But both users and
> gateway admins notices this and this needs fixing.
> h3. Resources
> * MySQL JDBC driver configuration:
> **
> https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html
> ** uses sendFractionalSeconds
> ** added in [MySQL Connector/J
> 5.1.37|https://docs.oracle.com/cd/E17952_01/connector-j-relnotes-en/news-5-1-37.html]
> * https://issues.apache.org/jira/browse/OPENJPA-2159
> * MariaDB JDBC driver configuration:
> ** https://mariadb.com/kb/en/mariadb/about-mariadb-connector-j/
> ** uses useFractionalSeconds
> * MariaDB JDBC driver defaults to useFractionalSeconds as true since 1.1.8
> ** https://mariadb.com/kb/en/mariadb/mariadb-java-client-118-changelog/
> * [MariaDB TIMESTAMP|https://mariadb.com/kb/en/mariadb/timestamp/]
> * [Microseconds in
> MariaDB|https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/]
> * How to specify TIMESTAMP precision in JPA annotations?
> ** [Confused why they seem to use scale instead of precision in
> OpenJPA|https://issues.apache.org/jira/browse/OPENJPA-2555]
> * [Supporting evidence that we need to use MariaDB driver when using MariaDB
> server|https://issues.alfresco.com/jira/browse/MNT-17613]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AIRAVATA-2376) Duplicating experiment records in Experiment statistics and Experiment browse
[ https://issues.apache.org/jira/browse/AIRAVATA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050715#comment-16050715 ] ASF subversion and git services commented on AIRAVATA-2376: --- Commit 93fa1f71da9a2438768c7db7f75a934a52577274 in airavata's branch refs/heads/develop from [~marcuschristie] [ https://git-wip-us.apache.org/repos/asf?p=airavata.git;h=93fa1f7 ] Merge branch 'AIRAVATA-2376' into develop > Duplicating experiment records in Experiment statistics and Experiment browse > - > > Key: AIRAVATA-2376 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2376 > Project: Airavata > Issue Type: Bug > Components: PGA PHP Web Gateway, Registry API >Affects Versions: 0.17 > Environment: https://seagrid.org and all other gateways as well >Reporter: Eroma >Assignee: Marcus Christie >Priority: Critical > Fix For: 0.17 > > > When experiments are launched there are two records in EXPERIMENT SUMMARY > table for LAUNCHED and EXECUTING state which has the exact same state change > time. In PGA however two records show up for the same experiment and with > same status. e.g.: two exact records for a single experiment. This only > appears until the experiment is FAILED or COMPLETED. But both users and > gateway admins notices this and this needs fixing. > h3. Resources > * MySQL JDBC driver configuration: > ** > https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html > ** uses sendFractionalSeconds > ** added in [MySQL Connector/J > 5.1.37|https://docs.oracle.com/cd/E17952_01/connector-j-relnotes-en/news-5-1-37.html] > * https://issues.apache.org/jira/browse/OPENJPA-2159 > * MariaDB JDBC driver configuration: > ** https://mariadb.com/kb/en/mariadb/about-mariadb-connector-j/ > ** uses useFractionalSeconds > * MariaDB JDBC driver defaults to useFractionalSeconds as true since 1.1.8 > ** https://mariadb.com/kb/en/mariadb/mariadb-java-client-118-changelog/ > * [MariaDB TIMESTAMP|https://mariadb.com/kb/en/mariadb/timestamp/] > * [Microseconds in > MariaDB|https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/] > * How to specify TIMESTAMP precision in JPA annotations? > ** [Confused why they seem to use scale instead of precision in > OpenJPA|https://issues.apache.org/jira/browse/OPENJPA-2555] > * [Supporting evidence that we need to use MariaDB driver when using MariaDB > server|https://issues.alfresco.com/jira/browse/MNT-17613] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AIRAVATA-2376) Duplicating experiment records in Experiment statistics and Experiment browse
[ https://issues.apache.org/jira/browse/AIRAVATA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050716#comment-16050716 ] ASF subversion and git services commented on AIRAVATA-2376: --- Commit 2b7a9a344e9c29173a532e6e8cfb9afcb65d27ef in airavata's branch refs/heads/develop from [~marcuschristie] [ https://git-wip-us.apache.org/repos/asf?p=airavata.git;h=2b7a9a3 ] AIRAVATA-2376 Switch to MariaDB for Profile Service > Duplicating experiment records in Experiment statistics and Experiment browse > - > > Key: AIRAVATA-2376 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2376 > Project: Airavata > Issue Type: Bug > Components: PGA PHP Web Gateway, Registry API >Affects Versions: 0.17 > Environment: https://seagrid.org and all other gateways as well >Reporter: Eroma >Assignee: Marcus Christie >Priority: Critical > Fix For: 0.17 > > > When experiments are launched there are two records in EXPERIMENT SUMMARY > table for LAUNCHED and EXECUTING state which has the exact same state change > time. In PGA however two records show up for the same experiment and with > same status. e.g.: two exact records for a single experiment. This only > appears until the experiment is FAILED or COMPLETED. But both users and > gateway admins notices this and this needs fixing. > h3. Resources > * MySQL JDBC driver configuration: > ** > https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html > ** uses sendFractionalSeconds > ** added in [MySQL Connector/J > 5.1.37|https://docs.oracle.com/cd/E17952_01/connector-j-relnotes-en/news-5-1-37.html] > * https://issues.apache.org/jira/browse/OPENJPA-2159 > * MariaDB JDBC driver configuration: > ** https://mariadb.com/kb/en/mariadb/about-mariadb-connector-j/ > ** uses useFractionalSeconds > * MariaDB JDBC driver defaults to useFractionalSeconds as true since 1.1.8 > ** https://mariadb.com/kb/en/mariadb/mariadb-java-client-118-changelog/ > * [MariaDB TIMESTAMP|https://mariadb.com/kb/en/mariadb/timestamp/] > * [Microseconds in > MariaDB|https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/] > * How to specify TIMESTAMP precision in JPA annotations? > ** [Confused why they seem to use scale instead of precision in > OpenJPA|https://issues.apache.org/jira/browse/OPENJPA-2555] > * [Supporting evidence that we need to use MariaDB driver when using MariaDB > server|https://issues.alfresco.com/jira/browse/MNT-17613] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AIRAVATA-2376) Duplicating experiment records in Experiment statistics and Experiment browse
[ https://issues.apache.org/jira/browse/AIRAVATA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050714#comment-16050714 ] ASF subversion and git services commented on AIRAVATA-2376: --- Commit 047521f3a8f54dac9edf6bdc66f89f1d75003400 in airavata's branch refs/heads/develop from [~marcuschristie] [ https://git-wip-us.apache.org/repos/asf?p=airavata.git;h=047521f ] AIRAVATA-2376 Switch to MariaDB jar in Ansible > Duplicating experiment records in Experiment statistics and Experiment browse > - > > Key: AIRAVATA-2376 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2376 > Project: Airavata > Issue Type: Bug > Components: PGA PHP Web Gateway, Registry API >Affects Versions: 0.17 > Environment: https://seagrid.org and all other gateways as well >Reporter: Eroma >Assignee: Marcus Christie >Priority: Critical > Fix For: 0.17 > > > When experiments are launched there are two records in EXPERIMENT SUMMARY > table for LAUNCHED and EXECUTING state which has the exact same state change > time. In PGA however two records show up for the same experiment and with > same status. e.g.: two exact records for a single experiment. This only > appears until the experiment is FAILED or COMPLETED. But both users and > gateway admins notices this and this needs fixing. > h3. Resources > * MySQL JDBC driver configuration: > ** > https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html > ** uses sendFractionalSeconds > ** added in [MySQL Connector/J > 5.1.37|https://docs.oracle.com/cd/E17952_01/connector-j-relnotes-en/news-5-1-37.html] > * https://issues.apache.org/jira/browse/OPENJPA-2159 > * MariaDB JDBC driver configuration: > ** https://mariadb.com/kb/en/mariadb/about-mariadb-connector-j/ > ** uses useFractionalSeconds > * MariaDB JDBC driver defaults to useFractionalSeconds as true since 1.1.8 > ** https://mariadb.com/kb/en/mariadb/mariadb-java-client-118-changelog/ > * [MariaDB TIMESTAMP|https://mariadb.com/kb/en/mariadb/timestamp/] > * [Microseconds in > MariaDB|https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/] > * How to specify TIMESTAMP precision in JPA annotations? > ** [Confused why they seem to use scale instead of precision in > OpenJPA|https://issues.apache.org/jira/browse/OPENJPA-2555] > * [Supporting evidence that we need to use MariaDB driver when using MariaDB > server|https://issues.alfresco.com/jira/browse/MNT-17613] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AIRAVATA-2376) Duplicating experiment records in Experiment statistics and Experiment browse
[ https://issues.apache.org/jira/browse/AIRAVATA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050713#comment-16050713 ] ASF subversion and git services commented on AIRAVATA-2376: --- Commit 7ff713008d69fc62f521128f2a3f8169d513f416 in airavata's branch refs/heads/develop from [~marcuschristie] [ https://git-wip-us.apache.org/repos/asf?p=airavata.git;h=7ff7130 ] AIRAVATA-2376 Add millisecond precision to TIME_OF_STATE_CHANGE > Duplicating experiment records in Experiment statistics and Experiment browse > - > > Key: AIRAVATA-2376 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2376 > Project: Airavata > Issue Type: Bug > Components: PGA PHP Web Gateway, Registry API >Affects Versions: 0.17 > Environment: https://seagrid.org and all other gateways as well >Reporter: Eroma >Assignee: Marcus Christie >Priority: Critical > Fix For: 0.17 > > > When experiments are launched there are two records in EXPERIMENT SUMMARY > table for LAUNCHED and EXECUTING state which has the exact same state change > time. In PGA however two records show up for the same experiment and with > same status. e.g.: two exact records for a single experiment. This only > appears until the experiment is FAILED or COMPLETED. But both users and > gateway admins notices this and this needs fixing. > h3. Resources > * MySQL JDBC driver configuration: > ** > https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html > ** uses sendFractionalSeconds > ** added in [MySQL Connector/J > 5.1.37|https://docs.oracle.com/cd/E17952_01/connector-j-relnotes-en/news-5-1-37.html] > * https://issues.apache.org/jira/browse/OPENJPA-2159 > * MariaDB JDBC driver configuration: > ** https://mariadb.com/kb/en/mariadb/about-mariadb-connector-j/ > ** uses useFractionalSeconds > * MariaDB JDBC driver defaults to useFractionalSeconds as true since 1.1.8 > ** https://mariadb.com/kb/en/mariadb/mariadb-java-client-118-changelog/ > * [MariaDB TIMESTAMP|https://mariadb.com/kb/en/mariadb/timestamp/] > * [Microseconds in > MariaDB|https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/] > * How to specify TIMESTAMP precision in JPA annotations? > ** [Confused why they seem to use scale instead of precision in > OpenJPA|https://issues.apache.org/jira/browse/OPENJPA-2555] > * [Supporting evidence that we need to use MariaDB driver when using MariaDB > server|https://issues.alfresco.com/jira/browse/MNT-17613] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AIRAVATA-2376) Duplicating experiment records in Experiment statistics and Experiment browse
[
https://issues.apache.org/jira/browse/AIRAVATA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991236#comment-15991236
]
Marcus Christie commented on AIRAVATA-2376:
---
Just documenting a quick fix I tried to apply but it didn't work but it might
be the start of a complete solution.
The problem is in the LATEST_EXPERIMENT_STATUS view. It orders
EXPERIMENT_STATUS by TIME_OF_STATE_CHANGE but sometimes two statuses have the
same TIME_OF_STATE_CHANGE.
There are a couple of ways to fix this. One way that I tried is that the
TIME_OF_STATE_CHANGE only has accuracy to the nearest second. I tried adding
millisecond accuracy like so
{code:sql}
ALTER TABLE EXPERIMENT_STATUS MODIFY COLUMN TIME_OF_STATE_CHANGE TIMESTAMP(6)
NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6);
{code}
See also https://mariadb.com/kb/en/mariadb/microseconds-in-mariadb/
This changed the definition of the column but the values stored are still being
put in as a whole second. I think maybe the client code needs to be updated
somehow. According to http://stackoverflow.com/a/25803063 one has to enable
useFractionalSeconds in the JDBC driver.
> Duplicating experiment records in Experiment statistics and Experiment browse
> -
>
> Key: AIRAVATA-2376
> URL: https://issues.apache.org/jira/browse/AIRAVATA-2376
> Project: Airavata
> Issue Type: Bug
> Components: PGA PHP Web Gateway, Registry API
>Affects Versions: 0.17
> Environment: https://seagrid.org and all other gateways as well
>Reporter: Eroma
>Assignee: Marcus Christie
> Fix For: 0.17
>
>
> When experiments are launched there are two records in EXPERIMENT SUMMARY
> table for LAUNCHED and EXECUTING state which has the exact same state change
> time. In PGA however two records show up for the same experiment and with
> same status. e.g.: two exact records for a single experiment. This only
> appears until the experiment is FAILED or COMPLETED. But both users and
> gateway admins notices this and this needs fixing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
