Re: [opnfv-tech-discuss] [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

2017-08-09 Thread Tim Rozet
Great Juraj.  We merged a fix:
https://gerrit.opnfv.org/gerrit/#/c/38987/

Will make a note to look into it upstream.

Thanks,

Tim Rozet
Red Hat SDN Team

- Original Message -
From: "Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)" 

To: "Tim Rozet" 
Cc: opnfv-tech-discuss@lists.opnfv.org, fds-...@lists.opnfv.org
Sent: Wednesday, August 9, 2017 8:47:46 AM
Subject: RE: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Hi Tim,

There were indeed many more processes than 12 per each openstack service. I'm 
running functest with the configuration and seems to run smoothly.

Could you find out why it hasn't been fixed upstream? 

I also looked at ci results. After Danube we made a number of refactoring 
patches and the version Apex uses is not recent enough (when I use latest 
Carbon the results look better). We also haven't merged all of the patches 
still, so this one is on us.

Juraj

-Original Message-
From: Tim Rozet [mailto:tro...@redhat.com] 
Sent: Tuesday, 08 August, 2017 22:45
To: Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco) 

Cc: opnfv-tech-discuss@lists.opnfv.org; fds-...@lists.opnfv.org
Subject: Re: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Juraj,
Now I'm thinking this is the same problem as before.  I see we had this in 
Danube:
https://github.com/opnfv/apex/blob/stable/danube/build/baremetal-environment.yaml

which we are not using in master because this problem was supposed to be fixed 
upstream.  Can you try including this file in your overcloud deployment and see 
if it works?

Thanks,

Tim Rozet
Red Hat SDN Team

- Original Message -
From: "Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)" 

To: "Tim Rozet" , opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Sent: Friday, August 4, 2017 11:44:16 AM
Subject: RE: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

The box is not OOM – it has more than 200GB spare. One thing I forgot to 
mention was that I let the setup as is for a day or two (without doing anything 
with it) and then the issue appeared.

Juraj

From: Tim Rozet [mailto:tro...@redhat.com]
Sent: Friday, 04 August, 2017 16:09
To: Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco) 
; opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Subject: Re: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Hi Juraj,
This error looks different than the files limit problem.  This actually looks 
to be an error in the python sqlalchemy call, complaining there isn't enough 
ram to spawn a session thread.  Is your box oom?

You can check the daily job's functest results.  I believe it deployed that 
scenario in the last iteration.  Otherwise ping me Monday with the setup and 
I'll try to debug.



Sent via the Samsung Galaxy S®6 active, an AT 4G LTE smartphone

 Original message 
From: "Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)" 
>
Date: 8/4/17 8:34 AM (GMT-05:00)
To: 
opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Subject: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Hi Apex team,

I tried an os-odl-fdio-ha euphrates baremetal deployment and I ran into mariadb 
issues when I ran functest: 
https://paste.fedoraproject.org/paste/R4tBkzeub0N2ksYVMsabNg/raw

I've seen this issue before when the number of open files was too low in 
mariadb, but I seems to be fine:
MariaDB [(none)]> show variables like 'open_files_limit';
+--+---+
| Variable_name| Value |
+--+---+
| open_files_limit | 20485 |
+--+---+

Has anyone seen this with any other scenario?

Juraj
___
opnfv-tech-discuss mailing list
opnfv-tech-discuss@lists.opnfv.org
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss


Re: [opnfv-tech-discuss] [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

2017-08-09 Thread Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)
Hi Tim,

There were indeed many more processes than 12 per each openstack service. I'm 
running functest with the configuration and seems to run smoothly.

Could you find out why it hasn't been fixed upstream? 

I also looked at ci results. After Danube we made a number of refactoring 
patches and the version Apex uses is not recent enough (when I use latest 
Carbon the results look better). We also haven't merged all of the patches 
still, so this one is on us.

Juraj

-Original Message-
From: Tim Rozet [mailto:tro...@redhat.com] 
Sent: Tuesday, 08 August, 2017 22:45
To: Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco) 

Cc: opnfv-tech-discuss@lists.opnfv.org; fds-...@lists.opnfv.org
Subject: Re: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Juraj,
Now I'm thinking this is the same problem as before.  I see we had this in 
Danube:
https://github.com/opnfv/apex/blob/stable/danube/build/baremetal-environment.yaml

which we are not using in master because this problem was supposed to be fixed 
upstream.  Can you try including this file in your overcloud deployment and see 
if it works?

Thanks,

Tim Rozet
Red Hat SDN Team

- Original Message -
From: "Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)" 

To: "Tim Rozet" , opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Sent: Friday, August 4, 2017 11:44:16 AM
Subject: RE: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

The box is not OOM – it has more than 200GB spare. One thing I forgot to 
mention was that I let the setup as is for a day or two (without doing anything 
with it) and then the issue appeared.

Juraj

From: Tim Rozet [mailto:tro...@redhat.com]
Sent: Friday, 04 August, 2017 16:09
To: Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco) 
; opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Subject: Re: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Hi Juraj,
This error looks different than the files limit problem.  This actually looks 
to be an error in the python sqlalchemy call, complaining there isn't enough 
ram to spawn a session thread.  Is your box oom?

You can check the daily job's functest results.  I believe it deployed that 
scenario in the last iteration.  Otherwise ping me Monday with the setup and 
I'll try to debug.



Sent via the Samsung Galaxy S®6 active, an AT 4G LTE smartphone

 Original message 
From: "Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)" 
>
Date: 8/4/17 8:34 AM (GMT-05:00)
To: 
opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Subject: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Hi Apex team,

I tried an os-odl-fdio-ha euphrates baremetal deployment and I ran into mariadb 
issues when I ran functest: 
https://paste.fedoraproject.org/paste/R4tBkzeub0N2ksYVMsabNg/raw

I've seen this issue before when the number of open files was too low in 
mariadb, but I seems to be fine:
MariaDB [(none)]> show variables like 'open_files_limit';
+--+---+
| Variable_name| Value |
+--+---+
| open_files_limit | 20485 |
+--+---+

Has anyone seen this with any other scenario?

Juraj
___
opnfv-tech-discuss mailing list
opnfv-tech-discuss@lists.opnfv.org
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss


Re: [opnfv-tech-discuss] [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

2017-08-08 Thread Tim Rozet
Juraj,
Now I'm thinking this is the same problem as before.  I see we had this in 
Danube:
https://github.com/opnfv/apex/blob/stable/danube/build/baremetal-environment.yaml

which we are not using in master because this problem was supposed to be fixed 
upstream.  Can you try including this file in your overcloud deployment and see 
if it works?

Thanks,

Tim Rozet
Red Hat SDN Team

- Original Message -
From: "Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)" 

To: "Tim Rozet" , opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Sent: Friday, August 4, 2017 11:44:16 AM
Subject: RE: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

The box is not OOM – it has more than 200GB spare. One thing I forgot to 
mention was that I let the setup as is for a day or two (without doing anything 
with it) and then the issue appeared.

Juraj

From: Tim Rozet [mailto:tro...@redhat.com]
Sent: Friday, 04 August, 2017 16:09
To: Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco) 
; opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Subject: Re: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Hi Juraj,
This error looks different than the files limit problem.  This actually looks 
to be an error in the python sqlalchemy call, complaining there isn't enough 
ram to spawn a session thread.  Is your box oom?

You can check the daily job's functest results.  I believe it deployed that 
scenario in the last iteration.  Otherwise ping me Monday with the setup and 
I'll try to debug.



Sent via the Samsung Galaxy S®6 active, an AT 4G LTE smartphone

 Original message 
From: "Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)" 
>
Date: 8/4/17 8:34 AM (GMT-05:00)
To: 
opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Subject: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Hi Apex team,

I tried an os-odl-fdio-ha euphrates baremetal deployment and I ran into mariadb 
issues when I ran functest: 
https://paste.fedoraproject.org/paste/R4tBkzeub0N2ksYVMsabNg/raw

I've seen this issue before when the number of open files was too low in 
mariadb, but I seems to be fine:
MariaDB [(none)]> show variables like 'open_files_limit';
+--+---+
| Variable_name| Value |
+--+---+
| open_files_limit | 20485 |
+--+---+

Has anyone seen this with any other scenario?

Juraj
___
opnfv-tech-discuss mailing list
opnfv-tech-discuss@lists.opnfv.org
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss


Re: [opnfv-tech-discuss] [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

2017-08-04 Thread Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)
The box is not OOM – it has more than 200GB spare. One thing I forgot to 
mention was that I let the setup as is for a day or two (without doing anything 
with it) and then the issue appeared.

Juraj

From: Tim Rozet [mailto:tro...@redhat.com]
Sent: Friday, 04 August, 2017 16:09
To: Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco) 
; opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Subject: Re: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Hi Juraj,
This error looks different than the files limit problem.  This actually looks 
to be an error in the python sqlalchemy call, complaining there isn't enough 
ram to spawn a session thread.  Is your box oom?

You can check the daily job's functest results.  I believe it deployed that 
scenario in the last iteration.  Otherwise ping me Monday with the setup and 
I'll try to debug.



Sent via the Samsung Galaxy S®6 active, an AT 4G LTE smartphone

 Original message 
From: "Juraj Linkes -X (jlinkes - PANTHEON TECHNOLOGIES at Cisco)" 
>
Date: 8/4/17 8:34 AM (GMT-05:00)
To: 
opnfv-tech-discuss@lists.opnfv.org
Cc: fds-...@lists.opnfv.org
Subject: [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

Hi Apex team,

I tried an os-odl-fdio-ha euphrates baremetal deployment and I ran into mariadb 
issues when I ran functest: 
https://paste.fedoraproject.org/paste/R4tBkzeub0N2ksYVMsabNg/raw

I've seen this issue before when the number of open files was too low in 
mariadb, but I seems to be fine:
MariaDB [(none)]> show variables like 'open_files_limit';
+--+---+
| Variable_name| Value |
+--+---+
| open_files_limit | 20485 |
+--+---+

Has anyone seen this with any other scenario?

Juraj
___
opnfv-tech-discuss mailing list
opnfv-tech-discuss@lists.opnfv.org
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss


Re: [opnfv-tech-discuss] [Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues

2017-08-04 Thread Tim Rozet
Hi Juraj,This error looks different than the files limit problem.  This 
actually looks to be an error in the python sqlalchemy call, complaining there 
isn't enough ram to spawn a session thread.  Is your box oom?
You can check the daily job's functest results.  I believe it deployed that 
scenario in the last iteration.  Otherwise ping me Monday with the setup and 
I'll try to debug.


Sent via the Samsung Galaxy S®6 active, an AT 4G LTE smartphone
 Original message From: "Juraj Linkes -X (jlinkes - PANTHEON 
TECHNOLOGIES at Cisco)"  Date: 8/4/17  8:34 AM  (GMT-05:00) 
To: opnfv-tech-discuss@lists.opnfv.org Cc: fds-...@lists.opnfv.org Subject: 
[Fds-dev] [APEX] [Euphrates] Euphrates testing - database issues 


Hi Apex team,
 
I tried an os-odl-fdio-ha euphrates baremetal deployment and I ran into mariadb 
issues when I ran functest:
https://paste.fedoraproject.org/paste/R4tBkzeub0N2ksYVMsabNg/raw
 
I've seen this issue before when the number of open files was too low in 
mariadb, but I seems to be fine:
MariaDB [(none)]> show variables like 'open_files_limit';
+--+---+
| Variable_name    | Value |
+--+---+
| open_files_limit | 20485 |
+--+---+
 
Has anyone seen this with any other scenario?
 
Juraj

___
opnfv-tech-discuss mailing list
opnfv-tech-discuss@lists.opnfv.org
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss