Re: [Freeipa-users] Gateway_timeout Error

[deepak dimri]

Hi All,

I am stuck with this gateway error on my replicas. I recreated the replicas
but that did not help either. I realised that if i just keep my primary ipa
up then i do not get the error on the secondary/replica server. The error
logs on replica shows hits are getting successfully executed but i am
certain that its trying to bind to primary ipa server when i am trying to
open the hosts/users entries. It seems its failing to make ldap bind to
primary server and then eventually timing out.

Any idea why in my case replica is trying to connect to ipa master?

Thanks,
Deepak



On Thu, Feb 2, 2017 at 10:12 AM, deepak dimri 
wrote:

> Hey Martin,
>
>
> Is gateway error has anything to do with --no-wait-for-dns flag that i
> used when i created the replica image? i have another test IPA setup
> working fine in the same env and the only difference i see that in that env
> i did not use --no-wait-for-dns for replicas
>
> Thanks,
> Deepak
>
> On Wed, Feb 1, 2017 at 10:52 PM, deepak dimri 
> wrote:
>
>> sorry for not replying to all!
>>
>> I have apache reverse proxy front ending the ipa servers. As i mentioned
>> if i try hitting ipa replica WebUI directly then i do get the objects
>> loaded on the browser after waiting for over a minute or so. replica server
>> (/var/log/dirsrv/slapd-$YOUR_REALM/{access,errors}) shows hits coming
>> through fine but for some reasons web browser ends up with the gateway
>> error.
>>
>> both the ipa masters are running VERSION: 4.4.0, API_VERSION: 2.213
>>
>> Kind Regards,
>> Deepak
>>
>>
>> On Wed, Feb 1, 2017 at 9:21 PM, Martin Babinsky 
>> wrote:
>>
>>> On 02/01/2017 04:26 PM, deepak dimri wrote:
>>>
 Yes, Martin - i do see requests hitting
 replica.. /var/log/httpd/error_log shows:

 [Wed Feb 01 15:16:47.469766 2017] [:error] [pid 2464] ipa: INFO:
 ad...@xxx.xyz.com : batch:
 host_show(u'xxx.abx.xyz ', rights=True, all=True):
 SUCCESS

 I used ansible playbook to build the replica server. ran
 ipa-replica-prepare on the primary:
 ipa-replica-prepare {{ replica_dns }} --password={{ipa_password}}
 --no-wait-for-dns

 copied the replica file over to replica server:
 scp -oStrictHostKeyChecking=no -i ~/.ssh/{{ssh_keyname}}.pem
 /var/lib/ipa/replica-info-{{ replica_dns }}.gpg root@{{
 replica_dns }}:/var/lib/ipa/

 ran the replica install on the replica server:
 ipa-replica-install /var/lib/ipa/replica-info-{{  replica_dns }}.gpg
 --password={{ipa_password}} --admin-password={{ipa_password}}

 I have notices that if i directly use the replica (bypassing proxy)  URL
 then the objects shows after waiting for over a minute or so. When i use
 proxy pass then it just times out after few seconds.

 No clue why its behaving like this

 Many Thanks,
 Deepak

 On Wed, Feb 1, 2017 at 6:45 PM, Martin Babinsky > wrote:

 On 02/01/2017 11:17 AM, deepak dimri wrote:

 Hello Martin, Thank you so much for your reply.

 I checked /etc/ipa/default.conf 'xmlrpc_uri' on my secondary
 server and
 its pointing to its own hostname and not to primary server
 hostname :(

 any other clue, Martin?

 I have tried without proxy and again to luck either its throwing
 same
 gateway_error

 Regards,
 Deepak

 On Wed, Feb 1, 2017 at 3:03 PM, Martin Babinsky
 
 >>
 wrote:

 On 02/01/2017 10:22 AM, deepak dimri wrote:

 Hi All,

 I have two IPA servers - primary and secondary running.
 the
 secondary
 ipa server is installed using ipa replica image of
 primary.
 While doing
 the testing i realised that when i manually shut down my
 primary ipa
 server making my secondary server to serve the UI. And
 now when
 i try to
 access user or hosts details using my secondary server
 then i am
 getting
 below error in the UI. I am able to login fine though;
 it is
 just that
 when i double click on host objects then i get the
 error.


   An error has occurred (GATEWAY_TIMEOUT)


 I am still trying to troubleshoot as why i am getting
 timeout
 error but
 thought of asking the 

Re: [Freeipa-users] Gateway_timeout Error

[deepak dimri]

Hey Martin,


Is gateway error has anything to do with --no-wait-for-dns flag that i used
when i created the replica image? i have another test IPA setup working
fine in the same env and the only difference i see that in that env i did
not use --no-wait-for-dns for replicas

Thanks,
Deepak

On Wed, Feb 1, 2017 at 10:52 PM, deepak dimri 
wrote:

> sorry for not replying to all!
>
> I have apache reverse proxy front ending the ipa servers. As i mentioned
> if i try hitting ipa replica WebUI directly then i do get the objects
> loaded on the browser after waiting for over a minute or so. replica server
> (/var/log/dirsrv/slapd-$YOUR_REALM/{access,errors}) shows hits coming
> through fine but for some reasons web browser ends up with the gateway
> error.
>
> both the ipa masters are running VERSION: 4.4.0, API_VERSION: 2.213
>
> Kind Regards,
> Deepak
>
>
> On Wed, Feb 1, 2017 at 9:21 PM, Martin Babinsky 
> wrote:
>
>> On 02/01/2017 04:26 PM, deepak dimri wrote:
>>
>>> Yes, Martin - i do see requests hitting
>>> replica.. /var/log/httpd/error_log shows:
>>>
>>> [Wed Feb 01 15:16:47.469766 2017] [:error] [pid 2464] ipa: INFO:
>>> ad...@xxx.xyz.com : batch:
>>> host_show(u'xxx.abx.xyz ', rights=True, all=True):
>>> SUCCESS
>>>
>>> I used ansible playbook to build the replica server. ran
>>> ipa-replica-prepare on the primary:
>>> ipa-replica-prepare {{ replica_dns }} --password={{ipa_password}}
>>> --no-wait-for-dns
>>>
>>> copied the replica file over to replica server:
>>> scp -oStrictHostKeyChecking=no -i ~/.ssh/{{ssh_keyname}}.pem
>>> /var/lib/ipa/replica-info-{{ replica_dns }}.gpg root@{{
>>> replica_dns }}:/var/lib/ipa/
>>>
>>> ran the replica install on the replica server:
>>> ipa-replica-install /var/lib/ipa/replica-info-{{  replica_dns }}.gpg
>>> --password={{ipa_password}} --admin-password={{ipa_password}}
>>>
>>> I have notices that if i directly use the replica (bypassing proxy)  URL
>>> then the objects shows after waiting for over a minute or so. When i use
>>> proxy pass then it just times out after few seconds.
>>>
>>> No clue why its behaving like this
>>>
>>> Many Thanks,
>>> Deepak
>>>
>>> On Wed, Feb 1, 2017 at 6:45 PM, Martin Babinsky >> > wrote:
>>>
>>> On 02/01/2017 11:17 AM, deepak dimri wrote:
>>>
>>> Hello Martin, Thank you so much for your reply.
>>>
>>> I checked /etc/ipa/default.conf 'xmlrpc_uri' on my secondary
>>> server and
>>> its pointing to its own hostname and not to primary server
>>> hostname :(
>>>
>>> any other clue, Martin?
>>>
>>> I have tried without proxy and again to luck either its throwing
>>> same
>>> gateway_error
>>>
>>> Regards,
>>> Deepak
>>>
>>> On Wed, Feb 1, 2017 at 3:03 PM, Martin Babinsky
>>> 
>>> >>
>>> wrote:
>>>
>>> On 02/01/2017 10:22 AM, deepak dimri wrote:
>>>
>>> Hi All,
>>>
>>> I have two IPA servers - primary and secondary running.
>>> the
>>> secondary
>>> ipa server is installed using ipa replica image of
>>> primary.
>>> While doing
>>> the testing i realised that when i manually shut down my
>>> primary ipa
>>> server making my secondary server to serve the UI. And
>>> now when
>>> i try to
>>> access user or hosts details using my secondary server
>>> then i am
>>> getting
>>> below error in the UI. I am able to login fine though;
>>> it is
>>> just that
>>> when i double click on host objects then i get the error.
>>>
>>>
>>>   An error has occurred (GATEWAY_TIMEOUT)
>>>
>>>
>>> I am still trying to troubleshoot as why i am getting
>>> timeout
>>> error but
>>> thought of asking the group here to see if some one can
>>> share
>>> some pointers
>>>
>>> Many Thanks,
>>> Deepak
>>>
>>>
>>> Hi Deepak,
>>>
>>> please check /etc/ipa/default.conf on the secondary server
>>> and check
>>> the value of 'xmlrpc_uri'. Maybe it points to the URL of
>>> primary
>>> server and that's why you get timeouts when it is down.
>>>
>>> Re-setting it to the secondary server itself should fix it.
>>>
>>> --
>>> Martin^3 Babinsky
>>>
>>> --
>>> Manage your subscription for the Freeipa-users mailing list:
>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>> 

Re: [Freeipa-users] Gateway_timeout Error

[deepak dimri]

sorry for not replying to all!

I have apache reverse proxy front ending the ipa servers. As i mentioned if
i try hitting ipa replica WebUI directly then i do get the objects loaded
on the browser after waiting for over a minute or so. replica server
(/var/log/dirsrv/slapd-$YOUR_REALM/{access,errors}) shows hits coming
through fine but for some reasons web browser ends up with the gateway
error.

both the ipa masters are running VERSION: 4.4.0, API_VERSION: 2.213

Kind Regards,
Deepak


On Wed, Feb 1, 2017 at 9:21 PM, Martin Babinsky  wrote:

> On 02/01/2017 04:26 PM, deepak dimri wrote:
>
>> Yes, Martin - i do see requests hitting
>> replica.. /var/log/httpd/error_log shows:
>>
>> [Wed Feb 01 15:16:47.469766 2017] [:error] [pid 2464] ipa: INFO:
>> ad...@xxx.xyz.com : batch:
>> host_show(u'xxx.abx.xyz ', rights=True, all=True):
>> SUCCESS
>>
>> I used ansible playbook to build the replica server. ran
>> ipa-replica-prepare on the primary:
>> ipa-replica-prepare {{ replica_dns }} --password={{ipa_password}}
>> --no-wait-for-dns
>>
>> copied the replica file over to replica server:
>> scp -oStrictHostKeyChecking=no -i ~/.ssh/{{ssh_keyname}}.pem
>> /var/lib/ipa/replica-info-{{ replica_dns }}.gpg root@{{
>> replica_dns }}:/var/lib/ipa/
>>
>> ran the replica install on the replica server:
>> ipa-replica-install /var/lib/ipa/replica-info-{{  replica_dns }}.gpg
>> --password={{ipa_password}} --admin-password={{ipa_password}}
>>
>> I have notices that if i directly use the replica (bypassing proxy)  URL
>> then the objects shows after waiting for over a minute or so. When i use
>> proxy pass then it just times out after few seconds.
>>
>> No clue why its behaving like this
>>
>> Many Thanks,
>> Deepak
>>
>> On Wed, Feb 1, 2017 at 6:45 PM, Martin Babinsky > > wrote:
>>
>> On 02/01/2017 11:17 AM, deepak dimri wrote:
>>
>> Hello Martin, Thank you so much for your reply.
>>
>> I checked /etc/ipa/default.conf 'xmlrpc_uri' on my secondary
>> server and
>> its pointing to its own hostname and not to primary server
>> hostname :(
>>
>> any other clue, Martin?
>>
>> I have tried without proxy and again to luck either its throwing
>> same
>> gateway_error
>>
>> Regards,
>> Deepak
>>
>> On Wed, Feb 1, 2017 at 3:03 PM, Martin Babinsky
>> 
>> >> wrote:
>>
>> On 02/01/2017 10:22 AM, deepak dimri wrote:
>>
>> Hi All,
>>
>> I have two IPA servers - primary and secondary running.
>> the
>> secondary
>> ipa server is installed using ipa replica image of
>> primary.
>> While doing
>> the testing i realised that when i manually shut down my
>> primary ipa
>> server making my secondary server to serve the UI. And
>> now when
>> i try to
>> access user or hosts details using my secondary server
>> then i am
>> getting
>> below error in the UI. I am able to login fine though; it
>> is
>> just that
>> when i double click on host objects then i get the error.
>>
>>
>>   An error has occurred (GATEWAY_TIMEOUT)
>>
>>
>> I am still trying to troubleshoot as why i am getting
>> timeout
>> error but
>> thought of asking the group here to see if some one can
>> share
>> some pointers
>>
>> Many Thanks,
>> Deepak
>>
>>
>> Hi Deepak,
>>
>> please check /etc/ipa/default.conf on the secondary server
>> and check
>> the value of 'xmlrpc_uri'. Maybe it points to the URL of
>> primary
>> server and that's why you get timeouts when it is down.
>>
>> Re-setting it to the secondary server itself should fix it.
>>
>> --
>> Martin^3 Babinsky
>>
>> --
>> Manage your subscription for the Freeipa-users mailing list:
>> https://www.redhat.com/mailman/listinfo/freeipa-users
>> 
>> > >
>> Go to http://freeipa.org for more info on the project
>>
>>
>>
>> Adding freeipa-users back to loop.
>>
>> That is strange, how did you stand up the replica?
>>
>> You can also inspect /var/log/http/error_log on the replica to see
>> whether the commands from the WebUI reach the local HTTP server at
>> all.
>>
>> --
>> Martin^3 

Re: [Freeipa-users] Gateway_timeout Error

[Martin Babinsky]

On 02/01/2017 04:26 PM, deepak dimri wrote:

Yes, Martin - i do see requests hitting
replica.. /var/log/httpd/error_log shows:

[Wed Feb 01 15:16:47.469766 2017] [:error] [pid 2464] ipa: INFO:
ad...@xxx.xyz.com : batch:
host_show(u'xxx.abx.xyz ', rights=True, all=True):
SUCCESS

I used ansible playbook to build the replica server. ran
ipa-replica-prepare on the primary:
ipa-replica-prepare {{ replica_dns }} --password={{ipa_password}}
--no-wait-for-dns

copied the replica file over to replica server:
scp -oStrictHostKeyChecking=no -i ~/.ssh/{{ssh_keyname}}.pem
/var/lib/ipa/replica-info-{{ replica_dns }}.gpg root@{{
replica_dns }}:/var/lib/ipa/

ran the replica install on the replica server:
ipa-replica-install /var/lib/ipa/replica-info-{{  replica_dns }}.gpg
--password={{ipa_password}} --admin-password={{ipa_password}}

I have notices that if i directly use the replica (bypassing proxy)  URL
then the objects shows after waiting for over a minute or so. When i use
proxy pass then it just times out after few seconds.

No clue why its behaving like this

Many Thanks,
Deepak

On Wed, Feb 1, 2017 at 6:45 PM, Martin Babinsky > wrote:

On 02/01/2017 11:17 AM, deepak dimri wrote:

Hello Martin, Thank you so much for your reply.

I checked /etc/ipa/default.conf 'xmlrpc_uri' on my secondary
server and
its pointing to its own hostname and not to primary server
hostname :(

any other clue, Martin?

I have tried without proxy and again to luck either its throwing
same
gateway_error

Regards,
Deepak

On Wed, Feb 1, 2017 at 3:03 PM, Martin Babinsky

>> wrote:

On 02/01/2017 10:22 AM, deepak dimri wrote:

Hi All,

I have two IPA servers - primary and secondary running. the
secondary
ipa server is installed using ipa replica image of primary.
While doing
the testing i realised that when i manually shut down my
primary ipa
server making my secondary server to serve the UI. And
now when
i try to
access user or hosts details using my secondary server
then i am
getting
below error in the UI. I am able to login fine though; it is
just that
when i double click on host objects then i get the error.


  An error has occurred (GATEWAY_TIMEOUT)


I am still trying to troubleshoot as why i am getting
timeout
error but
thought of asking the group here to see if some one can
share
some pointers

Many Thanks,
Deepak


Hi Deepak,

please check /etc/ipa/default.conf on the secondary server
and check
the value of 'xmlrpc_uri'. Maybe it points to the URL of primary
server and that's why you get timeouts when it is down.

Re-setting it to the secondary server itself should fix it.

--
Martin^3 Babinsky

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users

>
Go to http://freeipa.org for more info on the project



Adding freeipa-users back to loop.

That is strange, how did you stand up the replica?

You can also inspect /var/log/http/error_log on the replica to see
whether the commands from the WebUI reach the local HTTP server at all.

--
Martin^3 Babinsky




Deepak,

please keep replying to freeipa-users mailing list, otherwise other 
members do not get updates on your problem.


As for the issues with replica, I did not notice before that you are 
connecting to WebUI through a proxy, what kind of proxy is that and how 
is it configured?


Nevertheless waiting for over a minute to display entries does not sound 
right. I would investigate the root cause of this performance regression 
by checking DS access and error logs on the replica 
(/var/log/dirsrv/slapd-$YOUR_REALM/{access,errors}).


Does the master also take so long time to respond? What are the IPA 
versions of master/replica?


--
Martin^3 Babinsky

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Gateway_timeout Error

[Martin Babinsky]

On 02/01/2017 11:17 AM, deepak dimri wrote:

Hello Martin, Thank you so much for your reply.

I checked /etc/ipa/default.conf 'xmlrpc_uri' on my secondary server and
its pointing to its own hostname and not to primary server hostname :(

any other clue, Martin?

I have tried without proxy and again to luck either its throwing same
gateway_error

Regards,
Deepak

On Wed, Feb 1, 2017 at 3:03 PM, Martin Babinsky > wrote:

On 02/01/2017 10:22 AM, deepak dimri wrote:

Hi All,

I have two IPA servers - primary and secondary running. the
secondary
ipa server is installed using ipa replica image of primary.
While doing
the testing i realised that when i manually shut down my primary ipa
server making my secondary server to serve the UI. And now when
i try to
access user or hosts details using my secondary server then i am
getting
below error in the UI. I am able to login fine though; it is
just that
when i double click on host objects then i get the error.


  An error has occurred (GATEWAY_TIMEOUT)


I am still trying to troubleshoot as why i am getting timeout
error but
thought of asking the group here to see if some one can share
some pointers

Many Thanks,
Deepak


Hi Deepak,

please check /etc/ipa/default.conf on the secondary server and check
the value of 'xmlrpc_uri'. Maybe it points to the URL of primary
server and that's why you get timeouts when it is down.

Re-setting it to the secondary server itself should fix it.

--
Martin^3 Babinsky

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users

Go to http://freeipa.org for more info on the project




Adding freeipa-users back to loop.

That is strange, how did you stand up the replica?

You can also inspect /var/log/http/error_log on the replica to see 
whether the commands from the WebUI reach the local HTTP server at all.


--
Martin^3 Babinsky

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Gateway_timeout Error

[Martin Babinsky]

On 02/01/2017 10:22 AM, deepak dimri wrote:

Hi All,

I have two IPA servers - primary and secondary running. the secondary
ipa server is installed using ipa replica image of primary.  While doing
the testing i realised that when i manually shut down my primary ipa
server making my secondary server to serve the UI. And now when i try to
access user or hosts details using my secondary server then i am getting
below error in the UI. I am able to login fine though; it is just that
when i double click on host objects then i get the error.


  An error has occurred (GATEWAY_TIMEOUT)


I am still trying to troubleshoot as why i am getting timeout error but
thought of asking the group here to see if some one can share some pointers

Many Thanks,
Deepak



Hi Deepak,

please check /etc/ipa/default.conf on the secondary server and check the 
value of 'xmlrpc_uri'. Maybe it points to the URL of primary server and 
that's why you get timeouts when it is down.


Re-setting it to the secondary server itself should fix it.

--
Martin^3 Babinsky

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] Gateway_timeout Error

[deepak dimri]

Hi All,

I have two IPA servers - primary and secondary running. the secondary ipa
server is installed using ipa replica image of primary.  While doing the
testing i realised that when i manually shut down my primary ipa server
making my secondary server to serve the UI. And now when i try to access
user or hosts details using my secondary server then i am getting below
error in the UI. I am able to login fine though; it is just that when i
double click on host objects then i get the error.
An error has occurred (GATEWAY_TIMEOUT)

I am still trying to troubleshoot as why i am getting timeout error but
thought of asking the group here to see if some one can share some pointers

Many Thanks,
Deepak
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project