Re: [Freeipa-users] Mostly working trust, SSH failure

2016-05-23 Thread Rob Crittenden

Erik Mackdanz wrote:

For the bug you mentioned ([1], downstream [2]), there is a patch but
it's not publicly accessible.  Are you able post the patch to this
list?  It may help us determine if we are directly affected.


https://lists.fedorahosted.org/archives/list/sssd-de...@lists.fedorahosted.org/thread/TUZ6ZWLRZ6QSMUHV44PRT75T6OVBGILK/

rob



Thanks,
Erik

[1] https://fedorahosted.org/sssd/ticket/3015
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1336688

On Sun, May 22, 2016 at 6:48 AM, Jakub Hrozek  wrote:



On 20 May 2016, at 19:31, Erik Mackdanz  wrote:

Thanks Jakub,

Yes, the "marking subdomain ... inactive" portion is below.

There are failures in resolving the Global Catalog via SRV, but what
I've read says that should be okay because we fall back to the
SID<->UID mapping.  With dig, I can reproduce sssd's finding that
those SRV records don't exist.  Is the DNS failure as fatal as it
appears?


Yes, I think that's the issue. I don't think we fall back to LDAP lookups. (btw 
we have a bug where we use the domain name, not the forest name for GC lookups 
SRV query..)



Yes, we can kinit AD users.  We can also 'getent' AD users and groups
(at least the group we authorized in our trust).

Does it matter that the user we used to establish the trust was later
demoted?  (Was domain admin, now regular user).

Cheers,
Erik


[ipa_srv_ad_acct_retried] (0x0400): Sudomain re-set, will retry lookup
[be_fo_reset_svc] (0x1000): Resetting all servers in service na.bazzlegroup.com
[set_srv_data_status] (0x0100): Marking SRV lookup of service
'na.bazzlegroup.com' as 'neutral'
[set_server_common_status] (0x0100): Marking server
'deda9w1004.na.bazzlegroup.com' as 'name not resolved'
[fo_set_port_status] (0x0100): Marking port 389 of server
'deda9w1004.na.bazzlegroup.com' as 'neutral'
[ad_user_data_cmp] (0x1000): Comparing LDAP with LDAP
[fo_set_port_status] (0x0400): Marking port 389 of duplicate server
'deda9w1004.na.bazzlegroup.com' as 'neutral'
[ad_user_data_cmp] (0x1000): Comparing LDAP with LDAP
[set_srv_data_status] (0x0100): Marking SRV lookup of service
'na.bazzlegroup.com' as 'neutral'
[set_server_common_status] (0x0100): Marking server
'usbe9w2003.na.bazzlegroup.com' as 'name not resolved'
[fo_set_port_status] (0x0100): Marking port 389 of server
'usbe9w2003.na.bazzlegroup.com' as 'neutral'
[ad_user_data_cmp] (0x1000): Comparing LDAP with LDAP
[ad_user_data_cmp] (0x1000): Comparing LDAP with LDAP
[fo_set_port_status] (0x0400): Marking port 389 of duplicate server
'usbe9w2003.na.bazzlegroup.com' as 'neutral'
[ipa_srv_ad_acct_lookup_step] (0x0400): Looking up AD account
[sdap_id_op_connect_step] (0x4000): beginning to connect
[fo_resolve_service_send] (0x0100): Trying to resolve service
'gc_na.bazzlegroup.com'
[get_port_status] (0x1000): Port status of port 0 for server '(no
name)' is 'not working'
[fo_resolve_service_send] (0x0020): No available servers for service
'gc_na.bazzlegroup.com'
[be_resolve_server_done] (0x1000): Server resolution failed: 5
[sdap_id_op_connect_done] (0x0400): Failed to connect to server, but
ignore mark offline is enabled.
[sdap_id_op_connect_done] (0x4000): notify error to op #1: 5
[Input/output error]
[be_mark_dom_offline] (0x1000): Marking subdomain na.bazzlegroup.com offline
[be_mark_subdom_offline] (0x1000): Marking subdomain
na.bazzlegroup.com as inactive
[ipa_srv_ad_acct_lookup_done] (0x0040): ipa_get_*_acct request failed:
[1432158262]: Subdomain is inactive.
[ipa_subdomain_account_done] (0x0040): ipa_get_*_acct request failed: 1432158262
[sdap_id_op_destroy] (0x4000): releasing operation connection

On Fri, May 20, 2016 at 2:02 AM, Jakub Hrozek  wrote:

On Thu, May 19, 2016 at 05:18:43PM -0500, Erik Mackdanz wrote:

Hello,

I've set up a one-way trust to an Active Directory domain.  Things
seem to roughly work, but something's missing.

Can any kind soul spot a problem with my configuration, or advise on
how to further troubleshoot?

Facts:

- An AD user gets 'Access denied' when SSH'ing by password to the
  FreeIPA host.  This is my concern.

- This AD user has not been locked out.

- getent passwd succeeds for the AD user

- A FreeIPA user can successfully SSH by password to the same FreeIPA
  host.

- That FreeIPA user can then successfully kinit as the AD user (the
  same AD user denied above)

- HBAC is set to the default allow_all rule, which is enabled.
  Running the HBAC Test tool on the AD user confirms that they are
  authorized for sshd.

This tells me something is awry in sssd.conf or sshd_config or pam.d
or HBAC.

Thanks,
Erik

I've got sssd debug to 9.  Here's some output:




[...]


(Thu May 19 20:43:34 2016) [sssd[be[platform.schlitz]]]
[ipa_srv_ad_acct_lookup_step] (0x0400): Looking up AD account
(Thu May 19 20:43:34 2016) [sssd[be[platform.schlitz]]]
[be_mark_dom_offline] (0x1000): Marking subdomain na.bazzlegroup.com
offline
(Thu May 19 20:43:34 2016) [sssd[be[platform.schlitz]]]
[be_mark_subdom_offline] (0x4000): Subdomain already inactive
(Th

Re: [Freeipa-users] increase the number of attempts to create /etc/krb5.keytab

2016-05-23 Thread Rob Crittenden

Ask Stack wrote:

Rob
Thanks for the reply.
I didn't find anything obvious in /var/log/dirsrv/slapd-/access and
errors  and /var/log/krb5kdc.log
Do you know which service is responsible for providing
"/etc/krb5.keytab" to the client?


It uses an LDAP extended operation so 389-ds. Any errors would be in the 
KDC log or, more likely, in the 389-ds logs.


rob



On Monday, May 23, 2016 2:57 PM, Rob Crittenden  wrote:


Ask Stack wrote:

 > My company's ipa-client-install fail very often. Debug logs show the
 > process always failed at getting the /etc/krb5.keytab .
 > Is there a way to modify the script to increase number of attempts to
 > create /etc/krb5.keytab ?
 >
 > I noticed "--kinit-attempts=KINIT_ATTEMPTS, number of attempts to obtain
 > host TGT (defaults to 5)." But it comes after setting up the
 > "/etc/krb5.keytab" file.
 > Thanks.
 >
 > server
 > ipa-server-3.0.0-47.el6_7.1.x86_64
 >
 > cleint
 > ipa-client-3.0.0-47.el6_7.2.x86_64
 > ipa-client-3.0.0-50.el6.1.x86_64
 >
 >
 > #SUCCESSFUL ATTEMPT
 >
 > \n
 > \n
 > \n
 > \n
 > \n
 > \n
 >
 > Keytab successfully retrieved and stored in: /etc/krb5.keytab
 > Certificate subject base is: O=TEST.COM
 >
 > 2016-05-23T14:40:49Z INFO Enrolled in IPA realm TEST.COM
 > 2016-05-23T14:40:49Z DEBUG args=kdestroy
 > 2016-05-23T14:40:49Z DEBUG stdout=
 > 2016-05-23T14:40:49Z DEBUG stderr=
 >
 >
 >
 > #FAILED ATTEMPT
 >
 > \n
 > \n
 > \n
 > \n
 > \n
 > \n
 >
 > ipa-getkeytab: ../../../libraries/libldap/extended.c:177:
 > ldap_parse_extended_result: Assertion `res != ((void *)0)' failed.
 > Certificate subject base is: O=TEST.COM
 >
 > 2016-05-23T14:37:08Z INFO Enrolled in IPA realm TEST.COM
 > 2016-05-23T14:37:08Z DEBUG args=kdestroy
 > 2016-05-23T14:37:08Z DEBUG stdout=
 > 2016-05-23T14:37:08Z DEBUG stderr=


There is no retry capability and in some cases would be impossible to
add (the one-time password case). Can you check /var/log/krb5kdc on the
IPA master it connected to, and the 389-ds access and errors logs as
well. Perhaps one of those will have more information on why things failed.

rob






--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project


Re: [Freeipa-users] Mostly working trust, SSH failure

2016-05-23 Thread Erik Mackdanz
For the bug you mentioned ([1], downstream [2]), there is a patch but
it's not publicly accessible.  Are you able post the patch to this
list?  It may help us determine if we are directly affected.

Thanks,
Erik

[1] https://fedorahosted.org/sssd/ticket/3015
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1336688

On Sun, May 22, 2016 at 6:48 AM, Jakub Hrozek  wrote:
>
>> On 20 May 2016, at 19:31, Erik Mackdanz  wrote:
>>
>> Thanks Jakub,
>>
>> Yes, the "marking subdomain ... inactive" portion is below.
>>
>> There are failures in resolving the Global Catalog via SRV, but what
>> I've read says that should be okay because we fall back to the
>> SID<->UID mapping.  With dig, I can reproduce sssd's finding that
>> those SRV records don't exist.  Is the DNS failure as fatal as it
>> appears?
>
> Yes, I think that's the issue. I don't think we fall back to LDAP lookups. 
> (btw we have a bug where we use the domain name, not the forest name for GC 
> lookups SRV query..)
>
>>
>> Yes, we can kinit AD users.  We can also 'getent' AD users and groups
>> (at least the group we authorized in our trust).
>>
>> Does it matter that the user we used to establish the trust was later
>> demoted?  (Was domain admin, now regular user).
>>
>> Cheers,
>> Erik
>>
>>
>> [ipa_srv_ad_acct_retried] (0x0400): Sudomain re-set, will retry lookup
>> [be_fo_reset_svc] (0x1000): Resetting all servers in service 
>> na.bazzlegroup.com
>> [set_srv_data_status] (0x0100): Marking SRV lookup of service
>> 'na.bazzlegroup.com' as 'neutral'
>> [set_server_common_status] (0x0100): Marking server
>> 'deda9w1004.na.bazzlegroup.com' as 'name not resolved'
>> [fo_set_port_status] (0x0100): Marking port 389 of server
>> 'deda9w1004.na.bazzlegroup.com' as 'neutral'
>> [ad_user_data_cmp] (0x1000): Comparing LDAP with LDAP
>> [fo_set_port_status] (0x0400): Marking port 389 of duplicate server
>> 'deda9w1004.na.bazzlegroup.com' as 'neutral'
>> [ad_user_data_cmp] (0x1000): Comparing LDAP with LDAP
>> [set_srv_data_status] (0x0100): Marking SRV lookup of service
>> 'na.bazzlegroup.com' as 'neutral'
>> [set_server_common_status] (0x0100): Marking server
>> 'usbe9w2003.na.bazzlegroup.com' as 'name not resolved'
>> [fo_set_port_status] (0x0100): Marking port 389 of server
>> 'usbe9w2003.na.bazzlegroup.com' as 'neutral'
>> [ad_user_data_cmp] (0x1000): Comparing LDAP with LDAP
>> [ad_user_data_cmp] (0x1000): Comparing LDAP with LDAP
>> [fo_set_port_status] (0x0400): Marking port 389 of duplicate server
>> 'usbe9w2003.na.bazzlegroup.com' as 'neutral'
>> [ipa_srv_ad_acct_lookup_step] (0x0400): Looking up AD account
>> [sdap_id_op_connect_step] (0x4000): beginning to connect
>> [fo_resolve_service_send] (0x0100): Trying to resolve service
>> 'gc_na.bazzlegroup.com'
>> [get_port_status] (0x1000): Port status of port 0 for server '(no
>> name)' is 'not working'
>> [fo_resolve_service_send] (0x0020): No available servers for service
>> 'gc_na.bazzlegroup.com'
>> [be_resolve_server_done] (0x1000): Server resolution failed: 5
>> [sdap_id_op_connect_done] (0x0400): Failed to connect to server, but
>> ignore mark offline is enabled.
>> [sdap_id_op_connect_done] (0x4000): notify error to op #1: 5
>> [Input/output error]
>> [be_mark_dom_offline] (0x1000): Marking subdomain na.bazzlegroup.com offline
>> [be_mark_subdom_offline] (0x1000): Marking subdomain
>> na.bazzlegroup.com as inactive
>> [ipa_srv_ad_acct_lookup_done] (0x0040): ipa_get_*_acct request failed:
>> [1432158262]: Subdomain is inactive.
>> [ipa_subdomain_account_done] (0x0040): ipa_get_*_acct request failed: 
>> 1432158262
>> [sdap_id_op_destroy] (0x4000): releasing operation connection
>>
>> On Fri, May 20, 2016 at 2:02 AM, Jakub Hrozek  wrote:
>>> On Thu, May 19, 2016 at 05:18:43PM -0500, Erik Mackdanz wrote:
 Hello,

 I've set up a one-way trust to an Active Directory domain.  Things
 seem to roughly work, but something's missing.

 Can any kind soul spot a problem with my configuration, or advise on
 how to further troubleshoot?

 Facts:

 - An AD user gets 'Access denied' when SSH'ing by password to the
  FreeIPA host.  This is my concern.

 - This AD user has not been locked out.

 - getent passwd succeeds for the AD user

 - A FreeIPA user can successfully SSH by password to the same FreeIPA
  host.

 - That FreeIPA user can then successfully kinit as the AD user (the
  same AD user denied above)

 - HBAC is set to the default allow_all rule, which is enabled.
  Running the HBAC Test tool on the AD user confirms that they are
  authorized for sshd.

 This tells me something is awry in sssd.conf or sshd_config or pam.d
 or HBAC.

 Thanks,
 Erik

 I've got sssd debug to 9.  Here's some output:


>>>
>>> [...]
>>>
 (Thu May 19 20:43:34 2016) [sssd[be[platform.schlitz]]]
 [ipa_srv_ad_acct_lookup_step] (0x0400): Looking up AD account
 (Thu May 19 20:43:

Re: [Freeipa-users] increase the number of attempts to create /etc/krb5.keytab

2016-05-23 Thread Ask Stack
RobThanks for the reply. 
I didn't find anything obvious in /var/log/dirsrv/slapd-/access and errors  and 
/var/log/krb5kdc.log Do you know which service is responsible for providing  
"/etc/krb5.keytab" to the client?


On Monday, May 23, 2016 2:57 PM, Rob Crittenden  wrote:
 

 Ask Stack wrote:
> My company's ipa-client-install fail very often. Debug logs show the
> process always failed at getting the /etc/krb5.keytab .
> Is there a way to modify the script to increase number of attempts to
> create /etc/krb5.keytab ?
>
> I noticed "--kinit-attempts=KINIT_ATTEMPTS, number of attempts to obtain
> host TGT (defaults to 5)." But it comes after setting up the
> "/etc/krb5.keytab" file.
> Thanks.
>
> server
> ipa-server-3.0.0-47.el6_7.1.x86_64
>
> cleint
> ipa-client-3.0.0-47.el6_7.2.x86_64
> ipa-client-3.0.0-50.el6.1.x86_64
>
>
> #SUCCESSFUL ATTEMPT
>
> \n
> \n
> \n
> \n
> \n
> \n
>
> Keytab successfully retrieved and stored in: /etc/krb5.keytab
> Certificate subject base is: O=TEST.COM
>
> 2016-05-23T14:40:49Z INFO Enrolled in IPA realm TEST.COM
> 2016-05-23T14:40:49Z DEBUG args=kdestroy
> 2016-05-23T14:40:49Z DEBUG stdout=
> 2016-05-23T14:40:49Z DEBUG stderr=
>
>
>
> #FAILED ATTEMPT
>
> \n
> \n
> \n
> \n
> \n
> \n
>
> ipa-getkeytab: ../../../libraries/libldap/extended.c:177:
> ldap_parse_extended_result: Assertion `res != ((void *)0)' failed.
> Certificate subject base is: O=TEST.COM
>
> 2016-05-23T14:37:08Z INFO Enrolled in IPA realm TEST.COM
> 2016-05-23T14:37:08Z DEBUG args=kdestroy
> 2016-05-23T14:37:08Z DEBUG stdout=
> 2016-05-23T14:37:08Z DEBUG stderr=

There is no retry capability and in some cases would be impossible to 
add (the one-time password case). Can you check /var/log/krb5kdc on the 
IPA master it connected to, and the 389-ds access and errors logs as 
well. Perhaps one of those will have more information on why things failed.

rob



  -- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] increase the number of attempts to create /etc/krb5.keytab

2016-05-23 Thread Rob Crittenden

Ask Stack wrote:

My company's ipa-client-install fail very often. Debug logs show the
process always failed at getting the /etc/krb5.keytab .
Is there a way to modify the script to increase number of attempts to
create /etc/krb5.keytab ?

I noticed "--kinit-attempts=KINIT_ATTEMPTS, number of attempts to obtain
host TGT (defaults to 5)." But it comes after setting up the
"/etc/krb5.keytab" file.
Thanks.

server
ipa-server-3.0.0-47.el6_7.1.x86_64

cleint
ipa-client-3.0.0-47.el6_7.2.x86_64
ipa-client-3.0.0-50.el6.1.x86_64


#SUCCESSFUL ATTEMPT

\n
\n
\n
\n
\n
\n

Keytab successfully retrieved and stored in: /etc/krb5.keytab
Certificate subject base is: O=TEST.COM

2016-05-23T14:40:49Z INFO Enrolled in IPA realm TEST.COM
2016-05-23T14:40:49Z DEBUG args=kdestroy
2016-05-23T14:40:49Z DEBUG stdout=
2016-05-23T14:40:49Z DEBUG stderr=



#FAILED ATTEMPT

\n
\n
\n
\n
\n
\n

ipa-getkeytab: ../../../libraries/libldap/extended.c:177:
ldap_parse_extended_result: Assertion `res != ((void *)0)' failed.
Certificate subject base is: O=TEST.COM

2016-05-23T14:37:08Z INFO Enrolled in IPA realm TEST.COM
2016-05-23T14:37:08Z DEBUG args=kdestroy
2016-05-23T14:37:08Z DEBUG stdout=
2016-05-23T14:37:08Z DEBUG stderr=


There is no retry capability and in some cases would be impossible to 
add (the one-time password case). Can you check /var/log/krb5kdc on the 
IPA master it connected to, and the 389-ds access and errors logs as 
well. Perhaps one of those will have more information on why things failed.


rob

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project


[Freeipa-users] increase the number of attempts to create /etc/krb5.keytab

2016-05-23 Thread Ask Stack
My company's ipa-client-install fail very often. Debug logs show the process 
always failed at getting the /etc/krb5.keytab .
Is there a way to modify the script to increase number of attempts to create 
/etc/krb5.keytab ?
I noticed "--kinit-attempts=KINIT_ATTEMPTS, number of attempts to obtain host 
TGT (defaults to 5)." But it comes after setting up the "/etc/krb5.keytab" 
file. 
Thanks.

server
ipa-server-3.0.0-47.el6_7.1.x86_64

cleint
ipa-client-3.0.0-47.el6_7.2.x86_64
ipa-client-3.0.0-50.el6.1.x86_64


#SUCCESSFUL ATTEMPT

\n
\n
\n
\n
\n
\n

Keytab successfully retrieved and stored in: /etc/krb5.keytab
Certificate subject base is: O=TEST.COM

2016-05-23T14:40:49Z INFO Enrolled in IPA realm TEST.COM
2016-05-23T14:40:49Z DEBUG args=kdestroy
2016-05-23T14:40:49Z DEBUG stdout=
2016-05-23T14:40:49Z DEBUG stderr=



#FAILED ATTEMPT

\n
\n
\n
\n
\n
\n

ipa-getkeytab: ../../../libraries/libldap/extended.c:177: 
ldap_parse_extended_result: Assertion `res != ((void *)0)' failed.
Certificate subject base is: O=TEST.COM

2016-05-23T14:37:08Z INFO Enrolled in IPA realm TEST.COM
2016-05-23T14:37:08Z DEBUG args=kdestroy
2016-05-23T14:37:08Z DEBUG stdout=
2016-05-23T14:37:08Z DEBUG stderr=

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] What id my AD domain user password not available

2016-05-23 Thread Ben .T.George
HI

in my case i have 2 domains

AD DNS : corp.example.kw.com
main DNS ( from appliance) : kw.example.com

and all the linux box are pointed to kw.example.com

so i put my IPA server hostname as : ipa.kw.example.com and created A & PTR
on kw.example.com

is that the correct way?

Regards,
Ben

On Mon, May 23, 2016 at 8:20 PM, Michael ORourke 
wrote:

> Ben,
>
> Yes, that is a requirement.  Just creating the A & PTR records for you
> FreeIPA server is not enough.  You will need to keep the DNS zones separate
> too, example:
> Windows AD Domain: mydomain.com
> FreeIPA Realm/Domain: subdomain.mydomain.com
>
> You cannot have a cross-forest trust between two domains with the same DNS
> zone name.  So if you have a flat DNS namespace, then you will want to plan
> accordingly to move all the linux boxes that will participate in the
> FreeIPA domain into the new DNS zone.
>
> -Mike
>
> -Original Message-
> From: "Ben .T.George"
> Sent: May 23, 2016 10:44 AM
> To: Michael ORourke
> Cc: freeipa-users
> Subject: Re: [Freeipa-users] What id my AD domain user password not
> available
>
> HI
>
> yea that GIf screen i shared with him. but that doesn't show how to take
> shared key.
>
> In my case DNS is handled by 3rd party appliances and from their side they
> created A record for my IPA server. bth forward and reverse is working
>
> is this forwader is mandatory thing from DNS side?
>
> Regards,
> ben
>
> On Mon, May 23, 2016 at 5:31 PM, Michael ORourke 
> wrote:
>
>> Actually one of his questions doesn't make sense, because last I checked,
>> normal domain users do not have permissions to create a forest trust.
>> I believe the default is a one-way trust, so maybe his concerns about the
>> bi-directional trust is really a non-issue.
>> If he refuses to type in the admin password in a linux console session
>> (extreme paranoia?), then perhaps you could give him a link to the tutorial
>> on using a pre-shared key and have him setup the AD side and give you the
>> key.  You don't have to be a Windows expert to do this, just ask your
>> domain admin to do the steps for you.  Also, you will need to setup a
>> separate DNS zone and some forwarding rules.  Otherwise you are going to
>> have problems.
>>
>> -Mike
>>
>>
>> -Original Message-
>> From: "Ben .T.George"
>> Sent: May 23, 2016 10:07 AM
>> To: Michael ORourke
>> Cc: freeipa-users
>> Subject: Re: [Freeipa-users] What id my AD domain user password not
>> available
>>
>> HI
>>
>> He is local only but he is asking so many questions.
>>
>> first of all he is refusing to give domain admin users password .
>>
>> questions he is asking is:
>>
>> Is this trust relationship is two directional? If, yes why IPA require
>> two directional trust?
>> can we build this trust one directional?
>> can we achieve this with normal domain user?
>>
>> and hs is opposing to enter password in command line and i was going
>> though the rust using a pre-shared key and its too hard for me to
>> understand as i have no windows experience
>>
>> regards,
>> Ben
>>
>> On Mon, May 23, 2016 at 4:22 PM, Michael ORourke > > wrote:
>>
>>> A couple of ways to go about this.  If he is local to you, you could
>>> explain that you need to establish a trust with his domain and you need his
>>> assistance for a few minutes while you type the command to join, then have
>>> him type in the password.  You need to assure that the DNS forward/stub
>>> zones are setup and working too.  If he is remote, you could use some
>>> screen share software and share out your desktop and walk him through the
>>> part where he has to type the admin password.  There is also a way to
>>> create a trust using a pre-shared key.  That may be more acceptable to
>>> him.
>>>
>>> -Mike
>>>
>>> -Original Message-
>>> From: "Ben .T.George"
>>> Sent: May 23, 2016 8:42 AM
>>> To: freeipa-users
>>> Subject: [Freeipa-users] What id my AD domain user password not
>>> available
>>>
>>> Hi LIst,
>>>
>>> my Windows domain Admin is not giving domain admin user password.
>>>
>>> in this case how can i proceed ipa trust-add
>>>
>>> regards,
>>> Ben
>>>
>>>
>>> --
>>> Manage your subscription for the Freeipa-users mailing list:
>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>> Go to http://freeipa.org for more info on the project
>>>
>>
>>
>> --
>> Manage your subscription for the Freeipa-users mailing list:
>> https://www.redhat.com/mailman/listinfo/freeipa-users
>> Go to http://freeipa.org for more info on the project
>>
>
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] FreeIPA 4.3 with PWM 1.7 ?

2016-05-23 Thread Zak Wolfinger
Does anyone have this combo working?  I’m running into problems with pki-tomcat 
and tomcat for pwm conflicting and need some pointers.

Thanks!


signature.asc
Description: Message signed with OpenPGP using GPGMail
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] What id my AD domain user password not available

2016-05-23 Thread Michael ORourke
Ben,Yes, that is a requirement.  Just creating the A & PTR records for you FreeIPA server is not enough.  You will need to keep the DNS zones separate too, example:Windows AD Domain: mydomain.comFreeIPA Realm/Domain: subdomain.mydomain.comYou cannot have a cross-forest trust between two domains with the same DNS zone name.  So if you have a flat DNS namespace, then you will want to plan accordingly to move all the linux boxes that will participate in the FreeIPA domain into the new DNS zone.-Mike-Original Message-
From: "Ben .T.George" 
Sent: May 23, 2016 10:44 AM
To: Michael ORourke 
Cc: freeipa-users 
Subject: Re: [Freeipa-users] What id my AD domain user password not available

HIyea that GIf screen i shared with him. but that doesn't show how to take shared key.In my case DNS is handled by 3rd party appliances and from their side they created A record for my IPA server. bth forward and reverse is working is this forwader is mandatory thing from DNS side?Regards,benOn Mon, May 23, 2016 at 5:31 PM, Michael ORourke  wrote:Actually one of his questions doesn't make sense, because last I checked, normal domain users do not have permissions to create a forest trust.I believe the default is a one-way trust, so maybe his concerns about the bi-directional trust is really a non-issue.If he refuses to type in the admin password in a linux console session (extreme paranoia?), then perhaps you could give him a link to the tutorial on using a pre-shared key and have him setup the AD side and give you the key.  You don't have to be a Windows expert to do this, just ask your domain admin to do the steps for you.  Also, you will need to setup a separate DNS zone and some forwarding rules.  Otherwise you are going to have problems.-Mike -Original Message-
From: "Ben .T.George" 
Sent: May 23, 2016 10:07 AM
To: Michael ORourke 
Cc: freeipa-users 
Subject: Re: [Freeipa-users] What id my AD domain user password not available

HIHe is local only but he is asking so many questions.first of all he is refusing to give domain admin users password .questions he is asking is:Is this trust relationship is two directional? If, yes why IPA require two directional trust?can we build this trust one directional?can we achieve this with normal domain user?and hs is opposing to enter password in command line and i was going though the rust using a pre-shared key and its too hard for me to understand as i have no windows experienceregards,BenOn Mon, May 23, 2016 at 4:22 PM, Michael ORourke  wrote:A couple of ways to go about this.  If he is local to you, you could explain that you need to establish a trust with his domain and you need his assistance for a few minutes while you type the command to join, then have him type in the password.  You need to assure that the DNS forward/stub zones are setup and working too.  If he is remote, you could use some screen share software and share out your desktop and walk him through the part where he has to type the admin password.  There is also a way to create a trust using a pre-shared key.  That may be more acceptable to him.  -Mike-Original Message-
From: "Ben .T.George" 
Sent: May 23, 2016 8:42 AM
To: freeipa-users 
Subject: [Freeipa-users] What id my AD domain user password not available

Hi LIst,my Windows domain Admin is not giving domain admin user password.in this case how can i proceed ipa trust-addregards,Ben


--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project


--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project


-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] AD replication and password passthrough

2016-05-23 Thread Redmond, Stacy
Is there a way to setup replication from AD, and just use passthrough to AD for 
passwords, vs having to synchronize passwords.  I am getting a lot of pushback 
from the AD team on installing the password sync software due to issues in the 
past.  I would like to setup replication, but still use AD to authenticate 
passwords.
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] question about automount config

2016-05-23 Thread Arthur Fayzullin
Good day, colleagues!
I am confused about how automount work and howto configure it. I have
tried to configure it according to
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/index.html
document (paragraph 9.1.1 and chapter 20).
I have tried to make it work on 3 servers:
1. ipa server;
2. nfs server (node00);
3. nfs client (postgres).


*** so here how it configured on ipa server:
$ ipa automountlocation-tofiles amantai
/etc/auto.master:
/-  /etc/auto.direct
/home   /etc/auto.home
---
/etc/auto.direct:
---
/etc/auto.home:
*   -sec=kr5i,rw,fstype=nfs4 node00.glavsn.ab:/home/&

maps not connected to /etc/auto.master:

$ ipa service-find nfs
--
2 services matched
--
  Основной: nfs/node00.glavsn...@glavsn.ab
  Keytab: True
  Managed by: node00.glavsn.ab

  Основной: nfs/postgres.glavsn...@glavsn.ab
  Keytab: True
  Managed by: postgres.glavsn.ab


*** here is nfs server config:
$ sudo klist -k
Пароль:
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal

--
   1 host/node00.glavsn...@glavsn.ab
   1 host/node00.glavsn...@glavsn.ab
   1 host/node00.glavsn...@glavsn.ab
   1 host/node00.glavsn...@glavsn.ab
   2 nfs/node00.glavsn...@glavsn.ab
   2 nfs/node00.glavsn...@glavsn.ab
   2 nfs/node00.glavsn...@glavsn.ab
   2 nfs/node00.glavsn...@glavsn.ab

$ cat /etc/exports
/home *(rw,sec=sys:krb5:krb5i:krb5p)

$ sudo firewall-cmd --list-all
public (default, active)
  interfaces: bridge0 enp1s0
  sources:
  services: dhcpv6-client nfs ssh
  ports: 8001/tcp
  masquerade: no
  forward-ports:
  icmp-blocks:
  rich rules:

$ getenforce
Enforcing


*** here nfs client config:
# klist -k
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal

--
   1 host/postgres.glavsn...@glavsn.ab
   1 host/postgres.glavsn...@glavsn.ab
   1 host/postgres.glavsn...@glavsn.ab
   1 host/postgres.glavsn...@glavsn.ab
   1 nfs/postgres.glavsn...@glavsn.ab
   1 nfs/postgres.glavsn...@glavsn.ab
   1 nfs/postgres.glavsn...@glavsn.ab
   1 nfs/postgres.glavsn...@glavsn.ab

# firewall-cmd --list-all
FedoraServer (default, active)
  interfaces: ens3
  sources:
  services: cockpit dhcpv6-client ssh
  ports:
  protocols:
  masquerade: no
  forward-ports:
  icmp-blocks:
  rich rules:

# mount -l  (contains next string)
auto.home on /home type autofs
(rw,relatime,fd=25,pgrp=960,timeout=300,minproto=5,maxproto=5,indirect)

# ll /home/afayzullin
ls says that it cannot access /home/afayzullin: no such file or directory

I have run
# ipa-client-automount --location=amantai
on client and it has completed successfully.

I have tried to disable selinux, drop iptables rules. And now I am
little confused about what to do next. May if someone has faced with
automount config can give me some advice, or if there is any howto
config automount, or some can advise howto debug this situation?

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] What id my AD domain user password not available

2016-05-23 Thread Ben .T.George
HI

yea that GIf screen i shared with him. but that doesn't show how to take
shared key.

In my case DNS is handled by 3rd party appliances and from their side they
created A record for my IPA server. bth forward and reverse is working

is this forwader is mandatory thing from DNS side?

Regards,
ben

On Mon, May 23, 2016 at 5:31 PM, Michael ORourke 
wrote:

> Actually one of his questions doesn't make sense, because last I checked,
> normal domain users do not have permissions to create a forest trust.
> I believe the default is a one-way trust, so maybe his concerns about the
> bi-directional trust is really a non-issue.
> If he refuses to type in the admin password in a linux console session
> (extreme paranoia?), then perhaps you could give him a link to the tutorial
> on using a pre-shared key and have him setup the AD side and give you the
> key.  You don't have to be a Windows expert to do this, just ask your
> domain admin to do the steps for you.  Also, you will need to setup a
> separate DNS zone and some forwarding rules.  Otherwise you are going to
> have problems.
>
> -Mike
>
>
> -Original Message-
> From: "Ben .T.George"
> Sent: May 23, 2016 10:07 AM
> To: Michael ORourke
> Cc: freeipa-users
> Subject: Re: [Freeipa-users] What id my AD domain user password not
> available
>
> HI
>
> He is local only but he is asking so many questions.
>
> first of all he is refusing to give domain admin users password .
>
> questions he is asking is:
>
> Is this trust relationship is two directional? If, yes why IPA require two
> directional trust?
> can we build this trust one directional?
> can we achieve this with normal domain user?
>
> and hs is opposing to enter password in command line and i was going
> though the rust using a pre-shared key and its too hard for me to
> understand as i have no windows experience
>
> regards,
> Ben
>
> On Mon, May 23, 2016 at 4:22 PM, Michael ORourke 
> wrote:
>
>> A couple of ways to go about this.  If he is local to you, you could
>> explain that you need to establish a trust with his domain and you need his
>> assistance for a few minutes while you type the command to join, then have
>> him type in the password.  You need to assure that the DNS forward/stub
>> zones are setup and working too.  If he is remote, you could use some
>> screen share software and share out your desktop and walk him through the
>> part where he has to type the admin password.  There is also a way to
>> create a trust using a pre-shared key.  That may be more acceptable to
>> him.
>>
>> -Mike
>>
>> -Original Message-
>> From: "Ben .T.George"
>> Sent: May 23, 2016 8:42 AM
>> To: freeipa-users
>> Subject: [Freeipa-users] What id my AD domain user password not available
>>
>> Hi LIst,
>>
>> my Windows domain Admin is not giving domain admin user password.
>>
>> in this case how can i proceed ipa trust-add
>>
>> regards,
>> Ben
>>
>>
>> --
>> Manage your subscription for the Freeipa-users mailing list:
>> https://www.redhat.com/mailman/listinfo/freeipa-users
>> Go to http://freeipa.org for more info on the project
>>
>
>
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] What id my AD domain user password not available

2016-05-23 Thread Michael ORourke
Actually one of his questions doesn't make sense, because last I checked, normal domain users do not have permissions to create a forest trust.I believe the default is a one-way trust, so maybe his concerns about the bi-directional trust is really a non-issue.If he refuses to type in the admin password in a linux console session (extreme paranoia?), then perhaps you could give him a link to the tutorial on using a pre-shared key and have him setup the AD side and give you the key.  You don't have to be a Windows expert to do this, just ask your domain admin to do the steps for you.  Also, you will need to setup a separate DNS zone and some forwarding rules.  Otherwise you are going to have problems.-Mike -Original Message-
From: "Ben .T.George" 
Sent: May 23, 2016 10:07 AM
To: Michael ORourke 
Cc: freeipa-users 
Subject: Re: [Freeipa-users] What id my AD domain user password not available

HIHe is local only but he is asking so many questions.first of all he is refusing to give domain admin users password .questions he is asking is:Is this trust relationship is two directional? If, yes why IPA require two directional trust?can we build this trust one directional?can we achieve this with normal domain user?and hs is opposing to enter password in command line and i was going though the rust using a pre-shared key and its too hard for me to understand as i have no windows experienceregards,BenOn Mon, May 23, 2016 at 4:22 PM, Michael ORourke  wrote:A couple of ways to go about this.  If he is local to you, you could explain that you need to establish a trust with his domain and you need his assistance for a few minutes while you type the command to join, then have him type in the password.  You need to assure that the DNS forward/stub zones are setup and working too.  If he is remote, you could use some screen share software and share out your desktop and walk him through the part where he has to type the admin password.  There is also a way to create a trust using a pre-shared key.  That may be more acceptable to him.  -Mike-Original Message-
From: "Ben .T.George" 
Sent: May 23, 2016 8:42 AM
To: freeipa-users 
Subject: [Freeipa-users] What id my AD domain user password not available

Hi LIst,my Windows domain Admin is not giving domain admin user password.in this case how can i proceed ipa trust-addregards,Ben


--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project


-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] What id my AD domain user password not available

2016-05-23 Thread Ben .T.George
HI

He is local only but he is asking so many questions.

first of all he is refusing to give domain admin users password .

questions he is asking is:

Is this trust relationship is two directional? If, yes why IPA require two
directional trust?
can we build this trust one directional?
can we achieve this with normal domain user?

and hs is opposing to enter password in command line and i was going though
the rust using a pre-shared key and its too hard for me to understand as i
have no windows experience

regards,
Ben

On Mon, May 23, 2016 at 4:22 PM, Michael ORourke 
wrote:

> A couple of ways to go about this.  If he is local to you, you could
> explain that you need to establish a trust with his domain and you need his
> assistance for a few minutes while you type the command to join, then have
> him type in the password.  You need to assure that the DNS forward/stub
> zones are setup and working too.  If he is remote, you could use some
> screen share software and share out your desktop and walk him through the
> part where he has to type the admin password.  There is also a way to
> create a trust using a pre-shared key.  That may be more acceptable to
> him.
>
> -Mike
>
> -Original Message-
> From: "Ben .T.George"
> Sent: May 23, 2016 8:42 AM
> To: freeipa-users
> Subject: [Freeipa-users] What id my AD domain user password not available
>
> Hi LIst,
>
> my Windows domain Admin is not giving domain admin user password.
>
> in this case how can i proceed ipa trust-add
>
> regards,
> Ben
>
>
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] What id my AD domain user password not available

2016-05-23 Thread Michael ORourke
A couple of ways to go about this.  If he is local to you, you could explain that you need to establish a trust with his domain and you need his assistance for a few minutes while you type the command to join, then have him type in the password.  You need to assure that the DNS forward/stub zones are setup and working too.  If he is remote, you could use some screen share software and share out your desktop and walk him through the part where he has to type the admin password.  There is also a way to create a trust using a pre-shared key.  That may be more acceptable to him.  -Mike-Original Message-
From: "Ben .T.George" 
Sent: May 23, 2016 8:42 AM
To: freeipa-users 
Subject: [Freeipa-users] What id my AD domain user password not available

Hi LIst,my Windows domain Admin is not giving domain admin user password.in this case how can i proceed ipa trust-addregards,Ben


-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] What id my AD domain user password not available

2016-05-23 Thread Ben .T.George
Hi

Thanks for your reply.

I saw this before but the thing is i cant able to follow up this one as i
am not completely getting those steps

ipa trust-add --type=ad "ad_domain" --trust-secret

Is asking for key and what i need to gave ?

And the shown gif screens and current AD windows are different for me.

Regards
Ben
On 23 May 2016 16:13, "Martin Babinsky"  wrote:

> On 05/23/2016 02:42 PM, Ben .T.George wrote:
>
>> Hi LIst,
>>
>> my Windows domain Admin is not giving domain admin user password.
>>
>> in this case how can i proceed ipa trust-add
>>
>> regards,
>> Ben
>>
>>
>>
> Hi Ben,
>
> You can ask your AD domain admin to create a shared secret for
> establishing trust. See the corresponding chapter in the guide for creating
> trusts[1] for more details.
>
> [1]
> http://www.freeipa.org/page/Active_Directory_trust_setup#When_AD_administrator_credentials_aren.27t_available
>
> --
> Martin^3 Babinsky
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] What id my AD domain user password not available

2016-05-23 Thread Martin Babinsky

On 05/23/2016 02:42 PM, Ben .T.George wrote:

Hi LIst,

my Windows domain Admin is not giving domain admin user password.

in this case how can i proceed ipa trust-add

regards,
Ben




Hi Ben,

You can ask your AD domain admin to create a shared secret for 
establishing trust. See the corresponding chapter in the guide for 
creating trusts[1] for more details.


[1] 
http://www.freeipa.org/page/Active_Directory_trust_setup#When_AD_administrator_credentials_aren.27t_available 



--
Martin^3 Babinsky

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project


[Freeipa-users] What id my AD domain user password not available

2016-05-23 Thread Ben .T.George
Hi LIst,

my Windows domain Admin is not giving domain admin user password.

in this case how can i proceed ipa trust-add

regards,
Ben
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] LDAP server failover via altServer attribute?

2016-05-23 Thread Răzvan Corneliu C.R. VILT
Hi Guillermo,

In February I published my findings for switching IPA in OpenDirectory 
compatible mode. See:
https://www.redhat.com/archives/freeipa-users/2016-February/msg00059.html 

Start by reading that thread.

More recently, Stefan Zecevic picked this up and opened up some interesting 
test cases for the setup in this thread:
https://www.redhat.com/archives/freeipa-users/2016-May/msg00310.html 


There's also a ticket for implementing these changes in IPA 4.4 
.

I'm willing to invest 4 hours per week into this if anyone else joins.

I have VMware virtual machines for every x86 OS X release possible (from Tiger 
to El Capitan) and for historical reasons I also have a few PPC releases in 
QEMU format.

I can host the VMs on a server but I need some help configuring the 389 
directory server plugins to automatically generate the needed extra attributes 
(authAuthority and altSecurityIdentities). I personally think that cn=config 
should be also automatically generated.

Cheers,
Răzvan


> On 22 mai 2016, at 21:31, Guillermo Fuentes 
>  wrote:
> 
> This is great info Razvan. Thanks for sharing it!
> We provision Macs by pushing configuration scripts via Munki.
> Can you point me where I can find more documentation about this?
> Thanks again,
> Guillermo
> 
> On Fri, May 20, 2016 at 3:45 PM, "Răzvan Corneliu C.R. VILT" 
> mailto:razvan.v...@me.com>> wrote:
> Hi guys,
> 
> Regarding the Macs, there are a few notes:
> 
> 1) The template kerberos setup can be pushed through LDAP (cn=KerberosClient 
> and cn=KerberosKDC,cn=config)
> 2) The LDAP replicas can be also configured in cn=config and it is cached by 
> OpenDirectory in the following format:
> 
> dn: cn=ldapreplicas, cn=config, dc=example, dc=com
> objectClass: apple-configuration
> apple-ldap-replica: ldap://192.168.1.1 <>
> apple-ldap-replica: ldap://192.168.2.2 <>
> apple-ldap-writable-replica: ldap://192.168.1.1 <>
> apple-ldap-writable-replica: ldap://192.168.2.2 <>
> apple-xml-plist: base64 encode of:
> -
> 
>  "http://www.apple.com/DTDs/PropertyList-1.0.dtd 
> ">
> 
> 
>   GUID
>   01234567-89AB-CDEF-0123-456789ABCDEF
>   IPaddresses
>   
>   192.168.1.1
> 10.0.0.1
>   
>   PrimaryMaster
>   ipa-server.example.org 
>   ReplicaName
>   Master
>   Replicas
>   
>ipa-bkserver.example.org 
> 
> 
>
> 
> 
> --
> 
> 3) The main problem with FreeIPA and Mac OS X comes from the SSL part (CRL 
> and/or OCSP are enforced). IPA refuses PLAIN authentication on SSL.
> 
> 
> If you do this manually instead of OpenDirectory compatible way, your machine 
> doesn't create an account for itself in IPA so service access without login 
> are not available, it doesn't download the root CA automatically and you 
> don't get SSO out of the box.
> 
> 
>> On 20 mai 2016, at 22:13, Guillermo Fuentes 
>> > > wrote:
>> 
>> SRV record failover works for Kerberos on the Mac. Setting "dns_lookup_kdc = 
>> yes" and removing the KDC server ("kdc = xxx") entries from the 
>> /Library/Preferences/edu.mit.Kerberos config file does the trick.
>> 
>> For LDAP, although you can enable it, I can't see it documented anywhere so 
>> I'm assuming that isn't the recommended way for the Mac. This can be enabled 
>> by running this for the LDAP server you're using:
>> sudo odutil set configuration /LDAPv3/ipa1.example.com 
>>  module ldap option "Use DNS replicas" "true"
>> 
>> Adding the altServer values with the Directory Manager credentials worked 
>> and I'm happy to report that the failover on the Mac works great with 
>> FreeIPA!
>> 
>> As suggested by Rob, for three servers, on server ipa1:
>> $ ldapmodify -x -D 'cn=directory manager' -W
>> Enter LDAP Password:
>> dn:
>> changetype: modify
>> add: altServer
>> altServer: ldap://ipa2.example.com 
>> -
>> add: altServer
>> altServer: ldap://ipa3.example.com 
>> 
>> modifying entry ""
>> ^D
>> 
>> The altServer values didn't replicate so I had to add them to each of the 
>> FreeIPA servers.
>> 
>> Then, tell the Mac (testing on OS X v10.11.5) to use the altServer attribute 
>> to look for replicas in case of failover: 
>> sudo odutil set configuration /LDAPv3/ipa1.example.com 
>>  module ldap option "Use altServer replicas" "true"
>> 
>> And, viola! Highly available authentication with a FreeIPA cluster for the 
>> Mac!
>> 
>> Thanks so much for your help!
>> Guillermo
>> 
>> 
>> On Fri, May 20, 2016 at 10:38 AM, Rob Crittenden >