I tested on HDP 2.6.4 and it works as expected, log file below shows that the
file gets updated, and I verified on disk my changed content was there:
2018-07-13 15:46:36,199 - Generating config:
/usr/hdp/current/knox-server/conf/gateway-site.xml
2018-07-13 15:46:36,199 -
File['/usr/hdp/current/knox-server/conf/gateway-site.xml'] {'owner': 'knox',
'content': InlineTemplate(...), 'group': 'knox', 'mode': None, 'encoding':
'UTF-8'}
2018-07-13 15:46:36,208 -
File['/usr/hdp/current/knox-server/conf/gateway-log4j.properties'] {'content':
InlineTemplate(...), 'owner': 'knox', 'group': 'knox', 'mode': 0644}
2018-07-13 15:46:36,217 -
File['/usr/hdp/current/knox-server/conf/topologies/default.xml'] {'content':
InlineTemplate(...), 'owner': 'knox', 'group': 'knox'}
2018-07-13 15:46:36,219 -
File['/usr/hdp/current/knox-server/conf/topologies/admin.xml'] {'content':
InlineTemplate(...), 'owner': 'knox', 'group': 'knox'}
2018-07-13 15:46:36,226 -
File['/usr/hdp/current/knox-server/conf/topologies/knoxsso.xml'] {'content':
InlineTemplate(...), 'owner': 'knox', 'group': 'knox'}
2018-07-13 15:46:36,227 - Writing
File['/usr/hdp/current/knox-server/conf/topologies/knoxsso.xml'] because
contents don't match
2018-07-13 15:46:36,227 - Execute['/usr/hdp/current/knox-server/bin/knoxcli.sh
create-master --master [PROTECTED]'] {'environment': {'JAVA_HOME':
u'/usr/jdk64/jdk1.8.0_112'}, 'not_if': "ambari-sudo.sh su knox -l -s /bin/bash
-c 'test -f /usr/hdp/current/knox-server/data/security/master'", 'user': 'knox'}
It appears to only affect my HDP 2.6.2 instance which I’ve tested a few and
they all exhibit the same behavior. I checked the files on those instances and
there are no permissions issues. The cluster was not upgraded from a previous
version. Again I think it is an issue with that condition, although like you, I
don’t know exactly what that condition check does as I’m not familiar with
those built in ambari functions.
I think I am just going to document this as a known issue for this particular
version and move on as it seems to behave correctly on subsequent HDP releases.
Thanks for you help in this matter.
Regards,
Christopher Jackson
> On Jul 12, 2018, at 11:43 PM, larry mccay <lmc...@apache.org> wrote:
>
> Hi -
>
> I just verified that it works as expected with the latest HDP sandbox for
> 2.6.5.
>
> I started the sandbox, changed the TTL param in the KnoxSSO service in
> knoxsso.xml from 30000 to -1, saved and restarted.
> I think used the Knox Admin UI to view the topologies that are deployed to
> the instance and it had the -1 value for the TTL param.
>
> While there may be a difference between the versions, I have not encountered
> this issue before at all.
>
> Is it possible that there is a permissions issue on the files?
> Was this cluster upgraded from a previous version?
>
> At any rate, I think that there is an issue with the cluster rather that with
> the knox.py script though I'm not really sure what that condition is even
> checking.
>
> This doesn't really solve anything for you but hope it is helpful in some way.
>
> thanks,
>
> --larry
>
>
> On Thu, Jul 12, 2018 at 8:04 PM, larry mccay <lmc...@apache.org
> <mailto:lmc...@apache.org>> wrote:
> Hi Christopher -
>
> I apologize, this dropped off my radar.
> I need to go back and try and reproduce it still.
>
> It would still be surprising to me but I am surprised often. :)
>
> thanks,
>
> --larry
>
> On Thu, Jul 12, 2018 at 5:29 PM, Christopher Jackson
> <jackson.christopher....@gmail.com
> <mailto:jackson.christopher....@gmail.com>> wrote:
> Hey Larry,
>
> I looked into this a bit deeper. It appears the knoxsso-topology is NOT
> updated because of the following code in
> /var/lib/ambari-server/resources/common-services/KNOX/0.5.0.
> <http://0.5.0.0/>2.2/package/scripts/knox.py:
>
> if params.version_formatted and
> check_stack_feature(StackFeature.KNOX_SSO_TOPOLOGY, params.version_formatted):
> File(os.path.join(params.knox_conf_dir, "topologies", "knoxsso.xml"),
> group=params.knox_group,
> owner=params.knox_user,
> content=InlineTemplate(params.knoxsso_topology_template)
> )
>
> I believe the if condition is evaluating to false and thus preventing the
> knoxsso.xml from being written as I do not see a corresponding output entry
> in the log of a restart of the Knox Service (IE. I would expect to see a
> File['/usr/hdp/current/knox-server/conf/topologies/knoxsso.xml’] line):
>
> 2018-07-12 12:42:43,870 - Generating config:
> /usr/hdp/current/knox-server/conf/gateway-site.xml
> 2018-07-12 12:42:43,871 -
> File['/usr/hdp/current/knox-server/conf/gateway-site.xml'] {'owner': 'knox',
> 'content': InlineTemplate(...), 'group': 'knox', 'mode': None, 'encoding':
> 'UTF-8'}
> 2018-07-12 12:42:43,879 -
> File['/usr/hdp/current/knox-server/conf/gateway-log4j.properties']
> {'content': InlineTemplate(...), 'owner': 'knox', 'group': 'knox', 'mode':
> 0644}
> 2018-07-12 12:42:43,887 -
> File['/usr/hdp/current/knox-server/conf/topologies/default.xml'] {'content':
> InlineTemplate(...), 'owner': 'knox', 'group': 'knox'}
> 2018-07-12 12:42:43,890 -
> File['/usr/hdp/current/knox-server/conf/topologies/admin.xml'] {'content':
> InlineTemplate(...), 'owner': 'knox', 'group': 'knox'}
> 2018-07-12 12:42:43,891 -
> Execute['/usr/hdp/current/knox-server/bin/knoxcli.sh create-master --master
> [PROTECTED]'] {'environment': {'JAVA_HOME': u'/usr/jdk64/jdk1.8.0_112'},
> 'not_if': "ambari-sudo.sh su knox -l -s /bin/bash -c 'test -f
> /usr/hdp/current/knox-server/data/security/master'", 'user': 'knox'}
>
> This is on HDP 2.6.2 using Knox 0.12.0. I’ve created issue
> https://issues.apache.org/jira/browse/AMBARI-24285
> <https://issues.apache.org/jira/browse/AMBARI-24285> to track.
>
> Regards,
> Christopher Jackson
>
>
>
>> On Jun 27, 2018, at 7:13 PM, larry mccay <lmc...@apache.org
>> <mailto:lmc...@apache.org>> wrote:
>>
>> Interesting, I will need to try and reproduce this but it is definitely the
>> first that I have heard of it.
>>
>> On Wed, Jun 27, 2018 at 6:36 PM, Christopher Jackson
>> <jackson.christopher....@gmail.com
>> <mailto:jackson.christopher....@gmail.com>> wrote:
>> Hi Larry,
>>
>>> 4) On HDP 2.6.2 I have noticed that when I make changes to the "Advanced
>>> knoxsso-topology” section for the Knox Service in Ambari and then restart
>>> the service that the changes are not persisted to disk at
>>> /usr/hdp/current/knox-server/conf/topologies/knoxsso.xml and thus the
>>> changes are not picked up until that file is hand edited to reflect the
>>> changes. Is this a known issue? For example changes to the
>>> “knoxsso.redirect.whitelist.regex” in the ambari config will not take
>>> effect until the file mentioned above is hand edited.
>>>
>>> The trick is that you have to restart the server in order for Ambari to
>>> actually push any config changes out to the Knox instances.
>>> This is unfortunate - since Knox can hot deploy topology changes but is
>>> what it is.
>>> Be aware that if you hand edit the files as you are, the next time you
>>> restart via Ambari it will overwrite any changes that you have made there.
>>
>>
>> To be clear. I am restarting the knox service via the ambari UI after I make
>> the changes to the knox configuration in the ambari UI. After restart I am
>> seeing that the above file is NOT updated, hence why I am forced to modify
>> it by hand.
>>
>> Changes made to “Advanced topology” are correctly written to disk after an
>> update to the config and a subsequent restart of the knox service. It seems
>> to just be the “Advanced knoxsso-topology” that has the issue.
>>
>> Regards,
>>
>> Christopher Jackson
>>
>>
>>
>>> On Jun 27, 2018, at 1:11 PM, larry mccay <lmc...@apache.org
>>> <mailto:lmc...@apache.org>> wrote:
>>>
>>> Hi Christopher -
>>>
>>> 1) Is it possible to include additional claims that contain group
>>> information for the user from LDAP?
>>>
>>> Not currently - there are a couple issues with this appproach but I
>>> wouldn't be against a patch that optionally enables it.
>>> * There can be 100's of groups sometimes for a given user
>>> * No one in the current ecosystem is expecting to extract groups from the
>>> cookie for authorization purposes and group lookup is done closer to the
>>> resource itself
>>> * Given that the token represents an authentication event as a snapshot in
>>> time, the group membership may change by the time you extract them from the
>>> token
>>>
>>> 2) Does the Knox SSO implementation support JSON Web Key (JWK)?
>>>
>>> Not currently.
>>>
>>> 3) Where is the signing key stored? I have the desire to validate the JWT
>>> in a third party web container. I am using Knox 0.12.0 on HDP 2.6.2.
>>>
>>> By default it uses the gateway-identity alias within the
>>> {GATEWAY_HOME}/data/security/keystores/gateway.jks keystore.
>>> It may also be configured to use custom signing keys [1] - via
>>> gateway.signing.keystore.name <http://gateway.signing.keystore.name/> and
>>> gateway.signing.key.alias
>>>
>>> 4) On HDP 2.6.2 I have noticed that when I make changes to the "Advanced
>>> knoxsso-topology” section for the Knox Service in Ambari and then restart
>>> the service that the changes are not persisted to disk at
>>> /usr/hdp/current/knox-server/conf/topologies/knoxsso.xml and thus the
>>> changes are not picked up until that file is hand edited to reflect the
>>> changes. Is this a known issue? For example changes to the
>>> “knoxsso.redirect.whitelist.regex” in the ambari config will not take
>>> effect until the file mentioned above is hand edited.
>>>
>>> The trick is that you have to restart the server in order for Ambari to
>>> actually push any config changes out to the Knox instances.
>>> This is unfortunate - since Knox can hot deploy topology changes but is
>>> what it is.
>>> Be aware that if you hand edit the files as you are, the next time you
>>> restart via Ambari it will overwrite any changes that you have made there.
>>>
>>> HTH.
>>>
>>> --larry
>>>
>>> 1.
>>> http://knox.apache.org/books/knox-1-0-0/user-guide.html#Gateway+Server+Configuration
>>>
>>> <http://knox.apache.org/books/knox-1-0-0/user-guide.html#Gateway+Server+Configuration>
>>>
>>>
>>> On Wed, Jun 27, 2018 at 1:00 PM, Christopher Jackson
>>> <jackson.christopher....@gmail.com
>>> <mailto:jackson.christopher....@gmail.com>> wrote:
>>> Hey Folks,
>>>
>>> I’ve enabled Knox SSO and I am able to navigate to the Knox SSO UI and
>>> enter credentials to log in. I am seeing that the JWT cookie is properly
>>> created with the claims that I would expect. Some questions:
>>>
>>> 1) Is it possible to include additional claims that contain group
>>> information for the user from LDAP?
>>>
>>> 2) Does the Knox SSO implementation support JSON Web Key (JWK)?
>>>
>>> 3) Where is the signing key stored? I have the desire to validate the JWT
>>> in a third party web container. I am using Knox 0.12.0 on HDP 2.6.2.
>>>
>>> 4) On HDP 2.6.2 I have noticed that when I make changes to the "Advanced
>>> knoxsso-topology” section for the Knox Service in Ambari and then restart
>>> the service that the changes are not persisted to disk at
>>> /usr/hdp/current/knox-server/conf/topologies/knoxsso.xml and thus the
>>> changes are not picked up until that file is hand edited to reflect the
>>> changes. Is this a known issue? For example changes to the
>>> “knoxsso.redirect.whitelist.re <http://knoxsso.redirect.whitelist.re/>gex”
>>> in the ambari config will not take effect until the file mentioned above is
>>> hand edited.
>>>
>>> Regards,
>>>
>>> Christopher Jackson
>>>
>>
>>
>
>
>
