[
https://issues.apache.org/jira/browse/KNOX-2579?focusedWorklogId=586058&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-586058
]
ASF GitHub Bot logged work on KNOX-2579:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Apr/21 18:03
Start Date: 20/Apr/21 18:03
Worklog Time Spent: 10m
Work Description: smolnar82 opened a new pull request #437:
URL: https://github.com/apache/knox/pull/437
## What changes were proposed in this pull request?
Saving the `token.id` claim (aka. tokenId or passcode) in a hashed form in
the database instead of plain text.
## How was this patch tested?
Updated JUnit tests using Derby DB. I also executed the following manual
steps with a Postgres DB configured as the token management backend. During my
tests, I did not declare hashing algorithm in `gateway-site.xml` so that the
default `HmacSha256` was used so I needed to configure an appropriate hashing
key.
0. Configuration:
```
bin/knoxcli.sh create-alias gateway_database_hash_key --value
sPj8FCgQhCEi6G18kBfpswxYSki33plbelGLs0hMSbk
```
1. Fetched a token using our tokengen application
(passcode/token.id=b1e78313-3635-4236-bd77-71886ef9a707)
```
2021-04-20 19:38:11,818 DEBUG knox.gateway
(GatewayFilter.java:doFilter(116)) - Received request: GET
/knoxtoken/api/v1/token
2021-04-20 19:38:11,907 INFO service.knoxtoken
(TokenResource.java:getAuthenticationToken(421)) - Knox Token service
(homepage) issued token eyJhbG...B9Ze9A (b1e78313...71886ef9a707)
2021-04-20 19:38:11,920 DEBUG token.state
(JDBCTokenStateService.java:addToken(82)) - Token b1e78313...71886ef9a707 has
been saved in the database; id = 4448b054-3467-4c29-9468-d4bddfdf5e00
2021-04-20 19:38:11,920 DEBUG token.state
(DefaultTokenStateService.java:addToken(144)) - Added token
b1e78313...71886ef9a707, expiration 2021-04-21T17:38:11.900Z
2021-04-20 19:38:11,927 DEBUG token.state
(JDBCTokenStateService.java:addMetadata(213)) - Updated metadata for
b1e78313...71886ef9a707 in the database
2021-04-20 19:38:11,927 DEBUG service.knoxtoken
(TokenResource.java:getAuthenticationToken(448)) - Knox Token service
(homepage) stored state for token eyJhbG...B9Ze9A (b1e78313...71886ef9a707)
postgres=# select * FROM KNOX_TOKENS;
id | token_id
| issue_time | expiration | max_lifetime | username |
comment
--------------------------------------+---------------------------------------------------------+---------------+---------------+---------------+----------+---------
4448b054-3467-4c29-9468-d4bddfdf5e00 |
[\x1A�\x15�w���\x04f\x7F+�S\x1D��\x06��\x1B�(3�س�\x1F�G | 1618940291911 |
1619026691900 | 1619545091911 | admin |
(1 row)
```
2. Renewed the token
```
$ curl -ku admin:admin-password -d "@token.txt" -X POST
https://localhost:8443/gateway/sandbox/knoxtoken/api/v1/token/renew
{
"renewed": "true",
"expires": "1619026870461"
}
2021-04-20 19:41:10,445 DEBUG knox.gateway
(GatewayFilter.java:doFilter(116)) - Received request: POST
/knoxtoken/api/v1/token/renew
2021-04-20 19:41:10,446 INFO knox.gateway
(KnoxLdapRealm.java:getUserDn(688)) - Computed userDn:
uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal:
admin
2021-04-20 19:41:10,476 DEBUG token.state
(JDBCTokenStateService.java:updateExpiration(135)) - Updated expiration for
b1e78313...71886ef9a707 in the database to 1,619,026,870,461
2021-04-20 19:41:10,477 DEBUG token.state
(DefaultTokenStateService.java:renewToken(219)) - Renewed token
b1e78313...71886ef9a707, expiration 2021-04-21T17:41:10.461Z
2021-04-20 19:41:10,477 INFO service.knoxtoken
(TokenResource.java:renew(282)) - Knox Token service (sandbox) renewed the
expiration for token eyJhbG...B9Ze9A (b1e78313...71886ef9a707) (renewer=admin)
postgres=# select * FROM KNOX_TOKENS;
id | token_id
| issue_time | expiration | max_lifetime | username |
comment
--------------------------------------+---------------------------------------------------------+---------------+---------------+---------------+----------+---------
4448b054-3467-4c29-9468-d4bddfdf5e00 |
[\x1A�\x15�w���\x04f\x7F+�S\x1D��\x06��\x1B�(3�س�\x1F�G | 1618940291911 |
1619026870461 | 1619545091911 | admin |
(1 row)
```
3. Used the token successfully (issued a WEBHDFS list command)
4. Revoked the token
```
$ curl -ku admin:admin-password -d "@token.txt" -X POST
https://localhost:8443/gateway/sandbox/knoxtoken/api/v1/token/revoke
{
"revoked": "true"
}
2021-04-20 19:44:32,405 DEBUG knox.gateway
(GatewayFilter.java:doFilter(116)) - Received request: POST
/knoxtoken/api/v1/token/revoke
2021-04-20 19:44:32,406 INFO knox.gateway
(KnoxLdapRealm.java:getUserDn(688)) - Computed userDn:
uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal:
admin
2021-04-20 19:44:32,440 DEBUG token.state
(DefaultTokenStateService.java:removeTokenState(290)) - Removed state for
tokens b1e78313...71886ef9a707
2021-04-20 19:44:32,440 DEBUG token.state
(JDBCTokenStateService.java:removeToken(186)) - Token b1e78313...71886ef9a707
has been removed from the database
2021-04-20 19:44:32,441 DEBUG token.state
(DefaultTokenStateService.java:revokeToken(244)) - Revoked token
b1e78313...71886ef9a707
2021-04-20 19:44:32,443 INFO service.knoxtoken
(TokenResource.java:revoke(329)) - Knox Token service (sandbox) revoked token
eyJhbG...B9Ze9A (b1e78313...71886ef9a707) (renewer=admin)
postgres=# select * FROM KNOX_TOKENS;
id | token_id | issue_time | expiration | max_lifetime | username | comment
----+----------+------------+------------+--------------+----------+---------
(0 rows)
```
5. Confirmed it cannot be used anymore (repeated step 3.):
```
$ curl -ku Passcode:b1e78313-3635-4236-bd77-71886ef9a707
https://localhost:8443/gateway/tokenbased/webhdfs/v1?op=LISTSTATUS
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 401 Unknown token: b1e78313...71886ef9a707</title>
</head>
<body><h2>HTTP ERROR 401 Unknown token: b1e78313...71886ef9a707</h2>
<table>
<tr><th>URI:</th><td>/gateway/tokenbased/webhdfs/v1</td></tr>
<tr><th>STATUS:</th><td>401</td></tr>
<tr><th>MESSAGE:</th><td>Unknown token: b1e78313...71886ef9a707</td></tr>
<tr><th>SERVLET:</th><td>tokenbased-knox-gateway-servlet</td></tr>
</table>
</body>
</html>
2021-04-20 19:47:02,751 DEBUG knox.gateway
(GatewayFilter.java:doFilter(116)) - Received request: GET /webhdfs/v1
2021-04-20 19:47:02,761 ERROR token.state
(DefaultTokenStateService.java:validateToken(321)) - Unknown token
b1e78313...71886ef9a707
2021-04-20 19:47:02,772 ERROR token.state
(DefaultTokenStateService.java:validateToken(321)) - Unknown token
b1e78313...71886ef9a707
2021-04-20 19:47:02,772 WARN federation.jwt
(AbstractJWTFilter.java:validateToken(404)) - Unable to verify token
expiration:
org.apache.knox.gateway.services.security.token.UnknownTokenException: Unknown
token: b1e78313...71886ef9a707
org.apache.knox.gateway.services.security.token.UnknownTokenException:
Unknown token: b1e78313...71886ef9a707
at
org.apache.knox.gateway.services.token.impl.DefaultTokenStateService.validateToken(DefaultTokenStateService.java:322)
at
org.apache.knox.gateway.services.token.impl.JDBCTokenStateService.getTokenExpiration(JDBCTokenStateService.java:110)
at
org.apache.knox.gateway.services.token.impl.DefaultTokenStateService.getTokenExpiration(DefaultTokenStateService.java:175)
at
org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter.getServerManagedStateExpiration(AbstractJWTFilter.java:197)
at
org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter.tokenIsStillValid(AbstractJWTFilter.java:190)
at
org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter.validateToken(AbstractJWTFilter.java:391)
at
org.apache.knox.gateway.provider.federation.jwt.filter.JWTFederationFilter.doFilter(JWTFederationFilter.java:121)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 586058)
Remaining Estimate: 0h
Time Spent: 10m
> Make token passcode secure in DB token state backend
> ----------------------------------------------------
>
> Key: KNOX-2579
> URL: https://issues.apache.org/jira/browse/KNOX-2579
> Project: Apache Knox
> Issue Type: New Feature
> Components: Server
> Affects Versions: 1.6.0
> Reporter: Sandor Molnar
> Assignee: Sandor Molnar
> Priority: Major
> Fix For: 1.6.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With KNOX-2554, we now have the ability to store passcode tokens in
> relational databases. However, it indicates poor security practice if
> sensitive data is stored in plain text format. Since the {{token_id}} JWT
> claim can be used as a passcode, we need to make sure it's saved in a hashed
> format. To be able to do this, the following is going to be implemented:
> * add a new column called {{id}} which will serve as the primary key of the
> {{KNOX_TOKENS}} table (this is also going to be a UUID)
> * keep the current {{token_id}} column as is, and store the {{token.id}}
> claim in a hashed form in this column
> By default, {{HS256}} is going to be used as a hash algorithm, but end-users
> can configure it via the {{gateway.database.hash.alg}} gateway level
> configuration. A new pre-defined alias name is to be introduced too:
> {{gateway_database_hash_key}}. End-users must save the desired key using this
> alias if they use the new {{JDBCTokenStateService}} as the token management
> backend. Please note that key size it's very important for hash-based
> algorithms so using the {{master secret}} is not an option here.
> The token verification logic has to be changed too (need to hash {{token.id}}
> before getting expiration from the database).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
