smolnar82 opened a new pull request #437:
URL: https://github.com/apache/knox/pull/437
## What changes were proposed in this pull request?
Saving the `token.id` claim (aka. tokenId or passcode) in a hashed form in
the database instead of plain text.
## How was this patch tested?
Updated JUnit tests using Derby DB. I also executed the following manual
steps with a Postgres DB configured as the token management backend. During my
tests, I did not declare hashing algorithm in `gateway-site.xml` so that the
default `HmacSha256` was used so I needed to configure an appropriate hashing
key.
0. Configuration:
```
bin/knoxcli.sh create-alias gateway_database_hash_key --value
sPj8FCgQhCEi6G18kBfpswxYSki33plbelGLs0hMSbk
```
1. Fetched a token using our tokengen application
(passcode/token.id=b1e78313-3635-4236-bd77-71886ef9a707)
```
2021-04-20 19:38:11,818 DEBUG knox.gateway
(GatewayFilter.java:doFilter(116)) - Received request: GET
/knoxtoken/api/v1/token
2021-04-20 19:38:11,907 INFO service.knoxtoken
(TokenResource.java:getAuthenticationToken(421)) - Knox Token service
(homepage) issued token eyJhbG...B9Ze9A (b1e78313...71886ef9a707)
2021-04-20 19:38:11,920 DEBUG token.state
(JDBCTokenStateService.java:addToken(82)) - Token b1e78313...71886ef9a707 has
been saved in the database; id = 4448b054-3467-4c29-9468-d4bddfdf5e00
2021-04-20 19:38:11,920 DEBUG token.state
(DefaultTokenStateService.java:addToken(144)) - Added token
b1e78313...71886ef9a707, expiration 2021-04-21T17:38:11.900Z
2021-04-20 19:38:11,927 DEBUG token.state
(JDBCTokenStateService.java:addMetadata(213)) - Updated metadata for
b1e78313...71886ef9a707 in the database
2021-04-20 19:38:11,927 DEBUG service.knoxtoken
(TokenResource.java:getAuthenticationToken(448)) - Knox Token service
(homepage) stored state for token eyJhbG...B9Ze9A (b1e78313...71886ef9a707)
postgres=# select * FROM KNOX_TOKENS;
id | token_id
| issue_time | expiration | max_lifetime | username |
comment
--------------------------------------+---------------------------------------------------------+---------------+---------------+---------------+----------+---------
4448b054-3467-4c29-9468-d4bddfdf5e00 |
[\x1A�\x15�w���\x04f\x7F+�S\x1D��\x06��\x1B�(3�س�\x1F�G | 1618940291911 |
1619026691900 | 1619545091911 | admin |
(1 row)
```
2. Renewed the token
```
$ curl -ku admin:admin-password -d "@token.txt" -X POST
https://localhost:8443/gateway/sandbox/knoxtoken/api/v1/token/renew
{
"renewed": "true",
"expires": "1619026870461"
}
2021-04-20 19:41:10,445 DEBUG knox.gateway
(GatewayFilter.java:doFilter(116)) - Received request: POST
/knoxtoken/api/v1/token/renew
2021-04-20 19:41:10,446 INFO knox.gateway
(KnoxLdapRealm.java:getUserDn(688)) - Computed userDn:
uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal:
admin
2021-04-20 19:41:10,476 DEBUG token.state
(JDBCTokenStateService.java:updateExpiration(135)) - Updated expiration for
b1e78313...71886ef9a707 in the database to 1,619,026,870,461
2021-04-20 19:41:10,477 DEBUG token.state
(DefaultTokenStateService.java:renewToken(219)) - Renewed token
b1e78313...71886ef9a707, expiration 2021-04-21T17:41:10.461Z
2021-04-20 19:41:10,477 INFO service.knoxtoken
(TokenResource.java:renew(282)) - Knox Token service (sandbox) renewed the
expiration for token eyJhbG...B9Ze9A (b1e78313...71886ef9a707) (renewer=admin)
postgres=# select * FROM KNOX_TOKENS;
id | token_id
| issue_time | expiration | max_lifetime | username |
comment
--------------------------------------+---------------------------------------------------------+---------------+---------------+---------------+----------+---------
4448b054-3467-4c29-9468-d4bddfdf5e00 |
[\x1A�\x15�w���\x04f\x7F+�S\x1D��\x06��\x1B�(3�س�\x1F�G | 1618940291911 |
1619026870461 | 1619545091911 | admin |
(1 row)
```
3. Used the token successfully (issued a WEBHDFS list command)
4. Revoked the token
```
$ curl -ku admin:admin-password -d "@token.txt" -X POST
https://localhost:8443/gateway/sandbox/knoxtoken/api/v1/token/revoke
{
"revoked": "true"
}
2021-04-20 19:44:32,405 DEBUG knox.gateway
(GatewayFilter.java:doFilter(116)) - Received request: POST
/knoxtoken/api/v1/token/revoke
2021-04-20 19:44:32,406 INFO knox.gateway
(KnoxLdapRealm.java:getUserDn(688)) - Computed userDn:
uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal:
admin
2021-04-20 19:44:32,440 DEBUG token.state
(DefaultTokenStateService.java:removeTokenState(290)) - Removed state for
tokens b1e78313...71886ef9a707
2021-04-20 19:44:32,440 DEBUG token.state
(JDBCTokenStateService.java:removeToken(186)) - Token b1e78313...71886ef9a707
has been removed from the database
2021-04-20 19:44:32,441 DEBUG token.state
(DefaultTokenStateService.java:revokeToken(244)) - Revoked token
b1e78313...71886ef9a707
2021-04-20 19:44:32,443 INFO service.knoxtoken
(TokenResource.java:revoke(329)) - Knox Token service (sandbox) revoked token
eyJhbG...B9Ze9A (b1e78313...71886ef9a707) (renewer=admin)
postgres=# select * FROM KNOX_TOKENS;
id | token_id | issue_time | expiration | max_lifetime | username | comment
----+----------+------------+------------+--------------+----------+---------
(0 rows)
```
5. Confirmed it cannot be used anymore (repeated step 3.):
```
$ curl -ku Passcode:b1e78313-3635-4236-bd77-71886ef9a707
https://localhost:8443/gateway/tokenbased/webhdfs/v1?op=LISTSTATUS
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 401 Unknown token: b1e78313...71886ef9a707</title>
</head>
<body><h2>HTTP ERROR 401 Unknown token: b1e78313...71886ef9a707</h2>
<table>
<tr><th>URI:</th><td>/gateway/tokenbased/webhdfs/v1</td></tr>
<tr><th>STATUS:</th><td>401</td></tr>
<tr><th>MESSAGE:</th><td>Unknown token: b1e78313...71886ef9a707</td></tr>
<tr><th>SERVLET:</th><td>tokenbased-knox-gateway-servlet</td></tr>
</table>
</body>
</html>
2021-04-20 19:47:02,751 DEBUG knox.gateway
(GatewayFilter.java:doFilter(116)) - Received request: GET /webhdfs/v1
2021-04-20 19:47:02,761 ERROR token.state
(DefaultTokenStateService.java:validateToken(321)) - Unknown token
b1e78313...71886ef9a707
2021-04-20 19:47:02,772 ERROR token.state
(DefaultTokenStateService.java:validateToken(321)) - Unknown token
b1e78313...71886ef9a707
2021-04-20 19:47:02,772 WARN federation.jwt
(AbstractJWTFilter.java:validateToken(404)) - Unable to verify token
expiration:
org.apache.knox.gateway.services.security.token.UnknownTokenException: Unknown
token: b1e78313...71886ef9a707
org.apache.knox.gateway.services.security.token.UnknownTokenException:
Unknown token: b1e78313...71886ef9a707
at
org.apache.knox.gateway.services.token.impl.DefaultTokenStateService.validateToken(DefaultTokenStateService.java:322)
at
org.apache.knox.gateway.services.token.impl.JDBCTokenStateService.getTokenExpiration(JDBCTokenStateService.java:110)
at
org.apache.knox.gateway.services.token.impl.DefaultTokenStateService.getTokenExpiration(DefaultTokenStateService.java:175)
at
org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter.getServerManagedStateExpiration(AbstractJWTFilter.java:197)
at
org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter.tokenIsStillValid(AbstractJWTFilter.java:190)
at
org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter.validateToken(AbstractJWTFilter.java:391)
at
org.apache.knox.gateway.provider.federation.jwt.filter.JWTFederationFilter.doFilter(JWTFederationFilter.java:121)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
