[
https://issues.apache.org/jira/browse/KNOX-2147?focusedWorklogId=359293&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-359293
]
ASF GitHub Bot logged work on KNOX-2147:
----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Dec/19 10:12
Start Date: 13/Dec/19 10:12
Worklog Time Spent: 10m
Work Description: smolnar82 commented on pull request #217: KNOX-2147 -
Mask username/password in case we display call history and keep them safely (by
setting proper file permissions) in JSON file
URL: https://github.com/apache/knox/pull/217
## What changes were proposed in this pull request?
Since [KNOX-2132](https://issues.apache.org/jira/browse/KNOX-2134) the
`JDBCKnoxShellTableBuilder` can handle `username` and `password` fields. These
are sensitive data which we do not want to display when:
- querying the call history
- converting to JSON format and displaying on STDOUT
To address this issue the following has been done:
- `KnoxShellTable.toJSON()` method got overloaded to let end-users saving
the JSON output into a file. The file will be protected with file permissions
of `600` so that only the owner can read/write it. In case the end-user wants
to write the call history in this file (i.e. not the data itself) the sensitive
data will be written there too.
- in any other case when the call history-related information is displayed
on the standard output the sensitive data is masked
## How was this patch tested?
Updated and ran JUnit tests:
```
$ mvn clean -Dshellcheck=true -T1C verify -Prelease,package -am -pl
gateway-shell-release
...
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 01:37 min (Wall Clock)
[INFO] Finished at: 2019-12-13T10:46:51+01:00
[INFO] Final Memory: 133M/1589M
[INFO]
------------------------------------------------------------------------
```
Additionally, the following E2E tests were executed:
Preparing a table using `DerbyDatabase`:
```
knox:000> locations =
KnoxShellTable.builder().jdbc().driver("org.apache.derby.jdbc.EmbeddedDriver").username("smolnar").pwd("mySecretPassword").connectTo("jdbc:derby:/Users/smolnar/test/derbyDb").sql("SELECT
* FROM sample.locations where zip < 10")
===> LOCATIONS
+--------+------------+----------+----------+--------------+
| ZIP | COUNTRY | STATE | CITY | POPULATION |
+--------+------------+----------+----------+--------------+
| 1 | US | NY | City1 | 100000 |
| 2 | US | NY | City2 | 100000 |
| 3 | US | NY | City3 | 100000 |
| 4 | US | NY | City4 | 100000 |
| 5 | US | NY | City5 | 100000 |
| 6 | US | NY | City6 | 100000 |
| 7 | US | NY | City7 | 100000 |
| 8 | US | NY | City8 | 100000 |
| 9 | US | NY | City9 | 100000 |
+--------+------------+----------+----------+--------------+
```
Testing JSON output with pure data on screen:
```
knox:000> locations.toJSON()
===> {
"headers" : [ "ZIP", "COUNTRY", "STATE", "CITY", "POPULATION" ],
"rows" : [ [ 1, "US", "NY", "City1", 100000 ], [ 2, "US", "NY", "City2",
100000 ], [ 3, "US", "NY", "City3", 100000 ], [ 4, "US", "NY", "City4", 100000
], [ 5, "US", "NY", "City5", 100000 ], [ 6, "US", "NY", "City6", 100000 ], [ 7,
"US", "NY", "City7", 100000 ], [ 8, "US", "NY", "City8", 100000 ], [ 9, "US",
"NY", "City9", 100000 ] ],
"title" : "LOCATIONS",
"id" : 1576221440030
}
```
Testing JSON output with call history on screen (sensitive data is masked):
```
knox:000> locations.toJSON(false)
===> {
"callHistoryList" : [ {
"invokerClass" :
"org.apache.knox.gateway.shell.table.KnoxShellTableBuilder",
"method" : "jdbc",
"builderMethod" : false,
"params" : { }
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "driver",
"builderMethod" : false,
"params" : {
"org.apache.derby.jdbc.EmbeddedDriver" : "java.lang.String"
}
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "username",
"builderMethod" : false,
"params" : {
"***" : "java.lang.String"
}
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "pwd",
"builderMethod" : false,
"params" : {
"***" : "java.lang.String"
}
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "connectTo",
"builderMethod" : false,
"params" : {
"jdbc:derby:/Users/smolnar/test/derbyDb" : "java.lang.String"
}
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "sql",
"builderMethod" : true,
"params" : {
"SELECT * FROM sample.locations where zip < 10" : "java.lang.String"
}
} ]
}
```
Testing JSON output with pure data written to a file:
```
knox:000>
locations.toJSON("/Users/smolnar/test/folderShouldBeCreated/testToJSONWithData.json")
===> Successfully saved into
/Users/smolnar/test/folderShouldBeCreated/testToJSONWithData.json
$ ls -al /Users/smolnar/test/folderShouldBeCreated/testToJSONWithData.json
-rw------- 1 smolnar staff 457 Dec 13 10:30
/Users/smolnar/test/folderShouldBeCreated/testToJSONWithData.json
$ cat /Users/smolnar/test/folderShouldBeCreated/testToJSONWithData.json
{
"headers" : [ "ZIP", "COUNTRY", "STATE", "CITY", "POPULATION" ],
"rows" : [ [ 1, "US", "NY", "City1", 100000 ], [ 2, "US", "NY", "City2",
100000 ], [ 3, "US", "NY", "City3", 100000 ], [ 4, "US", "NY", "City4", 100000
], [ 5, "US", "NY", "City5", 100000 ], [ 6, "US", "NY", "City6", 100000 ], [ 7,
"US", "NY", "City7", 100000 ], [ 8, "US", "NY", "City8", 100000 ], [ 9, "US",
"NY", "City9", 100000 ] ],
"title" : "LOCATIONS",
"id" : 1576221440030
}
```
Testing JSON output with call history written to a file (sensitive data is
**not** masked):
```
knox:000> locations.toJSON(false,
"/Users/smolnar/test/notYetExistingFolder/testToJSONWithCallHistory.json")
===> Successfully saved into
/Users/smolnar/test/notYetExistingFolder/testToJSONWithCallHistory.json
$ ls -al
/Users/smolnar/test/notYetExistingFolder/testToJSONWithCallHistory.json
-rw------- 1 smolnar staff 1222 Dec 13 08:17
/Users/smolnar/test/notYetExistingFolder/testToJSONWithCallHistory.json
$ cat /Users/smolnar/test/notYetExistingFolder/testToJSONWithCallHistory.json
[ {
"invokerClass" :
"org.apache.knox.gateway.shell.table.KnoxShellTableBuilder",
"method" : "jdbc",
"builderMethod" : false,
"params" : { }
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "driver",
"builderMethod" : false,
"params" : {
"org.apache.derby.jdbc.EmbeddedDriver" : "java.lang.String"
}
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "username",
"builderMethod" : false,
"params" : {
"smolnar" : "java.lang.String"
}
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "pwd",
"builderMethod" : false,
"params" : {
"mySecretPassword" : "java.lang.String"
}
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "connectTo",
"builderMethod" : false,
"params" : {
"jdbc:derby:/Users/smolnar/test/derbyDb" : "java.lang.String"
}
}, {
"invokerClass" :
"org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder",
"method" : "sql",
"builderMethod" : true,
"params" : {
"SELECT * FROM sample.locations where zip < 10" : "java.lang.String"
}
} ]
```
Testing call history display on screen (sensitive data is masked):
```
knox:000> locations.getCallHistory()
===> Call history (id=1576221440030)
Step 1:
invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableBuilder
method=jdbc
builderMethod=false
params={}
Step 2:
invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=driver
builderMethod=false
params={org.apache.derby.jdbc.EmbeddedDriver=class java.lang.String}
Step 3:
invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=username
builderMethod=false
params={***=class java.lang.String}
Step 4:
invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=pwd
builderMethod=false
params={***=class java.lang.String}
Step 5:
invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=connectTo
builderMethod=false
params={jdbc:derby:/Users/smolnar/test/derbyDb=class java.lang.String}
Step 6:
invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=sql
builderMethod=true
params={SELECT * FROM sample.locations where zip < 10=class java.lang.String}
```
Testing querying the call history as a list on screen (sensitive data is
masked):
```
knox:000> locations.getCallHistoryList()
===> [invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableBuilder
method=jdbc
builderMethod=false
params={}
, invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=driver
builderMethod=false
params={org.apache.derby.jdbc.EmbeddedDriver=class java.lang.String}
, invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=username
builderMethod=false
params={***=class java.lang.String}
, invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=pwd
builderMethod=false
params={***=class java.lang.String}
, invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=connectTo
builderMethod=false
params={jdbc:derby:/Users/smolnar/test/derbyDb=class java.lang.String}
, invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder
method=sql
builderMethod=true
params={SELECT * FROM sample.locations where zip < 10=class java.lang.String}
]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 359293)
Remaining Estimate: 0h
Time Spent: 10m
> Keep username and password out of KnoxShellTableCallHistory
> ------------------------------------------------------------
>
> Key: KNOX-2147
> URL: https://issues.apache.org/jira/browse/KNOX-2147
> Project: Apache Knox
> Issue Type: Improvement
> Components: KnoxShell
> Reporter: Larry McCay
> Assignee: Sandor Molnar
> Priority: Major
> Fix For: 1.4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In working on KNOX-2132, I couldn't actually get the call history to work and
> was therefore unable to make sure that the username and password params don't
> end up in the persisted history or at least not rendered in the listing.
> Either call history no longer works or I just don't know how to enable it.
> Tests don't seem to cover the actual AOP based capture but record hardcoded
> calls rather than actual table interactions. I also notice that the
> aspectjrt.jar isn't being placed in the lib dir for knoxshell which seems
> broken.
> So, first thing to do is ensure that call history is actually working and fix
> it if not. Then determine what to do about the username and password and
> persistence of call histories as the means for reconstituting a dataset. Do
> we build in a required login which would mean that the dataset rehydration
> would require a user interaction for login? Do we encrypt the credentials -
> if so, using what as a key and how to manage it? Do we just rely on file
> permissions?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
