smolnar82 opened a new pull request #162: KNOX-2023 - Recording KnoxShellTable builder/filter chain and providing rollback/replay capabilities using the call history as well as allowing end-users to export JSON without data (in this case only the call history will be serialized) URL: https://github.com/apache/knox/pull/162 ## What changes were proposed in this pull request? The following features have been added with this change in `knoxshell`: - recording KnoxShellTable builder/filter chain - providing rollback/replay capabilities using the call history - allowing end-users to export JSON without data (in this case, only the call history will be serialized) ## How was this patch tested? Executing a full build (where I added/updated new/existing JUnit tests): ``` $ mvn clean -Dshellcheck=true -T1C verify -Prelease,package ... [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 17:58 min (Wall Clock) [INFO] Finished at: 2019-10-04T18:08:41+02:00 [INFO] Final Memory: 389M/1837M [INFO] ------------------------------------------------------------------------ ``` In addition to running JUnit tests the following manual test steps were executed: 1.) Building a table with the JDBC builder: ``` locations = KnoxShellTable.builder().jdbc().driver("org.apache.derby.jdbc.EmbeddedDriver").connectTo("jdbc:derby:/Users/smolnar/test/derbyDb").sql("SELECT * FROM sample.locations where zip < 15") ===> LOCATIONS +--------+------------+----------+----------+--------------+ | ZIP | COUNTRY | STATE | CITY | POPULATION | +--------+------------+----------+----------+--------------+ | 1 | US | NY | City1 | 100000 | | 2 | US | NY | City2 | 100000 | | 3 | US | NY | City3 | 100000 | | 4 | US | NY | City4 | 100000 | | 5 | US | NY | City5 | 100000 | | 6 | US | NY | City6 | 100000 | | 7 | US | NY | City7 | 100000 | | 8 | US | NY | City8 | 100000 | | 9 | US | NY | City9 | 100000 | | 10 | US | NY | City10 | 100000 | | 11 | US | NY | City11 | 100000 | | 12 | US | NY | City12 | 100000 | | 13 | US | NY | City13 | 100000 | | 14 | US | NY | City14 | 100000 | +--------+------------+----------+----------+--------------+ knox:000> knox:000> locations.getCallHistory() ===> Call history (id=1570194513709) Step 1: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableBuilder method=jdbc builderMethod=false params={} Step 2: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=driver builderMethod=false params={org.apache.derby.jdbc.EmbeddedDriver=class java.lang.String} Step 3: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=connectTo builderMethod=false params={jdbc:derby:/Users/smolnar/test/derbyDb=class java.lang.String} Step 4: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=sql builderMethod=true params={SELECT * FROM sample.locations where zip < 15=class java.lang.String} ``` 2.) Filtering the previously built table: ``` knox:000> filteredLocations = locations.filter().name("ZIP").greaterThan(5) ===> LOCATIONS +--------+------------+----------+----------+--------------+ | ZIP | COUNTRY | STATE | CITY | POPULATION | +--------+------------+----------+----------+--------------+ | 6 | US | NY | City6 | 100000 | | 7 | US | NY | City7 | 100000 | | 8 | US | NY | City8 | 100000 | | 9 | US | NY | City9 | 100000 | | 10 | US | NY | City10 | 100000 | | 11 | US | NY | City11 | 100000 | | 12 | US | NY | City12 | 100000 | | 13 | US | NY | City13 | 100000 | | 14 | US | NY | City14 | 100000 | +--------+------------+----------+----------+--------------+ knox:000> knox:000> filteredLocations.getCallHistory() ===> Call history (id=1570194514968) Step 1: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableBuilder method=jdbc builderMethod=false params={} Step 2: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=driver builderMethod=false params={org.apache.derby.jdbc.EmbeddedDriver=class java.lang.String} Step 3: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=connectTo builderMethod=false params={jdbc:derby:/Users/smolnar/test/derbyDb=class java.lang.String} Step 4: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=sql builderMethod=true params={SELECT * FROM sample.locations where zip < 15=class java.lang.String} Step 5: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTable method=filter builderMethod=false params={} Step 6: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableFilter method=name builderMethod=false params={ZIP=class java.lang.String} Step 7: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableFilter method=greaterThan builderMethod=true params={5=class java.lang.Integer} knox:000> knox:000> filteredLocations.toJSON() ===> { "headers" : [ "ZIP", "COUNTRY", "STATE", "CITY", "POPULATION" ], "rows" : [ [ 6, "US", "NY", "City6", 100000 ], [ 7, "US", "NY", "City7", 100000 ], [ 8, "US", "NY", "City8", 100000 ], [ 9, "US", "NY", "City9", 100000 ], [ 10, "US", "NY", "City10", 100000 ], [ 11, "US", "NY", "City11", 100000 ], [ 12, "US", "NY", "City12", 100000 ], [ 13, "US", "NY", "City13", 100000 ], [ 14, "US", "NY", "City14", 100000 ] ], "title" : "LOCATIONS", "id" : 1570194514968 } knox:000> knox:000> filteredLocations.toJSON(false) ===> { "callHistoryList" : [ { "invokerClass" : "org.apache.knox.gateway.shell.table.KnoxShellTableBuilder", "method" : "jdbc", "builderMethod" : false, "params" : { } }, { "invokerClass" : "org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder", "method" : "driver", "builderMethod" : false, "params" : { "org.apache.derby.jdbc.EmbeddedDriver" : "java.lang.String" } }, { "invokerClass" : "org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder", "method" : "connectTo", "builderMethod" : false, "params" : { "jdbc:derby:/Users/smolnar/test/derbyDb" : "java.lang.String" } }, { "invokerClass" : "org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder", "method" : "sql", "builderMethod" : true, "params" : { "SELECT * FROM sample.locations where zip < 15" : "java.lang.String" } }, { "invokerClass" : "org.apache.knox.gateway.shell.table.KnoxShellTable", "method" : "filter", "builderMethod" : false, "params" : { } }, { "invokerClass" : "org.apache.knox.gateway.shell.table.KnoxShellTableFilter", "method" : "name", "builderMethod" : false, "params" : { "ZIP" : "java.lang.String" } }, { "invokerClass" : "org.apache.knox.gateway.shell.table.KnoxShellTableFilter", "method" : "greaterThan", "builderMethod" : true, "params" : { "5" : "java.lang.Integer" } } ] } ``` 3.) Replaying the filtered table up to `step 4`: ``` knox:000> replayed = filteredLocations.replay(4) ===> LOCATIONS +--------+------------+----------+----------+--------------+ | ZIP | COUNTRY | STATE | CITY | POPULATION | +--------+------------+----------+----------+--------------+ | 1 | US | NY | City1 | 100000 | | 2 | US | NY | City2 | 100000 | | 3 | US | NY | City3 | 100000 | | 4 | US | NY | City4 | 100000 | | 5 | US | NY | City5 | 100000 | | 6 | US | NY | City6 | 100000 | | 7 | US | NY | City7 | 100000 | | 8 | US | NY | City8 | 100000 | | 9 | US | NY | City9 | 100000 | | 10 | US | NY | City10 | 100000 | | 11 | US | NY | City11 | 100000 | | 12 | US | NY | City12 | 100000 | | 13 | US | NY | City13 | 100000 | | 14 | US | NY | City14 | 100000 | +--------+------------+----------+----------+--------------+ knox:000> knox:000> replayed.getCallHistory() ===> Call history (id=1570194515303) Step 1: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableBuilder method=jdbc builderMethod=false params={} Step 2: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=driver builderMethod=false params={org.apache.derby.jdbc.EmbeddedDriver=class java.lang.String} Step 3: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=connectTo builderMethod=false params={jdbc:derby:/Users/smolnar/test/derbyDb=class java.lang.String} Step 4: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=sql builderMethod=true params={SELECT * FROM sample.locations where zip < 15=class java.lang.String} ``` 4.) Replaying the filtered table entirely: ``` knox:000> replayed = filteredLocations.replayAll() ===> LOCATIONS +--------+------------+----------+----------+--------------+ | ZIP | COUNTRY | STATE | CITY | POPULATION | +--------+------------+----------+----------+--------------+ | 6 | US | NY | City6 | 100000 | | 7 | US | NY | City7 | 100000 | | 8 | US | NY | City8 | 100000 | | 9 | US | NY | City9 | 100000 | | 10 | US | NY | City10 | 100000 | | 11 | US | NY | City11 | 100000 | | 12 | US | NY | City12 | 100000 | | 13 | US | NY | City13 | 100000 | | 14 | US | NY | City14 | 100000 | +--------+------------+----------+----------+--------------+ knox:000> knox:000> replayed.getCallHistory() ===> Call history (id=1570194515309) Step 1: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableBuilder method=jdbc builderMethod=false params={} Step 2: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=driver builderMethod=false params={org.apache.derby.jdbc.EmbeddedDriver=class java.lang.String} Step 3: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=connectTo builderMethod=false params={jdbc:derby:/Users/smolnar/test/derbyDb=class java.lang.String} Step 4: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=sql builderMethod=true params={SELECT * FROM sample.locations where zip < 15=class java.lang.String} Step 5: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTable method=filter builderMethod=false params={} Step 6: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableFilter method=name builderMethod=false params={ZIP=class java.lang.String} Step 7: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableFilter method=greaterThan builderMethod=true params={5=class java.lang.Integer} ``` 5.) Rolled back to the previous valid state ``` knox:000> replayed.rollback() ===> Successfully rolled back knox:000> replayed.getCallHistory() ===> Call history (id=1570195550441) Step 1: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableBuilder method=jdbc builderMethod=false params={} Step 2: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=driver builderMethod=false params={org.apache.derby.jdbc.EmbeddedDriver=class java.lang.String} Step 3: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=connectTo builderMethod=false params={jdbc:derby:/Users/smolnar/test/derbyDb=class java.lang.String} Step 4: invokerClass=org.apache.knox.gateway.shell.table.JDBCKnoxShellTableBuilder method=sql builderMethod=true params={SELECT * FROM sample.locations where zip < 15=class java.lang.String} ``` 6.) Building a table from JSON source holding pure data: ``` knox:000> loadedFromJsonWithData = KnoxShellTable.builder().json().path("/Users/smolnar/sampleTableWithData.json") ===> +--------+------------+----------+----------+--------------+ | ZIP | COUNTRY | STATE | CITY | POPULATION | +--------+------------+----------+----------+--------------+ | 6 | US | NY | City6 | 100000 | | 7 | US | NY | City7 | 100000 | | 8 | US | NY | City8 | 100000 | | 9 | US | NY | City9 | 100000 | | 10 | US | NY | City10 | 100000 | | 11 | US | NY | City11 | 100000 | | 12 | US | NY | City12 | 100000 | | 13 | US | NY | City13 | 100000 | | 14 | US | NY | City14 | 100000 | +--------+------------+----------+----------+--------------+ knox:000> knox:000> loadedFromJsonWithData.getCallHistory() ===> Call history (id=1570194515913) Step 1: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableBuilder method=json builderMethod=false params={} Step 2: invokerClass=org.apache.knox.gateway.shell.table.JSONKnoxShellTableBuilder method=path builderMethod=true params={/Users/smolnar/sampleTableWithData.json=class java.lang.String} ``` 7.) Building a table from JSON source holding call history: ``` knox:000> loadedFromJsonWithCallHistory = KnoxShellTable.builder().json().path("/Users/smolnar/sampleTableWithCallHistory.json") ===> LOCATIONS +--------+------------+----------+----------+--------------+ | ZIP | COUNTRY | STATE | CITY | POPULATION | +--------+------------+----------+----------+--------------+ | 6 | US | NY | City6 | 100000 | | 7 | US | NY | City7 | 100000 | | 8 | US | NY | City8 | 100000 | | 9 | US | NY | City9 | 100000 | | 10 | US | NY | City10 | 100000 | | 11 | US | NY | City11 | 100000 | | 12 | US | NY | City12 | 100000 | | 13 | US | NY | City13 | 100000 | | 14 | US | NY | City14 | 100000 | +--------+------------+----------+----------+--------------+ knox:000> knox:000> loadedFromJsonWithCallHistory.getCallHistory() ===> Call history (id=1570194516193) Step 1: invokerClass=org.apache.knox.gateway.shell.table.KnoxShellTableBuilder method=json builderMethod=false params={} Step 2: invokerClass=org.apache.knox.gateway.shell.table.JSONKnoxShellTableBuilder method=path builderMethod=true params={/Users/smolnar/sampleTableWithCallHistory.json=class java.lang.String} ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
