Till Westmann has posted comments on this change. Change subject: Enabled Feed Tests and Added External Library tests ......................................................................
Patch Set 11: (40 comments) Did first round of reviews (up to IRecordReader). I also had to rebase this change to master in order to be able to build it. https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java File asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java: Line 82: // install if needed (i,e, add the functions, adapters, datasources, parsers to the metadata) <Not required for use> fix linebreak Line 92: * @return a map from dataverse -> list of uninstalled libraries. WS Line 127: * TODO Currently, external libraries only include functions and adapters. we need to extend this to include: Could you file a JIRA for this? Line 143: // make sure dataverse exists Do we need this check? Won't this happen automatically when checking for the library? Line 210: LOGGER.info("Added library " + libraryName + "to Metadata"); add space between quote and "to" Line 226: //throw new Exception("No library descriptor defined"); remove commented code? https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/java/org/apache/asterix/test/common/TestLibrarian.java File asterix-app/src/test/java/org/apache/asterix/test/common/TestLibrarian.java: Line 51: source.getName(); Why do we call this? Line 60: } catch (Exception e) { Would be enough to catch ZipException here? Line 62: throw new Exception("Couldn't unzip the file: " + source, e); remove empty line above? Line 67: // install if needed (i,e, add the functions, adapters, datasources, parsers to the metadata) <Not required for use> fix line break? https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.1.ddl.aql File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.1.ddl.aql: Line 22: Associate with the feed an external user-defined function. The UDF WS Line 23: finds topics in each tweet. A topic is identified by a #. WS https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.3.ddl.aql File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.3.ddl.aql: Line 22: Associate with the feed an external user-defined function. The UDF WS Line 23: finds topics in each tweet. A topic is identified by a #. WS https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql: Line 22: Associate with the feed an external user-defined function. The UDF WS Line 23: finds topics in each tweet. A topic is identified by a #. WS https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.5.query.aql File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.5.query.aql: Line 22: Associate with the feed an external user-defined function. The UDF WS Line 23: finds topics in each tweet. A topic is identified by a #. WS https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_06/feeds_06.3.sleep.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_06/feeds_06.3.sleep.aql: Line 1: 3000 We should not remove the header if the file ends in .aql ... https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.4.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.4.ddl.aql: Line 20: * Description : Create a feed using the synthetic feed simulator adapter. WS Line 22: The synthetic feed simulator uses the Social-Data generator to generate data and simulate a feed. WS Line 23: The feed lasts a configured duration with data arriving at a configured rate (tweets per second). WS Line 25: * Issue : 711 WS https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.4.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.4.ddl.aql: Line 20: * Description : Create a feed using the synthetic feed simulator adapter. WS Line 22: The synthetic feed simulator uses the Social-Data generator to generate data and simulate a feed. WS Line 23: The feed lasts a configured duration with data arriving at a configured rate (tweets per second). WS Line 25: * Issue : 711 WS https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.4.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.4.ddl.aql: Line 20: * Description : Create a feed using the synthetic feed simulator adapter. WS Line 22: The synthetic feed simulator uses the Social-Data generator to generate data and simulate a feed. WS Line 23: The feed lasts a configured duration with data arriving at a configured rate (tweets per second). WS Line 25: * Issue : 711 WS https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.4.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.4.ddl.aql: Line 20: * Description : Create a feed using the synthetic feed simulator adapter. WS Line 22: The synthetic feed simulator uses the Social-Data generator to generate data and simulate a feed. WS Line 23: The feed lasts a configured duration with data arriving at a configured rate (tweets per second). WS Line 25: * Issue : 711 WS https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-app/src/test/resources/runtimets/testsuite.xml File asterix-app/src/test/resources/runtimets/testsuite.xml: Line 103: <!-- WS https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java File asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java: Line 86: private void restoreExternalObjects() throws Exception { Why is this methods called "restore..."? It seems that it's initializing stuff. https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-external-data/src/main/java/org/apache/asterix/external/api/IInputStreamProvider.java File asterix-external-data/src/main/java/org/apache/asterix/external/api/IInputStreamProvider.java: Line 31: public void setFeedLogManager(FeedLogManager feedLogManager); Why do we have these methods here? Couldn't we pass the configuration in when we create the IInputStreamProvider instance? (Also, I'm not sure why every InputStream would get a "FeedLogManager".) https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordFlowController.java File asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordFlowController.java: Line 21: public interface IRecordFlowController<T> extends IDataFlowController { Remove the whole interface? https://asterix-gerrit.ics.uci.edu/#/c/625/11/asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordReader.java File asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordReader.java: Line 74: public void setFeedLogManager(FeedLogManager feedLogManager);; Why would every RecordReader get a FeedLogManager? Is this really Feeds-specific? -- To view, visit https://asterix-gerrit.ics.uci.edu/625 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Idd1fccd136fa2645b2707bbf7c04e60991ae8d4a Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Michael Blow <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-HasComments: Yes
