Murtadha Hubail has posted comments on this change. Change subject: Enabled Feed Tests and Added External Library tests ......................................................................
Patch Set 12: (34 comments) https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java File asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java: Line 92: * @return a map from dataverse -> list of uninstalled libraries. WS https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/java/org/apache/asterix/test/common/TestLibrarian.java File asterix-app/src/test/java/org/apache/asterix/test/common/TestLibrarian.java: Line 54: if (destinationDir.exists()) { Replace this if block by FileUtils.deleteQuietly(destinationDir); Line 76: public static void rmDirRecursively(String path) throws IOException { Remove this method. Line 95: if (installLibDir.exists()) { Replace this if block by FileUtils.deleteQuietly(installLibDir); https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.1.ddl.aql File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.1.ddl.aql: Line 22: Associate with the feed an external user-defined function. The UDF WS Line 23: finds topics in each tweet. A topic is identified by a #. WS https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.3.ddl.aql File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.3.ddl.aql: Line 22: Associate with the feed an external user-defined function. The UDF WS Line 23: finds topics in each tweet. A topic is identified by a #. WS https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.4.update.aql: Line 22: Associate with the feed an external user-defined function. The UDF WS Line 23: finds topics in each tweet. A topic is identified by a #. WS https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.5.query.aql File asterix-app/src/test/resources/runtimets/queries/external-library/typed_adapter/typed_adapter.5.query.aql: Line 22: Associate with the feed an external user-defined function. The UDF WS Line 23: finds topics in each tweet. A topic is identified by a #. WS https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.1.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.1.ddl.aql: Line 20: * Description : Create an adapter that uses external parser to parse data from files Update description https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.3.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.3.ddl.aql: Line 19: Just to be consistent, add the test description header. https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.4.update.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.4.update.aql: Line 20: * Description : Create a feed dataset and verify contents in Metadata Update description and date https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.5.query.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.5.query.aql: Line 18: */ Add description https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.7.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.7.ddl.aql: Line 20: * Description : Create a feed dataset and verify contents in Metadata update description https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.4.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_09/feeds_09.4.ddl.aql: Line 19: /* WSes https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.4.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_10/feeds_10.4.ddl.aql: Line 20: * Description : Create a feed using the synthetic feed simulator adapter. WSes https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.4.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/feeds_12/feeds_12.4.ddl.aql: Line 20: * Description : Create a feed using the synthetic feed simulator adapter. WSes https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.4.ddl.aql File asterix-app/src/test/resources/runtimets/queries/feeds/issue_230_feeds/issue_230_feeds.4.ddl.aql: Line 19: /* WSes https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java File asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java: Line 540: Delete this to avoid PASSED being printed twice in the case of txnqar https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java File asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java: Line 79: public synchronized IDataSourceAdapter createAdapter(IHyracksTaskContext ctx, int partition) throws Exception { why does this have to be synchronized? who shared the Factory object? https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordFlowController.java File asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordFlowController.java: Line 21: public interface IRecordFlowController<T> extends IDataFlowController { Unless you expect future uses for this interface, just delete it and replace its references by IDataFlowController https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java File asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java: Line 40: protected AtomicBoolean closed = new AtomicBoolean(false); final https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AbstractStreamRecordReader.java File asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AbstractStreamRecordReader.java: Line 98: reader.setFeedLogManager(feedLogManager); Just a note that if this method is called before setInputStream, then this will throw NPE https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/BasicInputStream.java File asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/BasicInputStream.java: Line 96: public void setFeedLogManager(FeedLogManager logManager) { Make the super implementation non abstract with unsupportedOp and remove this. Line 100: public void setController(AbstractFeedDataFlowController controller) { Make the super implementation non abstract with unsupportedOp and remove this. https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/SocketInputStream.java File asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/SocketInputStream.java: Line 111: public void setFeedLogManager(FeedLogManager logManager) { Make the super implementation non abstract with unsupportedOp and remove this. Line 115: public void setController(AbstractFeedDataFlowController controller) { Make the super implementation non abstract with unsupportedOp and remove this. https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/HDFSInputStreamProvider.java File asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/HDFSInputStreamProvider.java: Line 126: public void setFeedLogManager(FeedLogManager logManager) { Provide default implementation with unsupportedOp and remove this Line 130: public void setController(AbstractFeedDataFlowController controller) { Make the super implementation non abstract with unsupportedOp and remove this. https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/TwitterFirehoseInputStreamProvider.java File asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/provider/TwitterFirehoseInputStreamProvider.java: Line 122: public void setFeedLogManager(FeedLogManager logManager) { Make the super implementation non abstract with unsupportedOp and remove this. https://asterix-gerrit.ics.uci.edu/#/c/625/12/asterix-external-data/src/main/java/org/apache/asterix/external/parser/ADMDataParser.java File asterix-external-data/src/main/java/org/apache/asterix/external/parser/ADMDataParser.java: Line 87: private FeedLogManager feedLogManager; Remove this from here. -- To view, visit https://asterix-gerrit.ics.uci.edu/625 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Idd1fccd136fa2645b2707bbf7c04e60991ae8d4a Gerrit-PatchSet: 12 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Michael Blow <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-HasComments: Yes
