abdullah alamoudi has posted comments on this change. Change subject: Introduced Local Filesystem Feed Adapter ......................................................................
Patch Set 3: (2 comments) That is correct. I have discussed this with Mike and Ian. file deletes are not of interest to us and file modifications will most likely cause problems. The expected user behavior is to specify a regular expression which will read files with names matching the expression. For example the expression ".*\\.txt" will match files with the extension txt and so when a user writes a new file, they are expected to create it with a name that doesn't end with .txt and when it is written completely rename it to [filename].txt which will kick off the ingestion. https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-app/data/local-dir/data/hello-world File asterix-app/data/local-dir/data/hello-world: Line 2: > I'm assuming the misspelling is for a purpose? Haha. Can you believe I didn't notice. I will fix it (even though it doesn't matter for our purpose) https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/LocalFileSystemFeedAdapter.java File asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/LocalFileSystemFeedAdapter.java: Line 56: // This needs to be fixed > This comment isn't clear to me. This needs to be fixed. The current stop mechanism is by flipping the boolean continue ingestion doesn't actually stop the feed immediately but wait for one more record. What if that record never came?!! Right now, there is an easy but not so good looking way to do this by first disconnecting the feed (which will call this method) and then add a dummy file with some text to cause the parser to skip a record. and break out of the loop. A better approach would be to have a different thread for waiting for new files and making stop actually kill that thread and exit the loop waiting for events immediately. -- To view, visit https://asterix-gerrit.ics.uci.edu/376 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I707756e3b4c9ffca4b55ec9817a08e5c16333010 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Steven Jacobs <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]> Gerrit-HasComments: Yes
