Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Xikui Wang has abandoned this change.

Change subject: Add record reader and parser for CAP messages
..


Abandoned

This change will be resubmitted to bad branch.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Xikui Wang has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 16:

Mike and I had a discussion about moving these research specific patches into 
the BADAsterix branch. So I will resubmit this patch into asterix_bad branch 
instead of master. Thanks for your comments. Please remove this one. :)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Xikui Wang has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 16:

Sorry for making so many changes in this patch. I found a couple of bugs when I 
was addressing your previous comments. Thanks!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 16: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1296/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 16:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/25/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 16:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1296/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 16:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3518/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#16).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based SAXParser. Element attributes are ignored
currently.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).
6. Extract message timestamp from the comment. All datetimes in element
content are treated as string due to different timezones.

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/CAP_0.xml
A asterixdb/asterix-app/data/cap_messages/CAP_1.xml
A asterixdb/asterix-app/data/cap_messages/CAP_2.xml
A asterixdb/asterix-app/data/cap_messages/CAP_3.xml
A asterixdb/asterix-app/data/cap_messages/CAP_4.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
35 files changed, 1,721 insertions(+), 36 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/16
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To 

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Till Westmann (Code Review)]

Till Westmann has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 13:

(8 comments)

Submitting some comments on a older patchset - sorry.

https://asterix-gerrit.ics.uci.edu/#/c/1269/13/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java:

Line 36: if (collection != null) {
Could we replace this by

this.recordLvl = Boolean.parseBoolean(collection) ? 1 : 0;

?


Line 52: final int IN_SCHEMA_DEFINITION = 7;
These are really helpful, but I think that they could be static final class 
members (or you could us an enum if you don't need the fact that they are ints).


Line 82: if (state == IN_START_OF_ELEMENT_NAME) {
A nested switch might be better.


Line 88: } else if (state == START_OF_PROLOG) {
Should this be "IN_PROLOG"?


Line 114: if (state == START_OF_ELEMENT_NAME) {
A nested switch might be better.


Line 129: return newRecordFormed;
This is always true, right?


https://asterix-gerrit.ics.uci.edu/#/c/1269/13/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java:

Line 101: private void resetPools() {
Inline reset pools?


Line 171: parentPath = String.join(".", curPathStack);
As it's only used inside this method, parentPath could be a local variable.
However, another concern is that String.join will probably allocate a few 
objects for every element. It would be interesting to see, if parsing many of 
these messages will lead to a lot of garbage collection.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 15:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3511/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 15
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 15:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3510/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 15
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#15).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based SAXParser. Element attributes are ignored
currently.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/CAP_0.xml
A asterixdb/asterix-app/data/cap_messages/CAP_1.xml
A asterixdb/asterix-app/data/cap_messages/CAP_2.xml
A asterixdb/asterix-app/data/cap_messages/CAP_3.xml
A asterixdb/asterix-app/data/cap_messages/CAP_4.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
35 files changed, 1,702 insertions(+), 36 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/15
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: 

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#14).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based SAXParser. Element attributes are ignored
currently.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/CAP_0.xml
A asterixdb/asterix-app/data/cap_messages/CAP_1.xml
A asterixdb/asterix-app/data/cap_messages/CAP_2.xml
A asterixdb/asterix-app/data/cap_messages/CAP_3.xml
A asterixdb/asterix-app/data/cap_messages/CAP_4.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
35 files changed, 1,696 insertions(+), 36 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/14
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: 

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3509/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 13: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1235/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 13:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1235/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#13).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based SAXParser. Element attributes are ignored
currently.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/CAP_0.xml
A asterixdb/asterix-app/data/cap_messages/CAP_1.xml
A asterixdb/asterix-app/data/cap_messages/CAP_2.xml
A asterixdb/asterix-app/data/cap_messages/CAP_3.xml
A asterixdb/asterix-app/data/cap_messages/CAP_4.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
35 files changed, 1,525 insertions(+), 36 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/13
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: 

Change in asterixdb[master]: Add record reader and parser for CAP messages

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 12:

(2 comments)

Nothing beside what Till has already mentioned.
For the object allocation, let's file an issue and fix it later (if it cause a 
problem to those who use it?).

I did the same for the classad parser. first version created many objects, the 
last one reused object pools.

https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql:

Line 18:  */
> Should we just leave this file out and renumber the following one(s)? I kno
+1


https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java:

Line 63: // TODO: simplify the state numbers (xikui)
> It would be better to use named constants instead of numbers for the state.
+1


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Till Westmann (Code Review)]

Till Westmann has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 12:

(10 comments)

Just a few comments - didn't think about your object allocation question  ...

https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql:

Line 18:  */
Should we just leave this file out and renumber the following one(s)? I know 
that we have such empty files in other tests, but maybe we don't need to add 
more of those ..


https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java:

Line 63: // TODO: simplify the state numbers (xikui)
It would be better to use named constants instead of numbers for the state. 
Also it seems that "curState" or just "state" would be a better name than 
"curStatus".


https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java:

Line 75: private SAXParserFactory capMessageSAXParserFactory;
Could be a local variable.


Line 130: rbList.get(0).write(out, true);
Do you expect that parsing changes the value of rbList.get(0)? If not, we could 
have a local variable for the record builder instead of accessing the list 
every time.


Line 244: 
out.write(BuiltinType.AINT32.getTypeTag().serialize());
This looks strange. Why do we need to serialize the type tag after the value 
here?


Line 329: private void handleNestedOrderedList(String fullPathName, int 
fieldNameIdx) throws HyracksDataException {
Could you add a comment to this method that describes the case in which it is 
needed?


https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java:

Line 31: import org.apache.hyracks.api.exceptions.HyracksDataException;
Some imports seem to be unused. Could you remove them?


https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java:

Line 206: public static final char QUESTION_MARK = '?';
It seems that these 4 are not used. Can we remove them again?


https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java:

Line 240: } catch (InterruptedException x) {
I'm not completely aware of the lifecycle here, but it seems that not setting 
the interrupted state isn't right.
@Abdullah: Can you give some insight on how this should work?


https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java:

Line 71: private static boolean fileNotExistsInList(LinkedList files, 
Path path) {
Should we use List<> instead of LinkedList<>? It seems that the chosen 
implementation should be immaterial to the interface.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Xikui Wang has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 12:

(1 comment)

I replaced the XML2Json part with SAXParser. A lot of objects are introduced to 
construct the record Is there any better solution? Thanks.

https://asterix-gerrit.ics.uci.edu/#/c/1269/12/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java:

Line 256: default:
> MAJOR SonarQube violation:
will resolve this with other feedbacks.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 12: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1214/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 12:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1214/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#12).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based SAXParser. Element attributes are ignored
currently.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/CAP_1.xml
A asterixdb/asterix-app/data/cap_messages/CAP_2.xml
A asterixdb/asterix-app/data/cap_messages/CAP_3.xml
A asterixdb/asterix-app/data/cap_messages/CAP_4.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
35 files changed, 1,425 insertions(+), 34 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/12
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3414/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3412/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#11).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based SAXParser. Element attributes are ignored
currently.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/CAP_1.xml
A asterixdb/asterix-app/data/cap_messages/CAP_2.xml
A asterixdb/asterix-app/data/cap_messages/CAP_3.xml
A asterixdb/asterix-app/data/cap_messages/CAP_4.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
35 files changed, 1,415 insertions(+), 34 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/11
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Till Westmann (Code Review)]

Till Westmann has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 10: Code-Review-1

Marking -1 due to the org.json issue.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Till Westmann (Code Review)]

Till Westmann has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 10:

Also added Ian as a reviewer as he's currently working on 
https://issues.apache.org/jira/browse/ASTERIXDB-1714 .

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Till Westmann (Code Review)]

Till Westmann has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 10:

Indeed the license has been moved to category X [1] very recently and so we 
should replace the use of org.json code with Jackson which we have already 
started [2]. 

[1] https://www.apache.org/legal/resolved#category-x

[2] 
https://github.com/apache/asterixdb/commit/4ad756062d7afde7c03801705dad4ca9854ccf87

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[abdullah alamoudi (Code Review)]

abdullah alamoudi has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 10:

this change looks good to me. There is however one issue:

http://www.json.org/license.html

if I understand correctly, ASF will not accept releases which contain this 
license. unfortunately, I think we will have to find another library which 
provide the needed functionality.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 10: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1132/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 10:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1132/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 10:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3298/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#10).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based on JSON library and ADMParser.
3. Fix ASTERIXDB-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/ER.xml
A asterixdb/asterix-app/data/cap_messages/HSA.xml
A asterixdb/asterix-app/data/cap_messages/STA.xml
A asterixdb/asterix-app/data/cap_messages/bigger.xml
A asterixdb/asterix-app/data/cap_messages/small_ER.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
36 files changed, 1,255 insertions(+), 34 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/10
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit 

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Xikui Wang has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 9:

@Till & Abdullah,

One thing I am not very sure in this patch is ASTERIXDB-1690. If you can help 
me verify that, it will be very helpful. :)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 9:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1090/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 9:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3244/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#9).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based on JSON library and ADMParser.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/ER.xml
A asterixdb/asterix-app/data/cap_messages/HSA.xml
A asterixdb/asterix-app/data/cap_messages/STA.xml
A asterixdb/asterix-app/data/cap_messages/bigger.xml
A asterixdb/asterix-app/data/cap_messages/small_ER.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
36 files changed, 1,255 insertions(+), 34 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/9
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit 

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 8: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1068/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 8:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1068/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 8:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3217/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#8).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based on JSON library and ADMParser.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add two options in configuration to allow user use CAP adaptor on a
limited set of 'well-formated' XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).
- "primary-key-mapping" = "ACCESS-PATH-TO-PRIMARY-KEY"  (Map certain
element in document to Primary key when it cannot be translated
properly. Check "caps-adaptor-collection" test case for more
detail).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/ER.xml
A asterixdb/asterix-app/data/cap_messages/HSA.xml
A asterixdb/asterix-app/data/cap_messages/STA.xml
A asterixdb/asterix-app/data/cap_messages/bigger.xml
A asterixdb/asterix-app/data/cap_messages/small_ER.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
36 

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 7:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3211/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

[Xikui Wang (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#7).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based on JSON library and ADMParser.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add two options in configuration to allow user use CAP adaptor on a
limited set of 'well-formated' XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).
- "primary-key-mapping" = "ACCESS-PATH-TO-PRIMARY-KEY"  (Map certain
element in document to Primary key when it cannot be translated
properly. Check "caps-adaptor-collection" test case for more
detail).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/ER.xml
A asterixdb/asterix-app/data/cap_messages/HSA.xml
A asterixdb/asterix-app/data/cap_messages/STA.xml
A asterixdb/asterix-app/data/cap_messages/bigger.xml
A asterixdb/asterix-app/data/cap_messages/small_ER.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
M asterixdb/asterix-app/src/test/resources/runtimets/only.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M