Change in asterixdb[master]: Add record reader and parser for CAP messages

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 15:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3510/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 15
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add record reader and parser for CAP messages

2016-12-11 Thread Xikui Wang (Code Review) 
Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#15).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based SAXParser. Element attributes are ignored
currently.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/CAP_0.xml
A asterixdb/asterix-app/data/cap_messages/CAP_1.xml
A asterixdb/asterix-app/data/cap_messages/CAP_2.xml
A asterixdb/asterix-app/data/cap_messages/CAP_3.xml
A asterixdb/asterix-app/data/cap_messages/CAP_4.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
35 files changed, 1,702 insertions(+), 36 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/15
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id:

Change in asterixdb[master]: Add record reader and parser for CAP messages

2016-12-11 Thread Xikui Wang (Code Review) 
Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1269

to look at the new patch set (#14).

Change subject: Add record reader and parser for CAP messages
..

Add record reader and parser for CAP messages

1. Add record reader for CAP message.
2. Add CAP message parser based SAXParser. Element attributes are ignored
currently.
3. Fix ASTERIX-1690: deadlock between close() and take() in FileSystemWatcher
4. Add test cases for using CAP adaptor in feed and load statement.
5. Add an option in configuration to allow user use CAP adaptor loading
certain XML documents.
- "collection" = "true" (Multiple records in one document, check
"document_with_records_collection.xml" as an example).

Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
---
A asterixdb/asterix-app/data/cap_messages/CAP_0.xml
A asterixdb/asterix-app/data/cap_messages/CAP_1.xml
A asterixdb/asterix-app/data/cap_messages/CAP_2.xml
A asterixdb/asterix-app/data/cap_messages/CAP_3.xml
A asterixdb/asterix-app/data/cap_messages/CAP_4.xml
A asterixdb/asterix-app/data/document_with_records_collection.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-collection/caps-adaptor-collection.4.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.3.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.4.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/caps-adaptor-no-collection/caps-adaptor-no-collection.5.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-collection/load-cap-collection.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.0.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.1.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/load/load-cap-messages/load-cap-messages.2.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-collection/cap-adaptor-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/cap-adaptor-no-collection/cap-adaptor-no-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-collection/load-cap-collection.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/load/load-cap-messages/load-cap-messages.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/CAPMessageRecordReader.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/CAPMessageParser.java
A 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/CAPMessageParserFactory.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/ParserFactoryProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/StreamRecordReaderProvider.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FileSystemWatcher.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
35 files changed, 1,696 insertions(+), 36 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/1269/14
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id:

Change in asterixdb[master]: Add record reader and parser for CAP messages

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Add record reader and parser for CAP messages
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3509/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1269
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia36101a0761973a9edb96b42d3dcc117661301da
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1749: fix breaking of lines using '\r'

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1749: fix breaking of lines using '\r'
..


Patch Set 1: Integration-Tests-1

Integration Tests Timed Out

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1286/ 
: ABORTED

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1382
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ica5ce0b82f1d2c6f2033be2ce20bf56a563fb57b
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1749: fix breaking of lines using '\r'

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1749: fix breaking of lines using '\r'
..


Patch Set 1:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1286/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1382
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ica5ce0b82f1d2c6f2033be2ce20bf56a563fb57b
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1749: fix breaking of lines using '\r'

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1749: fix breaking of lines using '\r'
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3508/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1382
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ica5ce0b82f1d2c6f2033be2ce20bf56a563fb57b
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1749: fix breaking of lines using '\r'

2016-12-11 Thread Till Westmann (Code Review) 
Till Westmann has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1382

Change subject: ASTERIXDB-1749: fix breaking of lines using '\r'
..

ASTERIXDB-1749: fix breaking of lines using '\r'

Change-Id: Ica5ce0b82f1d2c6f2033be2ce20bf56a563fb57b
---
M 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
A 
asterixdb/asterix-lang-sqlpp/src/test/java/org/apache/asterix/lang/sqlpp/parser/ParserTest.java
2 files changed, 55 insertions(+), 1 deletion(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/82/1382/1

diff --git 
a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
 
b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
index 767eacb..1195d37 100644
--- 
a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
+++ 
b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/parser/ScopeChecker.java
@@ -51,7 +51,7 @@
 }
 
 protected void setInput(String s) {
-inputLines = s.split("\n");
+inputLines = s.split("\n|\r\n?");
 }
 
 // Forbidden scopes are used to disallow, in a limit clause, variables
diff --git 
a/asterixdb/asterix-lang-sqlpp/src/test/java/org/apache/asterix/lang/sqlpp/parser/ParserTest.java
 
b/asterixdb/asterix-lang-sqlpp/src/test/java/org/apache/asterix/lang/sqlpp/parser/ParserTest.java
new file mode 100644
index 000..27679a9
--- /dev/null
+++ 
b/asterixdb/asterix-lang-sqlpp/src/test/java/org/apache/asterix/lang/sqlpp/parser/ParserTest.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.lang.sqlpp.parser;
+
+import org.apache.asterix.common.exceptions.AsterixException;
+import org.apache.asterix.lang.common.base.IParser;
+import org.apache.asterix.lang.common.base.IParserFactory;
+import org.junit.Test;
+
+public class ParserTest {
+
+protected void testLineEnding(String query) throws Exception {
+IParserFactory factory = new SqlppParserFactory();
+IParser parser = factory.createParser(query);
+try {
+parser.parse();
+} catch (AsterixException e) {
+if (!e.getMessage().contains("Syntax error: In line 3")) {
+throw new Exception("Unexpected error", e);
+}
+}
+}
+
+@Test
+public void testCR() throws Exception {
+testLineEnding("select\rvalue\r1");
+}
+
+@Test
+public void testCRLF() throws Exception {
+testLineEnding("select\r\nvalue\r\n1");
+}
+
+@Test
+public void testLF() throws Exception {
+testLineEnding("select\nvalue\n1");
+}
+}

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1382
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ica5ce0b82f1d2c6f2033be2ce20bf56a563fb57b
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann

[jira] [Created] (ASTERIXDB-1749) ArrayIndexOutOfBoundsException parsing SQL++ file using '\r'

2016-12-11 Thread Till (JIRA) 
Till created ASTERIXDB-1749:
---

 Summary: ArrayIndexOutOfBoundsException parsing SQL++ file using 
'\r'
 Key: ASTERIXDB-1749
 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1749
 Project: Apache AsterixDB
  Issue Type: Bug
  Components: Translator - SQL++
Reporter: Till
Assignee: Till


When parsing a SQL++ query that 
a) contains an error and
b) separates lines using '\r'
an ArrayIndexOutOfBoundsException is thrown.
For example is "select\rvalue\r1" throws 
{noformat}
java.lang.ArrayIndexOutOfBoundsException: 2

at 
org.apache.asterix.lang.common.parser.ScopeChecker.getLine(ScopeChecker.java:308)
at 
org.apache.asterix.lang.sqlpp.parser.SQLPPParser.getMessage(SQLPPParser.java:283)
at 
org.apache.asterix.lang.sqlpp.parser.SQLPPParser.parse(SQLPPParser.java:267)
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Enable the parallel sort in the optimizor.
..


Patch Set 3: Integration-Tests-1

Integration Tests Timed Out

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1285/ 
: ABORTED

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1381
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Enable the parallel sort in the optimizor.
..


Patch Set 3:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1285/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1381
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Enable the parallel sort in the optimizor.
..


Patch Set 3:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1381
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Enable the parallel sort in the optimizor.
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3507/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1381
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Wenhai Li (Code Review) 
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1381

to look at the new patch set (#3).

Change subject: Enable the parallel sort in the optimizor.
..

Enable the parallel sort in the optimizor.

Based on the MaterializingForwardOperatorDescriptor, enable the parallel sort 
in full round.
1. Add LogicalOperator Foward to enable MaterializingForwardOperatorDescriptor 
in the logical optimizor.
2. Introduce hint "psort" before sort to enable the user to use parallel sort.
3. Add the ApplyParalleoSortRule to introduce ReplicateOperator, 
AggregateOperator, ForwardOperator before SortOperator.
4. Apply ParallelStableSortPOperator into SortOperator if the following 
conditions are satisfied:
   a. Single column sort.
   b. non-topK condition inlined.

Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
---
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/ParallelStableSortPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAggFuncIntoStandaloneAggregateRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SweepIllegalNonfunctionalFunctions.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanSpecialFlatteningCheckVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/FunctionCollection.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
A asterixdb/asterix-app/data/skew/zipfan1.tbl
A asterixdb/asterix-app/data/skew/zipfan2.tbl
M asterixdb/asterix-app/pom.xml
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/AsterixAlgebricksHyracksIntegrationUtil.java
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/ParallelSortRuntimeTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/TestPartitionComputerFactory.java
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_float/avg_float.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int16/avg_int16.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int32/avg_int32.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int64/avg_int64.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int8/avg_int8.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/count_01/count_01.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-avg_01/global-avg_01.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-rg/global-rg.1.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_double/local-avg_double.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_float/local-avg_float.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int32/local-avg_int32.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int64/local-avg_int64.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int8/local-avg_int8.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-rg/local-rg.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double-null/rg-double-null.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double/rg-double.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.1.ddl.aql
A

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Enable the parallel sort in the optimizor.
..


Patch Set 2:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1381
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Enable the parallel sort in the optimizor.
..


Patch Set 2:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3506/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1381
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Wenhai Li (Code Review) 
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1381

to look at the new patch set (#2).

Change subject: Enable the parallel sort in the optimizor.
..

Enable the parallel sort in the optimizor.

Based on the MaterializingForwardOperatorDescriptor, enable the parallel sort 
in full round.
1. Add LogicalOperator Foward to enable MaterializingForwardOperatorDescriptor 
in the logical optimizor.
2. Introduce hint "psort" before sort to enable the user to use parallel sort.
3. Add the ApplyParalleoSortRule to introduce ReplicateOperator, 
AggregateOperator, ForwardOperator before SortOperator.
4. Apply ParallelStableSortPOperator into SortOperator if the following 
conditions are satisfied:
   a. Single column sort.
   b. non-topK condition inlined.

Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
---
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/ParallelStableSortPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAggFuncIntoStandaloneAggregateRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SweepIllegalNonfunctionalFunctions.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanSpecialFlatteningCheckVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/FunctionCollection.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
A asterixdb/asterix-app/data/skew/zipfan1.tbl
A asterixdb/asterix-app/data/skew/zipfan2.tbl
M asterixdb/asterix-app/pom.xml
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/AsterixAlgebricksHyracksIntegrationUtil.java
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/ParallelSortRuntimeTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/TestPartitionComputerFactory.java
M asterixdb/asterix-app/src/test/resources/runtimets/only.xml
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_float/avg_float.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int16/avg_int16.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int32/avg_int32.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int64/avg_int64.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int8/avg_int8.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/count_01/count_01.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-avg_01/global-avg_01.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-rg/global-rg.1.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_double/local-avg_double.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_float/local-avg_float.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int32/local-avg_int32.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int64/local-avg_int64.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int8/local-avg_int8.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-rg/local-rg.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double-null/rg-double-null.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double/rg-double.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.3.query.aql
A

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Enable the parallel sort in the optimizor.
..


Patch Set 1:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1381
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Enable the parallel sort in the optimizor.
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3505/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1381
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Enable the parallel sort in the optimizor.

2016-12-11 Thread Wenhai Li (Code Review) 
Wenhai Li has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1381

Change subject: Enable the parallel sort in the optimizor.
..

Enable the parallel sort in the optimizor.

Based on the MaterializingForwardOperatorDescriptor, enable the parallel sort 
in full round.
1. Add LogicalOperator Foward to enable MaterializingForwardOperatorDescriptor 
in the logical optimizor.
2. Introduce hint "psort" before sort to enable the user to use parallel sort.
3. Add the ApplyParalleoSortRule to introduce ReplicateOperator, 
AggregateOperator, ForwardOperator before SortOperator.
4. Apply ParallelStableSortPOperator into SortOperator if the following 
conditions are satisfied:
   a. Single column sort.
   b. non-topK condition inlined.

Change-Id: I755d909b7fb34d97a1950195ed3a5a5c9d4afdd5
---
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/ParallelStableSortPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAggFuncIntoStandaloneAggregateRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SweepIllegalNonfunctionalFunctions.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanSpecialFlatteningCheckVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AqlPlusExpressionToPlanTranslator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/FunctionCollection.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlus.jj
A asterixdb/asterix-app/data/skew/zipfan1.tbl
A asterixdb/asterix-app/data/skew/zipfan2.tbl
M asterixdb/asterix-app/pom.xml
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/AsterixAlgebricksHyracksIntegrationUtil.java
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/ParallelSortRuntimeTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/TestPartitionComputerFactory.java
M asterixdb/asterix-app/src/test/resources/runtimets/only.xml
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_float/avg_float.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int16/avg_int16.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int32/avg_int32.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int64/avg_int64.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int8/avg_int8.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/count_01/count_01.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-avg_01/global-avg_01.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-rg/global-rg.1.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_double/local-avg_double.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_float/local-avg_float.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.1.ddl.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int32/local-avg_int32.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int64/local-avg_int64.3.query.aql
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int8/local-avg_int8.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-rg/local-rg.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double-null/rg-double-null.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double/rg-double.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.1.ddl.aql