http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-rest/src/site/markdown/index.md ---------------------------------------------------------------------- diff --git a/vxquery-rest/src/site/markdown/index.md b/vxquery-rest/src/site/markdown/index.md new file mode 100644 index 0000000..703df65 --- /dev/null +++ b/vxquery-rest/src/site/markdown/index.md @@ -0,0 +1,249 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> +# VXQuery REST Server + +VXQuery REST Server allows users to submit queries and receive results either synchronously or +asynchronously through the exposed REST API. Along with the statement to be executed, few other parameters can be given as +well. Complete REST API specification can be found at [REST API Specification](specification.html). + +## Installation + +No additional steps needed to be taken to get the REST Server up and running. That is, setting up a VXQuery cluster will +automatically start the REST Server on port `8080`. Please see [VXQuery Cluster Setup](../user_cluster_installation.html) +to understand how a VXQuery cluster is setup. + +## Getting Started + +Suppose we want to execute a very simple XQuery like: + +``` +for $x in (1, 2.0, 3) return $x +``` + +### Async (Default Mode) Example + +If we want to send this, following will be the plain HTTP request. + +``` +GET http://127.0.1.1:39003/vxquery/query?statement=for+%24x+in+%281%2C+2.0%2C+3%29+return+%24x HTTP/1.1 +``` + +Note the query parameter `statement=for+%24x+in+%281%2C+2.0%2C+3%29+return+%24x` in which the above mentioned statement +has been encoded. If we send this request using **cURL**, it will look like follows. + +#### Accept: application/json + +``` +curl -i -H "Accept: application/json" -X GET "http://localhost:39003/vxquery/query?statement=for+%24x+in+%281%2C+2.0%2C+3%29+return+%24x"; +``` + +and the response is, + +``` +HTTP/1.1 200 OK +transfer-encoding: chunked +connection: keep-alive +Access-Control-Allow-Origin: * +Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept +content-type: application/json +content-length: 320 + +{ + "status": "success", + "requestId": "b0cbe06f-3454-4422-ba23-59150e1c1400", + "statement": "for $x in (1, 2.0, 3) return $x", + "abstractSyntaxTree": null, + "translatedExpressionTree": null, + "optimizedExpressionTree": null, + "runtimePlan": null, + "metrics": { + "compileTime": 0, + "elapsedTime": 0 + }, + "resultId": 6, + "resultUrl": "/vxquery/query/result/6" +} +``` + +#### Accept: application/xml + +``` +curl -i -H "Accept: application/xml" -X GET "http://localhost:39003/vxquery/query?statement=for+%24x+in+%281%2C+2.0%2C+3%29+return+%24x"; +``` + +and the response is, + +``` +HTTP/1.1 200 OK +transfer-encoding: chunked +connection: keep-alive +Access-Control-Allow-Origin: * +Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept +content-type: application/xml +content-length: 403 + +<?xml version="1.0" encoding="UTF-8" standalone="yes"?> +<asyncQueryResponse> + <requestId>d0c2c0ef-2e46-4153-9d4b-1ef4593184e7</requestId> + <metrics> + <compileTime>0</compileTime> + <elapsedTime>0</elapsedTime> + </metrics> + <statement>for $x in (1, 2.0, 3) return $x</statement> + <resultId>8</resultId> + <resultUrl>/vxquery/query/result/8</resultUrl> +</asyncQueryResponse> +``` + +#### Result Fetching + +Since we have used the default mode (**async**), we only got the **resultId**. Now we have to send another request asking +for the actual results. Send a cURL request to `/vxquery/query/result/8` to fetch results for result ID 8. + +##### Accept: application/json + +``` +curl -i -H "Accept: application/json" -X GET "http://localhost:39003/vxquery/query/result/8"; +``` + +and the response is, + +``` +HTTP/1.1 200 OK +transfer-encoding: chunked +connection: keep-alive +Access-Control-Allow-Origin: * +Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept +content-type: application/json +content-length: 137 + +{ + "status": "success", + "requestId": "d0c2c0ef-2e46-4153-9d4b-1ef4593184e7", + "results": "1\n2\n3\n", + "metrics": { + "compileTime": 0, + "elapsedTime": 0 + } +} +``` + +Note the *results* in the JSON content in the response. + +##### Accept: application/xml + +``` +curl -i -H "Accept: application/xml" -X GET "http://localhost:39003/vxquery/query/result/8"; +``` + +and the response is, + +``` +HTTP/1.1 200 OK +transfer-encoding: chunked +connection: keep-alive +Access-Control-Allow-Origin: * +Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept +content-type: application/xml +content-length: 298 + +<?xml version="1.0" encoding="UTF-8" standalone="yes"?> +<queryResultResponse> + <requestId>d0c2c0ef-2e46-4153-9d4b-1ef4593184e7</requestId> + <metrics> + <compileTime>0</compileTime> + <elapsedTime>0</elapsedTime> + </metrics> + <results>1 +2 +3 +</results> +</queryResultResponse> +``` + +Note the *<results></results>* in the XML content in the response. + +### Sync (Synchronous Mode) Example + +Similarly to what we did under async requests, we can send the query requests here as well, but with the added query parameter +`mode=sync` which is to indicate that the response should be a synchronous one. That is, we wait for the query to be +executed and the response to arrive. + +``` +curl -i -H "Accept: application/xml" -X GET \ +"http://localhost:39003/vxquery/query?statement=for+%24x+in+%281%2C+2.0%2C+3%29+return+%24x&mode=sync"; +``` + +and the response now contains **results** instead of the **resultId** we received previously. + +``` +HTTP/1.1 200 OK +transfer-encoding: chunked +connection: keep-alive +Access-Control-Allow-Origin: * +Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept +content-type: application/xml +content-length: 353 + +<?xml version="1.0" encoding="UTF-8" standalone="yes"?> +<syncQueryResponse> + <requestId>93b67f50-4f14-4304-a9b2-f75b4a736df3</requestId> + <metrics> + <compileTime>0</compileTime> + <elapsedTime>0</elapsedTime> + </metrics> + <statement>for $x in (1, 2.0, 3) return $x</statement> + <results>1 +2 +3 +</results> +</syncQueryResponse> +``` + +Similarly with `accept:application/json`, + +``` +curl -i -H "Accept: application/json" -X GET \ +"http://localhost:39003/vxquery/query?statement=for+%24x+in+%281%2C+2.0%2C+3%29+return+%24x&mode=sync"; +``` + +and the response is, + +``` +HTTP/1.1 200 OK +transfer-encoding: chunked +connection: keep-alive +Access-Control-Allow-Origin: * +Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept +content-type: application/json +content-length: 291 + +{ + "status": "success", + "requestId": "8010a699-a6f2-423c-91e1-8ac17cd5c5cd", + "statement": "for $x in (1, 2.0, 3) return $x", + "abstractSyntaxTree": null, + "translatedExpressionTree": null, + "optimizedExpressionTree": null, + "runtimePlan": null, + "metrics": { + "compileTime": 0, + "elapsedTime": 0 + }, + "results": "1\n2\n3\n" +} +```
http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-rest/src/site/markdown/specification.md ---------------------------------------------------------------------- diff --git a/vxquery-rest/src/site/markdown/specification.md b/vxquery-rest/src/site/markdown/specification.md new file mode 100644 index 0000000..3977439 --- /dev/null +++ b/vxquery-rest/src/site/markdown/specification.md @@ -0,0 +1,102 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> +# REST API Specification + +Swagger configuration of the REST API can be found +[here](https://cwiki.apache.org/confluence/display/VXQUERY/SwaggerIO+Configuration). + +**NOTE:** This REST API supports both **content types**, `application/json` and `application/xml`. Based on the `accept` header +of your query request, REST API will return the results wither in *json* form or *XML* form. Returned content type +defaults to `application/json` if no `accept` header is present. + +Base Path **${host}/vxquery** + +## Query Request + +Request of this type should be submitted for a given *query* to be executed. Depending on the value of the parameter +`mode`, a **synchronous** response or an **asynchronous** response will be returned + +`*` required + +| Path | Method |Parameters | Type | Description | +| ------ | ------ | ------ | ----- |----- | +| /query | GET |statement* | string | Statement to be executed | +| | |mode | string | `sync` or `async`. **async** will return an asynchronous response **(default: async)** | +| | |compileOnly | boolean | If `true`, statement will be compiled, but won't be executed (default: false) | +| | |optimization | integer | Optimization level (0 - 2,147,483,647). (Default: Full optimization) | +| | |frameSize | integer | Frame size in bytes (default: 65536) | +| | |repeatExecutions|integer | Number of times to repeat execution (default: 1) | +| | |metrics | boolean | If `true`, returns metrics (compile and execution time) with the response (default: false) | +| | |showAbstractSyntaxTree | boolean | Shows abstract syntax tree if `true` (default: false) | +| | |showTranslatedExpressionTree | boolean | Shows translated expression tree if `true` (default: false) | +| | |showOptimizedExpressionTree | boolean | Shows optimized expression tree if `true` (default: false) | +| | |showRuntimePlan| boolean | Shows runtime plan if set to `true` (default: false) | + +### Synchronous Query Response + +Received only when `mode` is set to `sync` in the query request above. + +| Attribute | Type | Description | +| ------ | ------ | ------ | +|statement* |string | Statement submitted to be executed | +|status* |string | `success` to indicate that the query execution was successful | +|requestId* |string | A unique ID assigned to the request sent earlier | +|abstractSyntaxTree |string | Abstract Syntax Tree if requested in the query request. Else `null` | +|translatedExpressionTree |string | Translated Expression Tree if requested in the query request. Else `null` | +|optimizedExpressionTree |string | Optimized Expression Tree if requested in the query request. Else `null` | +|runtimePlan |string | Runtime plan if requested in the query request. Else `null` | +|metrics |metrics | Metrics (`compileTime` and `elapsedTime`) if requested in the query request | +|results* |string | Results of the query/statement submitted for execution | + +### Asynchronous Query Response + +Received only when `mode` is set to `async` (which is the default value) in the query request above. + +| Attribute | Type | Description | +| ------ | ------ | ------ | +|statement* |string | Statement submitted to be executed | +|status* |string | `success` to indicate that the query execution was successful | +|requestId* |string | A unique ID assigned to the request sent earlier | +|abstractSyntaxTree |string | Abstract Syntax Tree if requested in the query request. Else `null` | +|translatedExpressionTree |string | Translated Expression Tree if requested in the query request. Else `null` | +|optimizedExpressionTree |string | Optimized Expression Tree if requested in the query request. Else `null` | +|runtimePlan |string | Runtime plan if requested in the query request. Else `null` | +|metrics |metrics | Metrics (`compileTime` and `elapsedTime`) if requested in the query request | +|resultId* |string | Result ID of the query submitted for execution. This ID is required later for result fetching | +|resultUrl* |string | URL from which the results of the submitted query can be retrieved | + +### Result fetching (After an Asynchronous Query Response) + +The **resultId** received in the asynchronous query response needs to be submitted as a +**path parameter** (`/query/result/${resultId}`) to the REST API in order to retrieve the corresponding results. + +| Path | Method |Parameters | Type | Description | +| ------ | ------ | ------ | ----- |----- | +| /query/result/${resultId} | GET | metrics | boolean | If `true`, returns metrics (compile and execution time) with the response (default: false) | + +*** + +## Error Response + +In any of the above scenarios, if an error occurred while processing, REST API will return an *Error Response* as +specified below. + +| Attribute | Type | Description | +| ------ | ------ | ------ | +|status* |string | `fatal` to indicate that the query execution was failed at some point | +|requestId* |string | A unique ID assigned to the request sent | +|error* |Error | An error object which include an error code with an error message. {code: xxx, message: "error message"} | http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-rest/src/site/site.xml ---------------------------------------------------------------------- diff --git a/vxquery-rest/src/site/site.xml b/vxquery-rest/src/site/site.xml new file mode 100644 index 0000000..bd9b9ab --- /dev/null +++ b/vxquery-rest/src/site/site.xml @@ -0,0 +1,53 @@ +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one or more + ~ contributor license agreements. See the NOTICE file distributed with + ~ this work for additional information regarding copyright ownership. + ~ The ASF licenses this file to You under the Apache License, Version 2.0 + ~ (the "License"); you may not use this file except in compliance with + ~ the License. You may obtain a copy of the License at + ~ + ~ http://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, software + ~ distributed under the License is distributed on an "AS IS" BASIS, + ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + ~ See the License for the specific language governing permissions and + ~ limitations under the License. + --> +<project name="VXQuery"> + <bannerLeft> + <name>VXQuery</name> + <src>../images/VXQuery.png</src> + <href>../index.html</href> + </bannerLeft> + + <bannerRight> + <name>Apache Software Foundation</name> + <src>../images/asf_logo_wide.png</src> + <href>http://www.apache.org/</href> + </bannerRight> + + <skin> + <groupId>org.apache.maven.skins</groupId> + <artifactId>maven-fluido-skin</artifactId> + <version>1.5</version> + </skin> + + <body> + <menu name="VXQuery REST API"> + <item name="Overview" href="index.html"/> + <item name="Specification" href="specification.html"/> + </menu> + + <menu ref="reports"/> + <footer><![CDATA[ + <div class="row-fluid">Apache VXQuery, VXQuery, Apache, the Apache + feather logo, and the Apache VXQuery project logo are either + registered trademarks or trademarks of The Apache Software + Foundation in the United States and other countries. + All other marks mentioned may be trademarks or registered + trademarks of their respective owners. + </div>]]> + </footer> + </body> +</project> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-rest/src/test/java/org/apache/vxquery/rest/AbstractRestServerTest.java ---------------------------------------------------------------------- diff --git a/vxquery-rest/src/test/java/org/apache/vxquery/rest/AbstractRestServerTest.java b/vxquery-rest/src/test/java/org/apache/vxquery/rest/AbstractRestServerTest.java new file mode 100644 index 0000000..3d4b124 --- /dev/null +++ b/vxquery-rest/src/test/java/org/apache/vxquery/rest/AbstractRestServerTest.java @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.vxquery.rest; + +import java.net.URI; +import java.util.concurrent.TimeUnit; + +import javax.ws.rs.HttpMethod; + +import org.apache.http.HttpEntity; +import org.apache.http.HttpHeaders; +import org.apache.http.client.methods.CloseableHttpResponse; +import org.apache.http.client.methods.HttpGet; +import org.apache.http.client.methods.HttpPost; +import org.apache.http.client.methods.HttpUriRequest; +import org.apache.http.client.utils.HttpClientUtils; +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.http.impl.client.HttpClients; +import org.apache.vxquery.app.VXQueryApplication; +import org.apache.vxquery.app.util.LocalClusterUtil; +import org.apache.vxquery.app.util.RestUtils; +import org.apache.vxquery.rest.request.QueryRequest; +import org.apache.vxquery.rest.request.QueryResultRequest; +import org.apache.vxquery.rest.response.AsyncQueryResponse; +import org.apache.vxquery.rest.response.QueryResponse; +import org.apache.vxquery.rest.response.QueryResultResponse; +import org.apache.vxquery.rest.response.SyncQueryResponse; +import org.apache.vxquery.rest.service.VXQueryConfig; +import org.apache.vxquery.rest.service.VXQueryService; +import org.junit.AfterClass; +import org.junit.Assert; +import org.junit.BeforeClass; + +import io.netty.handler.codec.http.HttpResponseStatus; + +/** + * Abstract test class to be used for {@link VXQueryApplication} related tests. + * These tests are expected to use the REST API for querying and fetching + * results + * + * @author Erandi Ganepola + */ +public class AbstractRestServerTest { + + protected static LocalClusterUtil vxqueryLocalCluster = new LocalClusterUtil(); + protected static String restIpAddress; + protected static int restPort; + protected static VXQueryService vxQueryService; + + @BeforeClass + public static void setUp() throws Exception { + vxqueryLocalCluster.init(new VXQueryConfig()); + vxQueryService = vxqueryLocalCluster.getVxQueryService(); + restIpAddress = vxqueryLocalCluster.getIpAddress(); + restPort = vxqueryLocalCluster.getRestPort(); + } + + @AfterClass + public static void tearDown() throws Exception { + vxqueryLocalCluster.deinit(); + } + + protected static String normalize(String string) { + if (string == null) { + return null; + } + + return string.replace("\r\n", "").replace("\n", "").replace("\r", ""); + } + + protected static void checkMetrics(QueryResponse response, boolean showMetrics) { + if (showMetrics) { + Assert.assertTrue(response.getMetrics().getCompileTime() > 0); + Assert.assertTrue(response.getMetrics().getElapsedTime() > 0); + } else { + Assert.assertTrue(response.getMetrics().getCompileTime() == 0); + Assert.assertTrue(response.getMetrics().getElapsedTime() == 0); + } + } + + protected static void checkResults(AsyncQueryResponse response, boolean compileOnly) { + if (compileOnly) { + Assert.assertNull(response.getResultUrl()); + Assert.assertEquals(0, response.getResultId()); + } else { + Assert.assertTrue(response.getResultUrl().startsWith(Constants.RESULT_URL_PREFIX)); + Assert.assertNotEquals(0, response.getResultId()); + } + } + + protected static void checkResults(SyncQueryResponse response, boolean compileOnly) { + if (compileOnly) { + Assert.assertNull(response.getResults()); + } else { + Assert.assertNotNull(response.getResults()); + Assert.assertFalse(response.getResults().isEmpty()); + } + } + + /** + * Submit a {@link QueryRequest} and fetth the resulting + * {@link AsyncQueryResponse} + * + * @param uri + * uri of the GET request + * @param accepts + * application/json | application/xml + * @param method + * Http Method to be used to send the request + * @return Response received for the query request + * @throws Exception + */ + protected static <T> T getQuerySuccessResponse(URI uri, String accepts, Class<T> type, String method) + throws Exception { + CloseableHttpClient httpClient = HttpClients.custom().setConnectionTimeToLive(20, TimeUnit.SECONDS).build(); + + try { + HttpUriRequest request = getRequest(uri, method); + + if (accepts != null) { + request.setHeader(HttpHeaders.ACCEPT, accepts); + } + + try (CloseableHttpResponse httpResponse = httpClient.execute(request)) { + Assert.assertEquals(HttpResponseStatus.OK.code(), httpResponse.getStatusLine().getStatusCode()); + if (accepts != null) { + Assert.assertEquals(accepts, httpResponse.getFirstHeader(HttpHeaders.CONTENT_TYPE).getValue()); + } + + HttpEntity entity = httpResponse.getEntity(); + Assert.assertNotNull(entity); + + String response = RestUtils.readEntity(entity); + return RestUtils.mapEntity(response, type, accepts); + } + } finally { + HttpClientUtils.closeQuietly(httpClient); + } + } + + /** + * Fetch the {@link QueryResultResponse} from query result endpoint once the + * corresponding {@link QueryResultRequest} is given. + * + * @param resultRequest + * {@link QueryResultRequest} + * @param accepts + * expected + * + * <pre> + * Accepts + * </pre> + * + * header in responses + * @param method + * Http Method to be used to send the request + * @return query result response received + * @throws Exception + */ + protected static QueryResultResponse getQueryResultResponse(QueryResultRequest resultRequest, String accepts, + String method) throws Exception { + URI uri = RestUtils.buildQueryResultURI(resultRequest, restIpAddress, restPort); + CloseableHttpClient httpClient = HttpClients.custom().setConnectionTimeToLive(20, TimeUnit.SECONDS).build(); + try { + HttpUriRequest request = getRequest(uri, method); + + if (accepts != null) { + request.setHeader(HttpHeaders.ACCEPT, accepts); + } + + try (CloseableHttpResponse httpResponse = httpClient.execute(request)) { + if (accepts != null) { + Assert.assertEquals(accepts, httpResponse.getFirstHeader(HttpHeaders.CONTENT_TYPE).getValue()); + } + Assert.assertEquals(httpResponse.getStatusLine().getStatusCode(), HttpResponseStatus.OK.code()); + + HttpEntity entity = httpResponse.getEntity(); + Assert.assertNotNull(entity); + + String response = RestUtils.readEntity(entity); + return RestUtils.mapEntity(response, QueryResultResponse.class, accepts); + } + } finally { + HttpClientUtils.closeQuietly(httpClient); + } + } + + /** + * Creates a POST or GET request accordingly from the given {@link URI} + * + * @param uri + * URI to which the request us to be sent + * @param method + * Http method- GET or POST + * @return request + */ + protected static HttpUriRequest getRequest(URI uri, String method) { + HttpUriRequest request; + switch (method) { + case HttpMethod.POST: + request = new HttpPost(uri); + break; + case HttpMethod.GET: + default: + request = new HttpGet(uri); + } + + return request; + } +} http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-rest/src/test/java/org/apache/vxquery/rest/ErrorResponseTest.java ---------------------------------------------------------------------- diff --git a/vxquery-rest/src/test/java/org/apache/vxquery/rest/ErrorResponseTest.java b/vxquery-rest/src/test/java/org/apache/vxquery/rest/ErrorResponseTest.java new file mode 100644 index 0000000..12b61f3 --- /dev/null +++ b/vxquery-rest/src/test/java/org/apache/vxquery/rest/ErrorResponseTest.java @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.vxquery.rest; + +import static org.apache.vxquery.app.util.RestUtils.buildQueryResultURI; +import static org.apache.vxquery.app.util.RestUtils.buildQueryURI; +import static org.apache.vxquery.rest.Constants.ErrorCodes.INVALID_INPUT; +import static org.apache.vxquery.rest.Constants.ErrorCodes.NOT_FOUND; +import static org.apache.vxquery.rest.Constants.ErrorCodes.PROBLEM_WITH_QUERY; +import static org.apache.vxquery.rest.Constants.HttpHeaderValues.CONTENT_TYPE_JSON; +import static org.apache.vxquery.rest.Constants.HttpHeaderValues.CONTENT_TYPE_XML; + +import java.net.URI; +import java.util.concurrent.TimeUnit; + +import javax.ws.rs.HttpMethod; + +import org.apache.http.HttpEntity; +import org.apache.http.HttpHeaders; +import org.apache.http.client.methods.CloseableHttpResponse; +import org.apache.http.client.methods.HttpUriRequest; +import org.apache.http.client.utils.HttpClientUtils; +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.http.impl.client.HttpClients; +import org.apache.vxquery.app.util.RestUtils; +import org.apache.vxquery.rest.request.QueryRequest; +import org.apache.vxquery.rest.request.QueryResultRequest; +import org.apache.vxquery.rest.response.ErrorResponse; +import org.junit.Assert; +import org.junit.Test; + +/** + * Tests error codes of the possible error responses that can be received for + * erroneous queries. + * + * @author Erandi Ganepola + */ +public class ErrorResponseTest extends AbstractRestServerTest { + + @Test + public void testInvalidInput01() throws Exception { + QueryRequest request = new QueryRequest(" "); + runTest(buildQueryURI(request, restIpAddress, restPort), null, INVALID_INPUT); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, INVALID_INPUT); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, INVALID_INPUT); + } + + @Test + public void testInvalidInput02() throws Exception { + QueryRequest request = new QueryRequest(""); + runTest(buildQueryURI(request, restIpAddress, restPort), null, 405); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, 405); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, 405); + } + + @Test + public void testInvalidQuery01() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1,2,3) return $y"); + runTest(buildQueryURI(request, restIpAddress, restPort), null, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, PROBLEM_WITH_QUERY); + } + + @Test + public void testInvalidQuery02() throws Exception { + QueryRequest request = new QueryRequest("for x in (1,2,3) return $x"); + runTest(buildQueryURI(request, restIpAddress, restPort), null, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, PROBLEM_WITH_QUERY); + } + + @Test + public void testInvalidQuery03() throws Exception { + QueryRequest request = new QueryRequest("insert nodes <book></book> into doc(\"abcd.xml\")/books"); + runTest(buildQueryURI(request, restIpAddress, restPort), null, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, PROBLEM_WITH_QUERY); + } + + @Test + public void testInvalidQuery04() throws Exception { + QueryRequest request = new QueryRequest("delete nodes /a/b//node()"); + runTest(buildQueryURI(request, restIpAddress, restPort), null, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, PROBLEM_WITH_QUERY); + } + + @Test + public void testInvalidResultId() throws Exception { + QueryResultRequest request = new QueryResultRequest(1000); + runTest(buildQueryResultURI(request, restIpAddress, restPort), null, NOT_FOUND); + runTest(buildQueryResultURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, NOT_FOUND); + runTest(buildQueryResultURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, NOT_FOUND); + } + + @Test + public void testSyncInvalidInput01() throws Exception { + QueryRequest request = new QueryRequest(" "); + request.setAsync(false); + runTest(buildQueryURI(request, restIpAddress, restPort), null, INVALID_INPUT); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, INVALID_INPUT); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, INVALID_INPUT); + } + + @Test + public void testSyncInvalidInput02() throws Exception { + QueryRequest request = new QueryRequest(""); + request.setAsync(false); + runTest(buildQueryURI(request, restIpAddress, restPort), null, 405); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, 405); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, 405); + } + + @Test + public void testSyncInvalidQuery01() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1,2,3) return $y"); + request.setAsync(false); + runTest(buildQueryURI(request, restIpAddress, restPort), null, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, PROBLEM_WITH_QUERY); + } + + @Test + public void testSyncInvalidQuery02() throws Exception { + QueryRequest request = new QueryRequest("for x in (1,2,3) return $x"); + request.setAsync(false); + runTest(buildQueryURI(request, restIpAddress, restPort), null, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, PROBLEM_WITH_QUERY); + } + + @Test + public void testSyncInvalidQuery03() throws Exception { + QueryRequest request = new QueryRequest("insert nodes <book></book> into doc(\"abcd.xml\")/books"); + request.setAsync(false); + runTest(buildQueryURI(request, restIpAddress, restPort), null, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, PROBLEM_WITH_QUERY); + } + + @Test + public void testSyncInvalidQuery04() throws Exception { + QueryRequest request = new QueryRequest("delete nodes /a/b//node()"); + request.setAsync(false); + runTest(buildQueryURI(request, restIpAddress, restPort), null, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_JSON, PROBLEM_WITH_QUERY); + runTest(buildQueryURI(request, restIpAddress, restPort), CONTENT_TYPE_XML, PROBLEM_WITH_QUERY); + } + + private void runTest(URI uri, String accepts, int expectedStatusCode) throws Exception { + runTest(uri, accepts, expectedStatusCode, HttpMethod.GET); + runTest(uri, accepts, expectedStatusCode, HttpMethod.POST); + } + + private void runTest(URI uri, String accepts, int expectedStatusCode, String httpMethod) throws Exception { + CloseableHttpClient httpClient = HttpClients.custom().setConnectionTimeToLive(20, TimeUnit.SECONDS).build(); + + ErrorResponse errorResponse; + try { + HttpUriRequest request = getRequest(uri, httpMethod); + if (accepts != null) { + request.setHeader(HttpHeaders.ACCEPT, accepts); + } + + try (CloseableHttpResponse httpResponse = httpClient.execute(request)) { + Assert.assertEquals(expectedStatusCode, httpResponse.getStatusLine().getStatusCode()); + if (accepts != null) { + Assert.assertEquals(accepts, httpResponse.getFirstHeader(HttpHeaders.CONTENT_TYPE).getValue()); + } + + HttpEntity entity = httpResponse.getEntity(); + Assert.assertNotNull(entity); + + String response = RestUtils.readEntity(entity); + errorResponse = RestUtils.mapEntity(response, ErrorResponse.class, accepts); + } + } finally { + HttpClientUtils.closeQuietly(httpClient); + } + + Assert.assertNotNull(errorResponse); + Assert.assertNotNull(errorResponse.getError().getMessage()); + Assert.assertEquals(errorResponse.getError().getCode(), expectedStatusCode); + } +} http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-rest/src/test/java/org/apache/vxquery/rest/SuccessAsyncResponseTest.java ---------------------------------------------------------------------- diff --git a/vxquery-rest/src/test/java/org/apache/vxquery/rest/SuccessAsyncResponseTest.java b/vxquery-rest/src/test/java/org/apache/vxquery/rest/SuccessAsyncResponseTest.java new file mode 100644 index 0000000..d1385c8 --- /dev/null +++ b/vxquery-rest/src/test/java/org/apache/vxquery/rest/SuccessAsyncResponseTest.java @@ -0,0 +1,289 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.vxquery.rest; + +import static org.apache.vxquery.rest.Constants.HttpHeaderValues.CONTENT_TYPE_JSON; +import static org.apache.vxquery.rest.Constants.HttpHeaderValues.CONTENT_TYPE_XML; + +import java.net.URI; + +import javax.ws.rs.HttpMethod; + +import org.apache.vxquery.app.util.RestUtils; +import org.apache.vxquery.rest.request.QueryRequest; +import org.apache.vxquery.rest.request.QueryResultRequest; +import org.apache.vxquery.rest.response.APIResponse; +import org.apache.vxquery.rest.response.AsyncQueryResponse; +import org.apache.vxquery.rest.response.ErrorResponse; +import org.apache.vxquery.rest.response.QueryResultResponse; +import org.apache.vxquery.rest.service.Status; +import org.junit.Assert; +import org.junit.Test; + +/** + * This class tests the success responses received for XQueries submitted. i.e + * we are submitting correct queries which are expected to return a predictable + * result. All the parameters that are expected to be sent with query requests + * are subjected to test in this test class + * + * @author Erandi Ganepola + */ +public class SuccessAsyncResponseTest extends AbstractRestServerTest { + + @Test + public void testSimpleQuery001() throws Exception { + QueryRequest request = new QueryRequest("1+1"); + request.setShowAbstractSyntaxTree(true); + request.setShowOptimizedExpressionTree(true); + request.setShowRuntimePlan(true); + request.setShowTranslatedExpressionTree(true); + request.setShowMetrics(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSimpleQuery002() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setShowAbstractSyntaxTree(true); + request.setShowOptimizedExpressionTree(true); + request.setShowRuntimePlan(true); + request.setShowTranslatedExpressionTree(true); + request.setShowMetrics(true); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSimpleQuery003() throws Exception { + QueryRequest request = new QueryRequest("1+2+3"); + request.setShowAbstractSyntaxTree(false); + request.setShowOptimizedExpressionTree(false); + request.setShowRuntimePlan(false); + request.setShowTranslatedExpressionTree(false); + request.setShowMetrics(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSimpleQuery004() throws Exception { + QueryRequest request = new QueryRequest("fn:true()"); + request.setShowAbstractSyntaxTree(false); + request.setShowOptimizedExpressionTree(false); + request.setShowRuntimePlan(true); + request.setShowTranslatedExpressionTree(false); + request.setShowMetrics(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterNone() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterMetrics() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setShowMetrics(true); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterAST() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setShowAbstractSyntaxTree(true); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterOptimization() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setOptimization(10000); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterFrameSize() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setFrameSize((int) Math.pow(2, 12)); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterCompileOnly() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setCompileOnly(true); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterOET() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setShowOptimizedExpressionTree(true); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterTET() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setShowTranslatedExpressionTree(true); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterRP() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setShowRuntimePlan(true); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + private void runTest(String contentType, QueryRequest request) throws Exception { + runTest(contentType, request, HttpMethod.GET); + runTest(contentType, request, HttpMethod.POST); + } + + private void runTest(String contentType, QueryRequest request, String httpMethod) throws Exception { + URI queryEndpointUri = RestUtils.buildQueryURI(request, restIpAddress, restPort); + + /* + * ========== Query Response Testing ========== + */ + // Testing the accuracy of VXQueryService class + AsyncQueryResponse expectedAsyncQueryResponse = (AsyncQueryResponse) vxQueryService.execute(request); + + Assert.assertEquals(Status.SUCCESS.toString(), expectedAsyncQueryResponse.getStatus()); + Assert.assertEquals(request.getStatement(), expectedAsyncQueryResponse.getStatement()); + checkResults(expectedAsyncQueryResponse, request.isCompileOnly()); + checkMetrics(expectedAsyncQueryResponse, request.isShowMetrics()); + if (request.isShowMetrics()) { + Assert.assertTrue(expectedAsyncQueryResponse.getMetrics().getCompileTime() > 0); + } else { + Assert.assertTrue(expectedAsyncQueryResponse.getMetrics().getCompileTime() == 0); + } + if (request.isShowAbstractSyntaxTree()) { + Assert.assertNotNull(expectedAsyncQueryResponse.getAbstractSyntaxTree()); + } else { + Assert.assertNull(expectedAsyncQueryResponse.getAbstractSyntaxTree()); + } + if (request.isShowTranslatedExpressionTree()) { + Assert.assertNotNull(expectedAsyncQueryResponse.getTranslatedExpressionTree()); + } else { + Assert.assertNull(expectedAsyncQueryResponse.getTranslatedExpressionTree()); + } + if (request.isShowOptimizedExpressionTree()) { + Assert.assertNotNull(expectedAsyncQueryResponse.getOptimizedExpressionTree()); + } else { + Assert.assertNull(expectedAsyncQueryResponse.getOptimizedExpressionTree()); + } + if (request.isShowRuntimePlan()) { + Assert.assertNotNull(expectedAsyncQueryResponse.getRuntimePlan()); + } else { + Assert.assertNull(expectedAsyncQueryResponse.getRuntimePlan()); + } + + // Testing the accuracy of REST server and servlets + AsyncQueryResponse actualAsyncQueryResponse = + getQuerySuccessResponse(queryEndpointUri, contentType, AsyncQueryResponse.class, httpMethod); + + Assert.assertNotNull(actualAsyncQueryResponse.getRequestId()); + Assert.assertEquals(request.getStatement(), actualAsyncQueryResponse.getStatement()); + Assert.assertEquals(Status.SUCCESS.toString(), actualAsyncQueryResponse.getStatus()); + checkResults(actualAsyncQueryResponse, request.isCompileOnly()); + checkMetrics(actualAsyncQueryResponse, request.isShowMetrics()); + // Cannot check this because Runtime plan include some object IDs which differ + // Assert.assertEquals(expectedAsyncQueryResponse.getRuntimePlan(), + // actualAsyncQueryResponse.getRuntimePlan()); + if (request.isShowRuntimePlan()) { + Assert.assertNotNull(actualAsyncQueryResponse.getRuntimePlan()); + } else { + Assert.assertNull(actualAsyncQueryResponse.getRuntimePlan()); + } + Assert.assertEquals(normalize(expectedAsyncQueryResponse.getOptimizedExpressionTree()), + normalize(actualAsyncQueryResponse.getOptimizedExpressionTree())); + Assert.assertEquals(normalize(expectedAsyncQueryResponse.getTranslatedExpressionTree()), + normalize(actualAsyncQueryResponse.getTranslatedExpressionTree())); + Assert.assertEquals(normalize(expectedAsyncQueryResponse.getAbstractSyntaxTree()), + normalize(actualAsyncQueryResponse.getAbstractSyntaxTree())); + + /* + * ========== Query Result Response Testing ======== + */ + QueryResultRequest resultRequest = new QueryResultRequest(actualAsyncQueryResponse.getResultId()); + resultRequest.setShowMetrics(true); + + if (request.isCompileOnly()) { + APIResponse resultResponse = vxQueryService.getResult(resultRequest); + Assert.assertTrue(resultResponse instanceof ErrorResponse); + } else { + QueryResultResponse expectedResultResponse = (QueryResultResponse) vxQueryService.getResult(resultRequest); + Assert.assertEquals(expectedResultResponse.getStatus(), Status.SUCCESS.toString()); + Assert.assertNotNull(expectedResultResponse.getResults()); + + QueryResultResponse actualResultResponse = getQueryResultResponse(resultRequest, contentType, httpMethod); + Assert.assertEquals(actualResultResponse.getStatus(), Status.SUCCESS.toString()); + Assert.assertNotNull(actualResultResponse.getResults()); + Assert.assertNotNull(actualResultResponse.getRequestId()); + Assert.assertEquals(normalize(expectedResultResponse.getResults()), + normalize(actualResultResponse.getResults())); + if (resultRequest.isShowMetrics()) { + Assert.assertTrue(actualResultResponse.getMetrics().getElapsedTime() > 0); + } else { + Assert.assertTrue(actualResultResponse.getMetrics().getElapsedTime() == 0); + } + + } + } +} http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-rest/src/test/java/org/apache/vxquery/rest/SuccessSyncResponseTest.java ---------------------------------------------------------------------- diff --git a/vxquery-rest/src/test/java/org/apache/vxquery/rest/SuccessSyncResponseTest.java b/vxquery-rest/src/test/java/org/apache/vxquery/rest/SuccessSyncResponseTest.java new file mode 100644 index 0000000..31e0162 --- /dev/null +++ b/vxquery-rest/src/test/java/org/apache/vxquery/rest/SuccessSyncResponseTest.java @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.vxquery.rest; + +import static org.apache.vxquery.rest.Constants.HttpHeaderValues.CONTENT_TYPE_JSON; +import static org.apache.vxquery.rest.Constants.HttpHeaderValues.CONTENT_TYPE_XML; + +import java.net.URI; + +import javax.ws.rs.HttpMethod; + +import org.apache.vxquery.app.util.RestUtils; +import org.apache.vxquery.rest.request.QueryRequest; +import org.apache.vxquery.rest.response.SyncQueryResponse; +import org.apache.vxquery.rest.service.Status; +import org.junit.Assert; +import org.junit.Test; + +/** + * This class tests the success responses received for XQueries submitted. i.e + * we are submitting correct queries which are expected to return a predictable + * result. All the parameters that are expected to be sent with query requests + * are subjected to test in this test class + * + * @author Erandi Ganepola + */ +public class SuccessSyncResponseTest extends AbstractRestServerTest { + + @Test + public void testSimpleQuery001() throws Exception { + QueryRequest request = new QueryRequest("1+1"); + request.setShowAbstractSyntaxTree(true); + request.setShowOptimizedExpressionTree(true); + request.setShowRuntimePlan(true); + request.setShowTranslatedExpressionTree(true); + request.setShowMetrics(false); + request.setAsync(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSimpleQuery002() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setShowAbstractSyntaxTree(true); + request.setShowOptimizedExpressionTree(true); + request.setShowRuntimePlan(true); + request.setShowTranslatedExpressionTree(true); + request.setShowMetrics(true); + request.setAsync(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSimpleQuery003() throws Exception { + QueryRequest request = new QueryRequest("1+2+3"); + request.setShowAbstractSyntaxTree(false); + request.setShowOptimizedExpressionTree(false); + request.setShowRuntimePlan(false); + request.setShowTranslatedExpressionTree(false); + request.setShowMetrics(false); + request.setAsync(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSimpleQuery004() throws Exception { + QueryRequest request = new QueryRequest("fn:true()"); + request.setShowAbstractSyntaxTree(false); + request.setShowOptimizedExpressionTree(false); + request.setShowRuntimePlan(true); + request.setShowTranslatedExpressionTree(false); + request.setShowMetrics(false); + request.setAsync(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterNone() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setAsync(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterCompileOnly() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setCompileOnly(true); + request.setAsync(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + @Test + public void testSingleParameterRepeatExecutions() throws Exception { + QueryRequest request = new QueryRequest("for $x in (1, 2.0, 3) return $x"); + request.setRepeatExecutions(5); + request.setAsync(false); + + runTest(null, request); + runTest(CONTENT_TYPE_JSON, request); + runTest(CONTENT_TYPE_XML, request); + } + + private void runTest(String contentType, QueryRequest request) throws Exception { + runTest(contentType, request, HttpMethod.GET); + runTest(contentType, request, HttpMethod.POST); + } + + private void runTest(String contentType, QueryRequest request, String httpMethod) throws Exception { + URI queryEndpointUri = RestUtils.buildQueryURI(request, restIpAddress, restPort); + + /* + * ========== Query Response Testing ========== + */ + // Testing the accuracy of VXQueryService class + SyncQueryResponse expectedSyncQueryResponse = (SyncQueryResponse) vxQueryService.execute(request); + + Assert.assertEquals(Status.SUCCESS.toString(), expectedSyncQueryResponse.getStatus()); + Assert.assertEquals(request.getStatement(), expectedSyncQueryResponse.getStatement()); + checkResults(expectedSyncQueryResponse, request.isCompileOnly()); + checkMetrics(expectedSyncQueryResponse, request.isShowMetrics()); + if (request.isShowMetrics()) { + Assert.assertTrue(expectedSyncQueryResponse.getMetrics().getCompileTime() > 0); + } else { + Assert.assertTrue(expectedSyncQueryResponse.getMetrics().getCompileTime() == 0); + } + if (request.isShowAbstractSyntaxTree()) { + Assert.assertNotNull(expectedSyncQueryResponse.getAbstractSyntaxTree()); + } else { + Assert.assertNull(expectedSyncQueryResponse.getAbstractSyntaxTree()); + } + if (request.isShowTranslatedExpressionTree()) { + Assert.assertNotNull(expectedSyncQueryResponse.getTranslatedExpressionTree()); + } else { + Assert.assertNull(expectedSyncQueryResponse.getTranslatedExpressionTree()); + } + if (request.isShowOptimizedExpressionTree()) { + Assert.assertNotNull(expectedSyncQueryResponse.getOptimizedExpressionTree()); + } else { + Assert.assertNull(expectedSyncQueryResponse.getOptimizedExpressionTree()); + } + if (request.isShowRuntimePlan()) { + Assert.assertNotNull(expectedSyncQueryResponse.getRuntimePlan()); + } else { + Assert.assertNull(expectedSyncQueryResponse.getRuntimePlan()); + } + + // Testing the accuracy of REST server and servlets + SyncQueryResponse actualSyncQueryResponse = + getQuerySuccessResponse(queryEndpointUri, contentType, SyncQueryResponse.class, httpMethod); + + Assert.assertNotNull(actualSyncQueryResponse.getRequestId()); + Assert.assertEquals(request.getStatement(), actualSyncQueryResponse.getStatement()); + Assert.assertEquals(Status.SUCCESS.toString(), actualSyncQueryResponse.getStatus()); + checkMetrics(actualSyncQueryResponse, request.isShowMetrics()); + checkResults(actualSyncQueryResponse, request.isCompileOnly()); + // Cannot check this because Runtime plan include some object IDs which differ + // Assert.assertEquals(expectedSyncQueryResponse.getRuntimePlan(), + // actualSyncQueryResponse.getRuntimePlan()); + if (request.isShowRuntimePlan()) { + Assert.assertNotNull(actualSyncQueryResponse.getRuntimePlan()); + } else { + Assert.assertNull(actualSyncQueryResponse.getRuntimePlan()); + } + Assert.assertEquals(normalize(expectedSyncQueryResponse.getOptimizedExpressionTree()), + normalize(actualSyncQueryResponse.getOptimizedExpressionTree())); + Assert.assertEquals(normalize(expectedSyncQueryResponse.getTranslatedExpressionTree()), + normalize(actualSyncQueryResponse.getTranslatedExpressionTree())); + Assert.assertEquals(normalize(expectedSyncQueryResponse.getAbstractSyntaxTree()), + normalize(actualSyncQueryResponse.getAbstractSyntaxTree())); + + /* + * ========== Query Result Response Testing ======== + */ + String expectedResults = expectedSyncQueryResponse.getResults(); + String actualResults = actualSyncQueryResponse.getResults(); + if (!request.isCompileOnly()) { + Assert.assertNotNull(expectedResults); + Assert.assertNotNull(actualResults); + } + Assert.assertEquals(normalize(expectedResults), normalize(actualResults)); + } +} http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-rest/src/test/resources/vxquery.properties ---------------------------------------------------------------------- diff --git a/vxquery-rest/src/test/resources/vxquery.properties b/vxquery-rest/src/test/resources/vxquery.properties new file mode 100644 index 0000000..8870f79 --- /dev/null +++ b/vxquery-rest/src/test/resources/vxquery.properties @@ -0,0 +1,30 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Number of processors to be used for query processing +#org.apache.vxquery.available_processors=-1 + +# Number of local node controllers to be created when creating a local hyracks cluster +org.apache.vxquery.local_nc=1 + +# Join hash size +#org.apache.vxquery.join_hash=-1 + +# Maximum Data Size +#org.apache.vxquery.data_size=-1 + +# hdfs config directory +#org.apache.vxquery.hdfs_config=foo/bar http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-server/pom.xml ---------------------------------------------------------------------- diff --git a/vxquery-server/pom.xml b/vxquery-server/pom.xml index e572c36..7331fde 100644 --- a/vxquery-server/pom.xml +++ b/vxquery-server/pom.xml @@ -41,7 +41,7 @@ <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>appassembler-maven-plugin</artifactId> - <version>1.1.1</version> + <version>2.0.0</version> <executions> <execution> <configuration> @@ -53,6 +53,10 @@ <program> <mainClass>org.apache.hyracks.control.cc.CCDriver</mainClass> <name>vxquerycc</name> + <commandLineArguments> + <commandLineArgument>-app-cc-main-class</commandLineArgument> + <commandLineArgument>org.apache.vxquery.app.VXQueryApplication</commandLineArgument> + </commandLineArguments> </program> <program> <mainClass>org.apache.hyracks.control.nc.NCDriver</mainClass> @@ -139,15 +143,13 @@ <dependencies> <dependency> <groupId>org.apache.vxquery</groupId> - <artifactId>apache-vxquery-core</artifactId> + <artifactId>apache-vxquery-rest</artifactId> <version>0.7-SNAPSHOT</version> </dependency> - <dependency> <groupId>org.apache.hyracks</groupId> <artifactId>hyracks-control-cc</artifactId> </dependency> - <dependency> <groupId>org.apache.hyracks</groupId> <artifactId>hyracks-control-nc</artifactId> http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-xtest/pom.xml ---------------------------------------------------------------------- diff --git a/vxquery-xtest/pom.xml b/vxquery-xtest/pom.xml index a00bec2..a32d913 100644 --- a/vxquery-xtest/pom.xml +++ b/vxquery-xtest/pom.xml @@ -144,6 +144,12 @@ </dependency> <dependency> + <groupId>org.apache.vxquery</groupId> + <artifactId>apache-vxquery-rest</artifactId> + <version>0.7-SNAPSHOT</version> + </dependency> + + <dependency> <groupId>org.apache.hyracks</groupId> <artifactId>hyracks-api</artifactId> </dependency> http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestClusterUtil.java ---------------------------------------------------------------------- diff --git a/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestClusterUtil.java b/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestClusterUtil.java index 0e5b481..4d2ae8a 100644 --- a/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestClusterUtil.java +++ b/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestClusterUtil.java @@ -19,81 +19,45 @@ package org.apache.vxquery.xtest; import org.apache.hyracks.api.client.HyracksConnection; import org.apache.hyracks.client.dataset.HyracksDataset; -import org.apache.hyracks.control.cc.ClusterControllerService; -import org.apache.hyracks.control.common.controllers.CCConfig; -import org.apache.hyracks.control.common.controllers.NCConfig; -import org.apache.hyracks.control.nc.NodeControllerService; +import org.apache.vxquery.app.util.LocalClusterUtil; +import org.apache.vxquery.rest.service.VXQueryConfig; -import java.io.File; import java.io.IOException; -import java.net.InetAddress; -import java.net.UnknownHostException; public class TestClusterUtil { - private static final int CLIENT_NET_PORT = 39000; - private static final int CLUSTER_NET_PORT = 39001; - private static final int PROFILE_DUMP_PERIOD = 10000; - private static final String CC_HOST = "localhost"; - private static final String NODE_ID = "nc1"; - private static final String IO_DEVICES = "target/tmp/indexFolder"; - private static HyracksConnection hcc; private static HyracksDataset hds; + public static final LocalClusterUtil localClusterUtil = new LocalClusterUtil(); + private TestClusterUtil() { } - public static CCConfig createCCConfig() throws UnknownHostException { - String publicAddress = InetAddress.getLocalHost().getHostAddress(); - CCConfig ccConfig = new CCConfig(); - ccConfig.clientNetIpAddress = publicAddress; - ccConfig.clientNetPort = CLIENT_NET_PORT; - ccConfig.clusterNetIpAddress = publicAddress; - ccConfig.clusterNetPort = CLUSTER_NET_PORT; - ccConfig.profileDumpPeriod = PROFILE_DUMP_PERIOD; - return ccConfig; - } + private static VXQueryConfig loadConfiguration(XTestOptions opts) { + VXQueryConfig vxqConfig = new VXQueryConfig(); - public static NCConfig createNCConfig() throws UnknownHostException { - String publicAddress = InetAddress.getLocalHost().getHostAddress(); - NCConfig ncConfig1 = new NCConfig(); - ncConfig1.ccHost = CC_HOST; - ncConfig1.ccPort = CLUSTER_NET_PORT; - ncConfig1.clusterNetIPAddress = publicAddress; - ncConfig1.dataIPAddress = publicAddress; - ncConfig1.resultIPAddress = publicAddress; - ncConfig1.nodeId = NODE_ID; - ncConfig1.ioDevices = IO_DEVICES; - return ncConfig1; + vxqConfig.setAvailableProcessors(opts.threads); + vxqConfig.setFrameSize(opts.frameSize); + vxqConfig.setHdfsConf(opts.hdfsConf); + + return vxqConfig; } - public static ClusterControllerService startCC(XTestOptions opts) throws IOException { - CCConfig ccConfig = createCCConfig(); - File outDir = new File("target/ClusterController"); - outDir.mkdirs(); - File ccRoot = File.createTempFile(TestRunner.class.getName(), ".data", outDir); - ccRoot.delete(); - ccRoot.mkdir(); - ccConfig.ccRoot = ccRoot.getAbsolutePath(); + public static void startCluster(XTestOptions opts, LocalClusterUtil localClusterUtil) throws IOException { try { - ClusterControllerService cc = new ClusterControllerService(ccConfig); - cc.start(); - hcc = new HyracksConnection(ccConfig.clientNetIpAddress, ccConfig.clientNetPort); - hds = new HyracksDataset(hcc, opts.frameSize, opts.threads); - return cc; + VXQueryConfig config = loadConfiguration(opts); + localClusterUtil.init(config); + hcc = (HyracksConnection) localClusterUtil.getConnection(); + hds = (HyracksDataset) localClusterUtil.getDataset(); } catch (Exception e) { throw new IOException(e); } - } - public static NodeControllerService startNC() throws IOException { - NCConfig ncConfig = createNCConfig(); + public static void stopCluster(LocalClusterUtil localClusterUtil) throws IOException { try { - NodeControllerService nc = new NodeControllerService(ncConfig); - nc.start(); - return nc; + localClusterUtil.deinit(); } catch (Exception e) { throw new IOException(e); } @@ -107,13 +71,4 @@ public class TestClusterUtil { return hds; } - public static void stopCluster(ClusterControllerService cc, NodeControllerService nc) throws IOException { - try { - nc.stop(); - cc.stop(); - } catch (Exception e) { - throw new IOException(e); - } - } - } http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestRunner.java ---------------------------------------------------------------------- diff --git a/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestRunner.java b/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestRunner.java index fa0a900..8ef9426 100644 --- a/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestRunner.java +++ b/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/TestRunner.java @@ -14,101 +14,54 @@ */ package org.apache.vxquery.xtest; +import static org.apache.vxquery.rest.Constants.HttpHeaderValues.CONTENT_TYPE_JSON; + import java.io.File; -import java.io.FileInputStream; import java.io.IOException; -import java.io.InputStreamReader; -import java.io.Reader; +import java.net.URI; +import java.net.URISyntaxException; import java.net.UnknownHostException; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.EnumSet; -import java.util.List; +import java.nio.charset.StandardCharsets; import java.util.Map; import java.util.regex.Matcher; import java.util.regex.Pattern; -import org.apache.commons.io.IOUtils; -import org.apache.commons.lang3.StringUtils; -import org.apache.hyracks.api.client.IHyracksClientConnection; -import org.apache.hyracks.api.client.NodeControllerInfo; -import org.apache.hyracks.api.comm.IFrame; -import org.apache.hyracks.api.comm.IFrameTupleAccessor; -import org.apache.hyracks.api.comm.VSizeFrame; -import org.apache.hyracks.api.dataset.DatasetJobRecord; -import org.apache.hyracks.api.dataset.IHyracksDataset; -import org.apache.hyracks.api.dataset.IHyracksDatasetReader; -import org.apache.hyracks.api.dataset.ResultSetId; -import org.apache.hyracks.api.exceptions.HyracksException; -import org.apache.hyracks.api.job.JobFlag; -import org.apache.hyracks.api.job.JobId; -import org.apache.hyracks.api.job.JobSpecification; -import org.apache.hyracks.control.nc.resources.memory.FrameManager; -import org.apache.hyracks.dataflow.common.comm.io.ResultFrameTupleAccessor; -import org.apache.vxquery.compiler.CompilerControlBlock; -import org.apache.vxquery.compiler.algebricks.VXQueryGlobalDataFactory; -import org.apache.vxquery.context.DynamicContext; -import org.apache.vxquery.context.DynamicContextImpl; -import org.apache.vxquery.context.RootStaticContextImpl; -import org.apache.vxquery.context.StaticContextImpl; +import javax.xml.bind.JAXBException; + +import org.apache.commons.io.FileUtils; +import org.apache.http.HttpEntity; +import org.apache.http.HttpHeaders; +import org.apache.http.HttpStatus; +import org.apache.http.client.methods.CloseableHttpResponse; +import org.apache.http.client.methods.HttpPost; +import org.apache.http.client.utils.HttpClientUtils; +import org.apache.http.entity.StringEntity; +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.http.impl.client.HttpClients; +import org.apache.vxquery.app.util.RestUtils; import org.apache.vxquery.exceptions.ErrorCode; import org.apache.vxquery.exceptions.SystemException; -import org.apache.vxquery.result.ResultUtils; -import org.apache.vxquery.xmlquery.query.VXQueryCompilationListener; -import org.apache.vxquery.xmlquery.query.XMLQueryCompiler; +import org.apache.vxquery.rest.request.QueryRequest; +import org.apache.vxquery.rest.response.APIResponse; +import org.apache.vxquery.rest.response.ErrorResponse; +import org.apache.vxquery.rest.response.SyncQueryResponse; +import org.codehaus.jackson.map.ObjectMapper; public class TestRunner { + private static final Pattern EMBEDDED_SYSERROR_PATTERN = Pattern.compile("(\\p{javaUpperCase}{4}\\d{4})"); - private List<String> collectionList; + private XTestOptions opts; - private IHyracksClientConnection hcc; - private IHyracksDataset hds; public TestRunner(XTestOptions opts) throws UnknownHostException { this.opts = opts; - this.collectionList = new ArrayList<String>(); } public void open() throws Exception { - hcc = TestClusterUtil.getConnection(); - hds = TestClusterUtil.getDataset(); - } - - protected static TestConfiguration getIndexConfiguration(TestCase testCase) { - XTestOptions opts = new XTestOptions(); - opts.verbose = false; - opts.threads = 1; - opts.showQuery = true; - opts.showResult = true; - opts.hdfsConf = "src/test/resources/hadoop/conf"; - opts.catalog = StringUtils.join(new String[] { "src", "test", "resources", "VXQueryCatalog.xml" }, - File.separator); - TestConfiguration indexConf = new TestConfiguration(); - indexConf.options = opts; - String baseDir = new File(opts.catalog).getParent(); - try { - String root = new File(baseDir).getCanonicalPath(); - indexConf.testRoot = new File(root + "/./"); - indexConf.resultOffsetPath = new File(root + "/./ExpectedResults/"); - indexConf.sourceFileMap = testCase.getSourceFileMap(); - indexConf.xqueryFileExtension = ".xq"; - indexConf.xqueryxFileExtension = "xqx"; - indexConf.xqueryQueryOffsetPath = new File(root + "/./Queries/XQuery/"); - } catch (IOException e) { - e.printStackTrace(); - } - return indexConf; - } public TestCaseResult run(final TestCase testCase) { TestCaseResult res = new TestCaseResult(testCase); - TestCase testCaseIndex = new TestCase(getIndexConfiguration(testCase)); - testCaseIndex.setFolder("Indexing/Partition-1/"); - testCaseIndex.setName("showIndexes"); - runQuery(testCaseIndex, res); - String[] collections = res.result.split("\n"); - this.collectionList = Arrays.asList(collections); runQueries(testCase, res); return res; } @@ -121,83 +74,32 @@ public class TestRunner { long start = System.currentTimeMillis(); try { - try { - if (opts.showQuery) { + String query = FileUtils.readFileToString(testCase.getXQueryFile(), "UTF-8"); - FileInputStream query = new FileInputStream(testCase.getXQueryFile()); - System.err.println("***Query for " + testCase.getXQueryDisplayName() + ": "); - System.err.println(IOUtils.toString(query, "UTF-8")); - query.close(); - } - - VXQueryCompilationListener listener = new VXQueryCompilationListener(opts.showAST, opts.showTET, - opts.showOET, opts.showRP); - - Map<String, NodeControllerInfo> nodeControllerInfos = null; - if (hcc != null) { - nodeControllerInfos = hcc.getNodeControllerInfos(); - } + if (opts.showQuery) { + System.err.println("***Query for " + testCase.getXQueryDisplayName() + ": "); + System.err.println(query); + } - XMLQueryCompiler compiler = new XMLQueryCompiler(listener, nodeControllerInfos, opts.frameSize, - opts.hdfsConf); - Reader in = new InputStreamReader(new FileInputStream(testCase.getXQueryFile()), "UTF-8"); - CompilerControlBlock ccb = new CompilerControlBlock( - new StaticContextImpl(RootStaticContextImpl.INSTANCE), - new ResultSetId(testCase.getXQueryDisplayName().hashCode()), testCase.getSourceFileMap()); - compiler.compile(testCase.getXQueryDisplayName(), in, ccb, opts.optimizationLevel, collectionList); - JobSpecification spec = compiler.getModule().getHyracksJobSpecification(); - in.close(); - - DynamicContext dCtx = new DynamicContextImpl(compiler.getModule().getModuleContext()); - spec.setGlobalJobDataFactory(new VXQueryGlobalDataFactory(dCtx.createFactory())); - - spec.setMaxReattempts(0); - JobId jobId = hcc.startJob(spec, EnumSet.of(JobFlag.PROFILE_RUNTIME)); - - FrameManager resultDisplayFrameMgr = new FrameManager(spec.getFrameSize()); - IFrame frame = new VSizeFrame(resultDisplayFrameMgr); - IHyracksDatasetReader reader = hds.createReader(jobId, ccb.getResultSetId()); - // TODO(tillw) remove this loop once the IHyracksDatasetReader reliably returns the correct exception - while (reader.getResultStatus() == DatasetJobRecord.Status.RUNNING) { - Thread.sleep(1); - } - IFrameTupleAccessor frameTupleAccessor = new ResultFrameTupleAccessor(); - res.result = ""; - while (reader.read(frame) > 0) { - res.result += ResultUtils.getStringFromBuffer(frame.getBuffer(), frameTupleAccessor); - frame.getBuffer().clear(); - } - res.result.trim(); - hcc.waitForCompletion(jobId); - } catch (HyracksException e) { - Throwable t = e; - while (t.getCause() != null) { - t = t.getCause(); - } - final String message = t.getMessage(); - if (message != null) { - Matcher m = EMBEDDED_SYSERROR_PATTERN.matcher(message); - if (m.find()) { - String eCode = m.group(1); - throw new SystemException(ErrorCode.valueOf(eCode), e); - } + QueryRequest request = createQueryRequest(opts, query); + APIResponse response = sendQueryRequest(request, testCase.getSourceFileMap()); + if (response instanceof SyncQueryResponse) { + res.result = ((SyncQueryResponse) response).getResults(); + } else { + System.err.println("Error response: Failure when running the query"); + ErrorResponse errorResponse = (ErrorResponse) response; + Matcher m = EMBEDDED_SYSERROR_PATTERN.matcher(errorResponse.getError().getMessage()); + + Exception e = new RuntimeException("Failed to run the query"); + if (m.find()) { + String eCode = m.group(1); + throw new SystemException(ErrorCode.valueOf(eCode), e); + } else { + throw e; } - throw e; } } catch (Throwable e) { - // Check for nested SystemExceptions. - Throwable error = e; - while (error != null) { - if (error instanceof SystemException) { - res.error = error; - break; - } - error = error.getCause(); - } - // Default - if (res.error == null) { - res.error = e; - } + res.error = e; } finally { try { res.compare(); @@ -208,6 +110,7 @@ public class TestRunner { long end = System.currentTimeMillis(); res.time = end - start; } + if (opts.showResult) { if (res.result == null) { System.err.println("***Error: "); @@ -218,7 +121,55 @@ public class TestRunner { System.err.println(res.result); } } + } + + private static QueryRequest createQueryRequest(XTestOptions opts, String query) { + QueryRequest request = new QueryRequest(query); + request.setCompileOnly(opts.compileOnly); + request.setOptimization(opts.optimizationLevel); + request.setFrameSize(opts.frameSize); + request.setShowAbstractSyntaxTree(opts.showAST); + request.setShowTranslatedExpressionTree(opts.showTET); + request.setShowOptimizedExpressionTree(opts.showOET); + request.setShowRuntimePlan(opts.showRP); + request.setAsync(false); + + return request; + } + + private static APIResponse sendQueryRequest(QueryRequest request, Map<String, File> sourceFileMap) + throws IOException, URISyntaxException { + + URI uri = RestUtils.buildQueryURI(request, TestClusterUtil.localClusterUtil.getIpAddress(), + TestClusterUtil.localClusterUtil.getRestPort()); + CloseableHttpClient httpClient = HttpClients.custom().build(); + + try { + HttpPost httpRequest = new HttpPost(uri); + httpRequest.setHeader(HttpHeaders.ACCEPT, CONTENT_TYPE_JSON); + + ObjectMapper mapper = new ObjectMapper(); + String fileMap = mapper.writeValueAsString(sourceFileMap); + httpRequest.setEntity(new StringEntity(fileMap, StandardCharsets.UTF_8)); + + try (CloseableHttpResponse httpResponse = httpClient.execute(httpRequest)) { + HttpEntity entity = httpResponse.getEntity(); + String response = RestUtils.readEntity(entity); + if (httpResponse.getStatusLine().getStatusCode() == HttpStatus.SC_OK) { + return RestUtils.mapEntity(response, SyncQueryResponse.class, CONTENT_TYPE_JSON); + } else { + return RestUtils.mapEntity(response, ErrorResponse.class, CONTENT_TYPE_JSON); + } + } catch (IOException e) { + System.err.println("Error occurred when reading entity: " + e.getMessage()); + } catch (JAXBException e) { + System.err.println("Error occurred when mapping query response: " + e.getMessage()); + } + } finally { + HttpClientUtils.closeQuietly(httpClient); + } + return null; } public void runQueries(TestCase testCase, TestCaseResult res) { http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/XTest.java ---------------------------------------------------------------------- diff --git a/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/XTest.java b/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/XTest.java index 5aae691..df7a71d 100644 --- a/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/XTest.java +++ b/vxquery-xtest/src/main/java/org/apache/vxquery/xtest/XTest.java @@ -16,10 +16,6 @@ */ package org.apache.vxquery.xtest; -import org.apache.hyracks.control.cc.ClusterControllerService; -import org.apache.hyracks.control.nc.NodeControllerService; -import org.mortbay.jetty.Server; - import java.io.File; import java.io.IOException; import java.util.ArrayList; @@ -28,6 +24,8 @@ import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.TimeUnit; +import org.mortbay.jetty.Server; + public class XTest { private XTestOptions opts; private Server server; @@ -36,8 +34,6 @@ public class XTest { private TestRunnerFactory trf; private int count; private int finishCount; - private static NodeControllerService nc; - private static ClusterControllerService cc; XTest(XTestOptions opts) { this.opts = opts; @@ -81,8 +77,7 @@ public class XTest { } } }); - cc = TestClusterUtil.startCC(opts); - nc = TestClusterUtil.startNC(); + TestClusterUtil.startCluster(opts, TestClusterUtil.localClusterUtil); trf = new TestRunnerFactory(opts); trf.registerReporters(reporters); TestCaseFactory tcf = new TestCaseFactory(trf, eSvc, opts); @@ -104,7 +99,7 @@ public class XTest { r.close(); } try { - TestClusterUtil.stopCluster(cc, nc); + TestClusterUtil.stopCluster(TestClusterUtil.localClusterUtil); } catch (IOException e) { e.printStackTrace(); } http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-xtest/src/test/java/org/apache/vxquery/xtest/AbstractXQueryTest.java ---------------------------------------------------------------------- diff --git a/vxquery-xtest/src/test/java/org/apache/vxquery/xtest/AbstractXQueryTest.java b/vxquery-xtest/src/test/java/org/apache/vxquery/xtest/AbstractXQueryTest.java index 1e2dcf6..8f77de4 100644 --- a/vxquery-xtest/src/test/java/org/apache/vxquery/xtest/AbstractXQueryTest.java +++ b/vxquery-xtest/src/test/java/org/apache/vxquery/xtest/AbstractXQueryTest.java @@ -22,8 +22,6 @@ import java.io.File; import java.io.IOException; import org.apache.commons.io.FileUtils; -import org.apache.hyracks.control.cc.ClusterControllerService; -import org.apache.hyracks.control.nc.NodeControllerService; import org.junit.After; import org.junit.AfterClass; import org.junit.Before; @@ -38,8 +36,6 @@ public abstract class AbstractXQueryTest { private TestRunner tr; private static MiniDFS dfs; private final static String TMP = "target/tmp"; - private static NodeControllerService nc; - private static ClusterControllerService cc; protected abstract XTestOptions getTestOptions(); @@ -92,8 +88,7 @@ public abstract class AbstractXQueryTest { @BeforeClass public static void setup() throws IOException { - cc = TestClusterUtil.startCC(getDefaultTestOptions()); - nc = TestClusterUtil.startNC(); + TestClusterUtil.startCluster(getDefaultTestOptions(), TestClusterUtil.localClusterUtil); setupFS(); } @@ -116,7 +111,7 @@ public abstract class AbstractXQueryTest { @AfterClass public static void shutdown() throws IOException { removeFS(); - TestClusterUtil.stopCluster(cc, nc); + TestClusterUtil.stopCluster(TestClusterUtil.localClusterUtil); } public static void removeFS() throws IOException { http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-1.txt ---------------------------------------------------------------------- diff --git a/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-1.txt b/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-1.txt new file mode 100644 index 0000000..b1db973 --- /dev/null +++ b/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-1.txt @@ -0,0 +1,3 @@ +<value>33</value> +<value>32</value> +<value>31</value> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-2.txt ---------------------------------------------------------------------- diff --git a/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-2.txt b/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-2.txt new file mode 100644 index 0000000..d93567e --- /dev/null +++ b/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-2.txt @@ -0,0 +1,3 @@ +<value>33</value> +<value>31</value> +<value>32</value> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-3.txt ---------------------------------------------------------------------- diff --git a/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-3.txt b/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-3.txt new file mode 100644 index 0000000..2ab8764 --- /dev/null +++ b/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-3.txt @@ -0,0 +1,3 @@ +<value>32</value> +<value>33</value> +<value>31</value> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/vxquery/blob/f2e5fd90/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-4.txt ---------------------------------------------------------------------- diff --git a/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-4.txt b/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-4.txt new file mode 100644 index 0000000..d1d6bb7 --- /dev/null +++ b/vxquery-xtest/src/test/resources/ExpectedTestResults/GhcndRecords/Partition-2/q03_records-4.txt @@ -0,0 +1,3 @@ +<value>32</value> +<value>31</value> +<value>33</value> \ No newline at end of file
