Hello Anon. E. Moose #1000171, Jenkins, Dmitry Lychagin,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/2216
to look at the new patch set (#9).
Change subject: [ASTERIXDB-2194][COMP] Introduce datasource functions
......................................................................
[ASTERIXDB-2194][COMP] Introduce datasource functions
- user model changes: yes
Some functions can be datasources
- storage format changes: no
- interface changes: yes
- Add IDatasourceFunction: A function that is also a datasource
- Add IFunctionToDataSourceTransformer: transform an unnest
function into a datascan during compilation
Details:
- Currently, functions are location agnostic and are run on
parameters that are either passed through them during compile
time or runtime.
- An exception to this is the dataset function which has
an associated location constraints running on the nodes
which host the dataset.
- In this change, we introduce a general framework that allows
creation of new functions similar to the dataset function.
- Such functions are called datasource Functions.
- A datasource function takes constant parameters and run on
a set of partitions similar to the dataset function.
- The first example of such functions is the DatasetResources
function.
- The DatasetResources function takes two parameters, a dataverse
and a dataset. It is then run on all nodes and returns a set
of dataset resources.
- Test cases are added for this function.
Change-Id: Ibcf807ac713a21e8f4d59868525467386e801303
---
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/AnalysisUtil.java
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/MetaFunctionToMetaVariableRule.java
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/UnnestToDataScanRule.java
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroducePrimaryIndexForAggregationRule.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DatasetResourcesDatasource.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DatasetResourcesFunction.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DatasetResourcesReader.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DatasetResourcesRewriter.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DatasetRewriter.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/FeedRewriter.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/FunctionReader.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/FunctionRewriter.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/StorageComponentsDatasource.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/StorageComponentsFunction.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/StorageComponentsReader.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/StorageComponentsRewriter.java
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
A
asterixdb/asterix-app/src/main/java/org/apache/asterix/util/MetadataBuiltinFunctions.java
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.3.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.4.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/dataset-resources/dataset-resources.5.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset-resources/dataset-resources.3.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset-resources/dataset-resources.4.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/dataset-resources/dataset-resources.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedConstants.java
A
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IDatasourceFunction.java
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBuiltinEntities.java
A
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AbstractDatasourceFunction.java
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/DataSource.java
A
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/FunctionDataSource.java
A
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/FunctionDataSourceFactory.java
D
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/MetadataBuiltinFunctions.java
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
A
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/IFunctionToDataSourceRewriter.java
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/utils/RecordUtil.java
42 files changed, 1,574 insertions(+), 468 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/16/2216/9
--
To view, visit https://asterix-gerrit.ics.uci.edu/2216
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibcf807ac713a21e8f4d59868525467386e801303
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi <[email protected]>
Gerrit-Reviewer: Ali Alsuliman <[email protected]>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <[email protected]>
Gerrit-Reviewer: Jenkins <[email protected]>
Gerrit-Reviewer: Till Westmann <[email protected]>
Gerrit-Reviewer: abdullah alamoudi <[email protected]>
