>From Wail Alkowaileet <[email protected]>:
Hello Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17751
to look at the new patch set (#3).
Change subject: [ASTERIXDB-3255][EXT] Part1: Embed external filter values
......................................................................
[ASTERIXDB-3255][EXT] Part1: Embed external filter values
- user model changes: yes
- storage format changes: no
- interface changes: yes
Details:
- Embed filter values by default
-- Users can disable that by setting 'embed-filter-values'
to false when creating an external dataset.
-- Value embedding is performed at the parser level
- This patch includes the support of value embedding when
querying JSON-based external datasets.
- The support for Parquet, CSV, and TSV will be included
in follow-up patches.
Change-Id: Ia61a62ab1390d9d87105898cf8c17f77c136c434
---
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.030.query.sqlpp
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/pushdown/visitor/ColumnFilterPathBuilderVisitor.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/gcs/GCSReaderFactory.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.010.query.sqlpp
A
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/external/IExternalFilterEmbeddedValueInformation.java
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.020.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.011.query.sqlpp
M
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/projection/ExternalDatasetProjectionFiltrationInfo.java
A
asterixdb/asterix-app/data/json/external-filter/embed/department/accounting/0.json
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/query/test.000.ddl.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IDataParser.java
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/type-mismatch/test.000.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.021.plan
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.121.plan
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/google/gcs/GCSUtils.java
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/IndexUtil.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/azure/parquet/AzureBlobParquetReaderFactory.java
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.010.adm
A asterixdb/asterix-app/data/json/external-filter/embed/department/hr/0.json
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.110.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.131.query.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/gcs/GCSInputStreamFactory.java
A
asterixdb/asterix-app/data/json/external-filter/embed/department/engineering/0.json
A asterixdb/asterix-app/data/json/external-filter/embed/last-name/smith/0.json
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.120.adm
M
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/projection/ColumnDatasetProjectionFiltrationInfo.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.130.query.sqlpp
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/LoadableDataSource.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/ExternalFilterEvaluatorFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/embedder/IExternalFilterValueEmbedder.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/parquet/AwsS3ParquetReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/AwsS3InputStreamFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/azure/blob_storage/AzureUtils.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/StreamRecordReaderFactory.java
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/utils/ProjectionFiltrationTypeUtil.java
R
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/NoOpExternalFilterEvaluatorFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/embedder/EmbeddedValueBuilder.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DataflowControllerProvider.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.020.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.031.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.041.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.131.plan
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.040.query.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/abstracts/AbstractExternalInputStream.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.021.query.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordDataParser.java
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.041.plan
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/abstracts/AbstractExternalInputStreamFactory.java
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.011.plan
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.111.plan
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/azure/datalake/AzureDataLakeReaderFactory.java
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.130.adm
A asterixdb/asterix-app/data/json/external-filter/embed/last-name/miller/0.json
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataPrefix.java
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.040.adm
M
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/external/IExternalFilterEvaluatorFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/ExternalFilterValueEvaluator.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.120.query.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/azure/blob/AzureBlobInputStreamFactory.java
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/pushdown/PushdownProcessorsExecutor.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.111.query.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AbstractJsonDataParser.java
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.121.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/embed-one-value/embed-one-value.000.ddl.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/azure/blob/AzureBlobReaderFactory.java
M
asterixdb/asterix-app/src/test/resources/runtimets/testsuite_external_dataset_s3.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IExternalDataSourceFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IRecordReader.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/AwsS3ReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/aws/s3/S3Utils.java
M
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/external_dataset/PrefixComputedFieldsTest.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/AsterixInputStream.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/FeedIntakeOperatorDescriptor.java
R
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/NoOpExternalFilterEvaluator.java
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/common/dynamic-prefixes/one-field/one-field.000.ddl.sqlpp
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/filter/ExternalFilterBuilder.java
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.030.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.110.adm
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/embedder/ExternalFilterValueEmbedder.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/JSONDataParser.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/filter/NoOpFilterValueEmbedder.java
A asterixdb/asterix-app/data/json/external-filter/embed/last-name/Jones/0.json
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/ExternalAdapterFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/gcs/parquet/GCSParquetReaderFactory.java
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-dataset/common/dynamic-prefixes/embed-one-value/one-field.031.plan
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/StreamRecordReader.java
90 files changed, 1,521 insertions(+), 318 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/51/17751/3
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17751
To unsubscribe, or for help writing mail filters, visit
https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Ia61a62ab1390d9d87105898cf8c17f77c136c434
Gerrit-Change-Number: 17751
Gerrit-PatchSet: 3
Gerrit-Owner: Wail Alkowaileet <[email protected]>
Gerrit-Reviewer: Jenkins <[email protected]>
Gerrit-Reviewer: Wail Alkowaileet <[email protected]>
Gerrit-MessageType: newpatchset
