>From Calvin Thomas Dani <[email protected]>: Calvin Thomas Dani has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19044 )
Change subject: [NO ISSUE] Added schema inference for row based records. ...................................................................... [NO ISSUE] Added schema inference for row based records. - user mode changes: new built-in function - storage format changes: no - interface changes: no Details: -Provides an aggregate funciton "array_schema" to infer schema from records -Cloned schema inference structure from Columnar module to runtime -Final output of the schema is json_schema -Test cases for testing query to its result is also added to Maven Change-Id: I5550c89d56994996b5021f1cbf0e4f03f8fee45b --- A asterixdb/asterix-app/src/test/resources/runtimets/results/schema-inference/schema-inference-04/schema_inference.3.adm A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/AbstractRowSchemaNode.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/utils/RunRowLengthIntArray.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-07/schema_inference-07.1.ddl.sqlpp M asterixdb/asterix-om/pom.xml A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-06/schema_inference-06.2.update.sqlpp M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/lossless/AOptionalFieldPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/csv/AOptionalFieldPrinterFactory.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractSchemaClosedAggregateFunction.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-03/schema_inference-03.3.query.sqlpp A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/utils/JSONRowSchemaStringBuilderVisitor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/SqlSchemaClosedAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/Serialization/fieldNameSerialization.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/lazy/IObjectRowSchemaNodeVisitor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/UnionRowSchemaNode.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionSchemaInference.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-07/schema_inference-07.2.update.sqlpp A asterixdb/asterix-app/data/nontagged/diffFieldTypesData_5.json A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/Serialization/mapSerialization.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/SchemaAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/SqlSchemaAggregateDescriptor.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-03/schema_inference-03.1.ddl.sqlpp A asterixdb/asterix-app/data/nontagged/diffFieldTypesData_6.json A asterixdb/asterix-app/data/nontagged/diffFieldTypesData_3.json A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/primitive/MissingRowFieldSchemaNode.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/collection/ArrayRowSchemaNode.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-06/schema_inference-06.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/schema-inference/schema-inference-03/schema_inference.3.adm A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/utils/UnsafeUtil.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalSqlSchemaAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/SchemaAggregateFunction.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-02/schema_inference-02.2.update.sqlpp A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/api/IRowMetadata.java M README.md M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalSqlAvgAggregateDescriptor.java M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/clause/Projection.java M asterixdb/asterix-app/src/test/resources/cc.conf A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractSchemaAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalSchemaAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/IRowSchemaNodeVisitor.java A asterixdb/asterix-app/src/test/resources/runtimets/results/schema-inference/schema-inference-07/schema_inference.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-05/schema_inference-05.3.query.sqlpp A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/utils/RowValuesUtil.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalSchemaAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/collection/AbstractRowCollectionSchemaNode.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalSchemaAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/Serialization/itemNodeSerialization.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-01/schema_inference-01.2.update.sqlpp A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlSchemaAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/RowSchemaTransformer.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/collection/GenericListRowSchemaNode.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-03/schema_inference-03.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-02/schema_inference-02.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/schema-inference/schema-inference-06/schema_inference.3.adm M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/clean/AOptionalFieldPrinterFactory.java A asterixdb/asterix-app/src/test/resources/runtimets/results/schema-inference/schema-inference-02/schema_inference.3.adm A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalSqlSchemaAggregateFunction.java A asterixdb/asterix-app/data/nontagged/diffFieldTypesData.json M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/adm/AOptionalFieldPrinterFactory.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/lazy/metadata/RowFieldNamesDictionary.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/SqlSchemaClosedAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/SqlSchemaAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/RowTransformer.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateSchemaAggregateFunction.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-07/schema_inference-07.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-02/schema_inference-02.3.query.sqlpp M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java A asterixdb/asterix-app/data/nontagged/diffFieldTypesData_2.json A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/api/IRowWriteMultiPageOp.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalSqlSchemaAggregateFunction.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-05/schema_inference-05.1.ddl.sqlpp A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalSqlSchemaAggregateDescriptor.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-01/schema_inference-01.3.query.sqlpp M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/functions/FunctionCollection.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-01/schema_inference-01.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/schema-inference/schema-inference-05/schema_inference.3.adm A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/lazy/metadata/AbstractRowMetadata.java A asterixdb/asterix-app/src/test/resources/runtimets/results/schema-inference/schema-inference-01/schema_inference.3.adm A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/utils/RowSchemaStringBuilderVisitor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateSchemaAggregateDescriptor.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-04/schema_inference-04.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-06/schema_inference-06.3.query.sqlpp A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/ObjectRowSchemaNode.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/collection/MultisetRowSchemaNode.java A asterixdb/asterix-app/data/nontagged/diffFieldTypesData_4.json A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/primitive/PrimitiveRowSchemaNode.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/AbstractRowSchemaNestedNode.java A asterixdb/asterix-app/src/test/resources/runtimets/testsuite_schemaInferrence_sqlpp.xml A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/schemainferrence/RowMetadata.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-04/schema_inference-04.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-05/schema_inference-05.2.update.sqlpp A asterixdb/asterix-app/data/nontagged/diffFieldTypesData_7.json A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/schema-inference/schema-inference-04/schema_inference-04.1.ddl.sqlpp A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalSchemaAggregateDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/LocalFileSystemUtils.java 96 files changed, 5,642 insertions(+), 9 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/44/19044/1 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19044 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I5550c89d56994996b5021f1cbf0e4f03f8fee45b Gerrit-Change-Number: 19044 Gerrit-PatchSet: 1 Gerrit-Owner: Calvin Thomas Dani <[email protected]> Gerrit-MessageType: newchange
