srinipunuru commented on a change in pull request #1153: SAMZA-2320: Samza-sql:
Refactor validation to cover more cases and make it more extensible.
URL: https://github.com/apache/samza/pull/1153#discussion_r324306219
##########
File path:
samza-sql/src/main/java/org/apache/samza/sql/planner/SamzaSqlValidator.java
##########
@@ -121,6 +136,47 @@ protected void validateOutputRecords(RelRecordType
outputRecord, SqlSchema outpu
Map<String, RelDataType> projectRecordMap =
projectRecord.getFieldList().stream().collect(
Collectors.toMap(RelDataTypeField::getName,
RelDataTypeField::getType));
+ // Ensure that all fields from sql statement exist in the output schema
and are of the same type.
+ for (Map.Entry<String, RelDataType> entry : projectRecordMap.entrySet()) {
+ String projectedFieldName = entry.getKey();
+ RelDataType outputFieldType = outputRecordMap.get(projectedFieldName);
+ SqlFieldSchema outputSqlFieldSchema =
outputFieldSchemaMap.get(projectedFieldName);
+
+ if (outputFieldType == null) {
+ // If the field names are specified more than once in the select
query, calcite appends 'n' as suffix to the
+ // dup fields based on the order they are specified, where 'n' starts
from 0 for the first dup field.
+ // Take the following example: SELECT id as str, secondaryId as str,
tertiaryId as str FROM store.myTable
+ // Calcite renames the projected fieldNames in select query as str,
str0, str1 respectively.
+ // Samza Sql allows a field name to be specified up to 2 times. Do the
validation accordingly.
+ if (projectedFieldName.endsWith("0")) {
Review comment:
Can we check whether there is another field in the fieldList without "0" to
ensure that this is really a field that calcite appended "0" to and not a user
specified field?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
