MeihanLi opened a new issue, #9281: URL: https://github.com/apache/pinot/issues/9281
Hi, we just noticed there is change introduced by [#7374](https://github.com/apache/pinot/pull/7374). A new checking schema.isBackwardCompatibleWith(oldSchema) is added for POST /schemas API. With the checking, we can not modify any existing column maxLength any more. This is a breaking change for us. The previous behavior: If we set override to true, we can just update the existing schema with the new one. No extra checking. ``` /** * API 2.0 */ /** * Schema APIs */ public void addSchema(Schema schema, boolean override) { ZNRecord record = SchemaUtils.toZNRecord(schema); String schemaName = schema.getSchemaName(); Schema oldSchema = ZKMetadataProvider.getSchema(_propertyStore, schemaName); if (oldSchema != null && !override) { throw new RuntimeException(String.format("Schema %s exists. Not overriding it as requested.", schemaName)); } if (schema.equals(oldSchema)) { LOGGER.info("New schema is the same with the existing schema. Not updating schema " + schemaName); return; } PinotHelixPropertyStoreZnRecordProvider propertyStoreHelper = PinotHelixPropertyStoreZnRecordProvider.forSchema(_propertyStore); propertyStoreHelper.set(schemaName, record); } ``` The current behavior is that even we set override to true, we need to check if the new schema is backward compatible with the old one in isBackwardCompatibleWith function (Schema class). If not, we fail the request with SchemaBackwardIncompatibleException Exception. ``` /** * Check whether the current schema is backward compatible with oldSchema. * Backward compatibility requires all columns and fieldSpec in oldSchema should be retained. * * @param oldSchema old schema */ public boolean isBackwardCompatibleWith(Schema oldSchema) { Set<String> columnNames = getColumnNames(); for (Map.Entry<String, FieldSpec> entry : oldSchema.getFieldSpecMap().entrySet()) { String oldSchemaColumnName = entry.getKey(); if (!columnNames.contains(oldSchemaColumnName)) { return false; } FieldSpec oldSchemaFieldSpec = entry.getValue(); FieldSpec fieldSpec = getFieldSpecFor(oldSchemaColumnName); if (!fieldSpec.equals(oldSchemaFieldSpec)) { return false; } } return true; } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
