[
https://issues.apache.org/jira/browse/CALCITE-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17723250#comment-17723250
]
Abhishek Singh Chouhan commented on CALCITE-5690:
-------------------------------------------------
> Does {{RelDataTypeHolder}} claim to be thread-safe? Is its implementation
>actually thread-safe? Does the code using it need to be thread-safe, and if
>so, did it make a mistake using a non-thread-safe class?
RelDataTypeHolder is not thread safe. None of the associated documentation
suggests/claims thread-safety as well.
I believe RelDataTypeFactoryImpl assumes that the type is immutable
([here|[https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/type/RelDataTypeFactoryImpl.java#L402]]
and
[here|[https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/type/RelDataTypeFactoryImpl.java#L421]).]
The datatype cache is interning the type assuming immutable objects, however
the DynamicRecordTypeImpl is mutable. Maybe we should skip interning
DynamicRecordTypeImpl?
> ConcurrentModificationException during validation of table with
> DynamicRecordType
> ---------------------------------------------------------------------------------
>
> Key: CALCITE-5690
> URL: https://issues.apache.org/jira/browse/CALCITE-5690
> Project: Calcite
> Issue Type: Bug
> Affects Versions: 1.32.0
> Reporter: Abhishek Singh Chouhan
> Priority: Major
>
> When multiple threads are doing validation on a Table with
> DynamicRecordTypeImpl, we run into a ConcurrentModificationException. One of
> the instances where this happens is when two validations are happening in
> parallel
> Thread1 -
> SELECT DS_GET_QUANTILE(DS_QUANTILES_SKETCH(COL1), 0.9) FROM TABLE1 GROUP BY
> COL1, COL2, COL3
> Thread2 - SELECT DS_GET_QUANTILE(DS_QUANTILES_SKETCH(COL1), 0.9) FROM TABLE1
> GROUP BY COL1, COL2
> While expanding the groupby in thread1 we intern the type at multiple places,
> eg. DelegatingScope#fullyQualify -> SqlValidatorScope#rowType ->
> RelDataTypeFactoryImpl#createTypeWithNullability -> canonize
> During the validation in thread1 we cache DynamicRecordTypeImpl with Col1 and
> Col2 as the fields.
> The race condition happens when thread2 gets a hold of this(abovementioned)
> cached DynamicRecordTypeImpl during its validation and is iterating over the
> fields eg during SqlValidatorImpl.validateGroupItem and thread 1 then goes
> and inserts Col3 field in the holder.
> {code:java}
> java.util.ConcurrentModificationException
> at
> java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
> at
> org.apache.calcite.rel.type.RelDataTypeHolder.getFieldOrInsert(RelDataTypeHolder.java:56)
> at
> org.apache.calcite.rel.type.DynamicRecordTypeImpl.getField(DynamicRecordTypeImpl.java:59)
> at
> org.apache.calcite.sql.validate.SqlNameMatchers$BaseMatcher.field(SqlNameMatchers.java:126)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6455)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6360)
> at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:324)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1869)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1854)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupItem(SqlValidatorImpl.java:4393)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupClause(SqlValidatorImpl.java:4366)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3701)
> at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)
> at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1107)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1078)
> at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:248)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1053)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:759)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
