sundapeng opened a new pull request, #10991:
URL: https://github.com/apache/paimon/pull/10991
### Purpose
`FileIndexerFactoryUtils` and `GlobalIndexerFactoryUtils` discover their
factories once, in a static initializer, with `ServiceLoader.load(Class)`, i.e.
with the context classloader of whichever thread touches the class first. If
that classloader cannot see Paimon's service files, the registry stays empty
for the life of the classloader and every later lookup fails.
Since #10850 moved schema validation into the first `FileStoreTable#store()`
call, a Flink job hits this when its `OperatorCoordinator` builds a
`TableWriteCoordinator` in `start()`. That runs on the JobMaster RPC thread,
whose context classloader is Flink's own, so the JobMaster fails to start for
any table that declares a file index:
```
Caused by: java.lang.RuntimeException: Can't find file index for type:
bloom-filter
at
org.apache.paimon.fileindex.FileIndexerFactoryUtils.load(FileIndexerFactoryUtils.java:56)
at
org.apache.paimon.schema.SchemaValidation.validateFileIndex(SchemaValidation.java:1164)
at
org.apache.paimon.schema.SchemaValidation.validateTableSchema(SchemaValidation.java:409)
at
org.apache.paimon.table.AbstractFileStoreTable.runtimeStore(AbstractFileStoreTable.java:362)
at
org.apache.paimon.table.PrimaryKeyFileStoreTable.store(PrimaryKeyFileStoreTable.java:78)
at
org.apache.paimon.flink.sink.coordinator.TableWriteCoordinator.<init>(TableWriteCoordinator.java:71)
...
at
org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.start(OperatorCoordinatorHolder.java:204)
```
Before #10850 the coordinator path never touched these registries, so the
same jobs started fine.
This PR discovers the factories with the interfaces' own classloader, as
`FileIO#discoverLoaders`, `FileFormat#fromIdentifier` and the field aggregator
factories already do.
### Tests
- `IndexerFactoryUtilsClassLoaderTest` initializes each utils class in a
fresh classloader while the context classloader sees nothing, then loads
`bloom-filter` and `btree`. Both cases fail without the change with the errors
above, and pass with it.
- `org.apache.paimon.fileindex.**` and `org.apache.paimon.globalindex.**`
tests in paimon-common pass (666 tests).
- Also reproduced against a shaded Flink job jar: deserializing a
primary-key table with `file-index.bloom-filter.columns` in a new classloader
and calling `store()` under a context classloader without Paimon fails before
the change and succeeds after it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]