zuotingbing opened a new issue, #9187:
URL: https://github.com/apache/gravitino/issues/9187
### Version
main branch
### Describe what's wrong
**error1:**
### SQL: DELETE FROM policy_meta WHERE policy_id IN (SELECT policy_id FROM
policy_meta WHERE deleted_at = 0 AND legacy_timeline < ? LIMIT ?)
### Cause: org.postgresql.util.PSQLException: ERROR: column
"legacy_timeline" does not exist
**error2:**
### SQL: DELETE FROM statistic_meta WHERE deleted_at > 0 AND deleted_at < ?
LIMIT ?
### Cause: org.postgresql.util.PSQLException: ERROR: syntax error at or near
"LIMIT"
### Error message and/or stacktrace
2025-11-19 22:53:41.735 ERROR [RelationalBackend-Garbage-Collector]
[org.apache.gravitino.storage.relational.RelationalGarbageCollector.collectAndClean(RelationalGarbageCollector.java:90)]
- Failed to physically delete type of POLICY's legacy data:
org.apache.ibatis.exceptions.PersistenceException:
### Error updating database. Cause: org.postgresql.util.PSQLException:
ERROR: column "legacy_timeline" does not exist
Position: 104
### The error may exist in
org/apache/gravitino/storage/relational/mapper/PolicyMetaMapper.java (best
guess)
### The error may involve
org.apache.gravitino.storage.relational.mapper.PolicyMetaMapper.deletePolicyMetasByLegacyTimeline-Inline
### The error occurred while setting parameters
### SQL: DELETE FROM policy_meta WHERE policy_id IN (SELECT policy_id FROM
policy_meta WHERE deleted_at = 0 AND legacy_timeline < ? LIMIT ?)
### Cause: org.postgresql.util.PSQLException: ERROR: column
"legacy_timeline" does not exist
Position: 104
at
org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:199)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:212)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:72)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:152)
~[mybatis-3.5.6.jar:3.5.6]
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:85)
~[mybatis-3.5.6.jar:3.5.6]
at
jdk.proxy2/jdk.proxy2.$Proxy104.deletePolicyMetasByLegacyTimeline(Unknown
Source) ~[?:?]
at
org.apache.gravitino.storage.relational.service.PolicyMetaService.lambda$deletePolicyAndVersionMetasByLegacyTimeline$27(PolicyMetaService.java:373)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
org.apache.gravitino.storage.relational.utils.SessionUtils.doWithCommitAndFetchResult(SessionUtils.java:56)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
org.apache.gravitino.storage.relational.service.PolicyMetaService.deletePolicyAndVersionMetasByLegacyTimeline(PolicyMetaService.java:371)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
org.apache.gravitino.storage.relational.JDBCBackend.hardDeleteLegacyData(JDBCBackend.java:375)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
org.apache.gravitino.storage.relational.RelationalGarbageCollector.collectAndClean(RelationalGarbageCollector.java:87)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
~[?:?]
at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
~[?:?]
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
~[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
~[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
~[?:?]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: org.postgresql.util.PSQLException: ERROR: column
"legacy_timeline" does not exist
Position: 104
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2713)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2401)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498)
~[postgresql-42.6.0.jar:42.6.0]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:415)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:177)
~[postgresql-42.6.0.jar:42.6.0]
at
org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:95)
~[commons-dbcp2-2.11.0.jar:2.11.0]
at
org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:95)
~[commons-dbcp2-2.11.0.jar:2.11.0]
at
org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:47)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:197)
~[mybatis-3.5.6.jar:3.5.6]
... 16 more
2025-11-19 22:53:41.746 ERROR [RelationalBackend-Garbage-Collector]
[org.apache.gravitino.storage.relational.RelationalGarbageCollector.collectAndClean(RelationalGarbageCollector.java:90)]
- Failed to physically delete type of TABLE_STATISTIC's legacy data:
org.apache.ibatis.exceptions.PersistenceException:
### Error updating database. Cause: org.postgresql.util.PSQLException:
ERROR: syntax error at or near "LIMIT"
Position: 69
### The error may exist in
org/apache/gravitino/storage/relational/mapper/StatisticMetaMapper.java (best
guess)
### The error may involve
org.apache.gravitino.storage.relational.mapper.StatisticMetaMapper.deleteStatisticsByLegacyTimeline-Inline
### The error occurred while setting parameters
### SQL: DELETE FROM statistic_meta WHERE deleted_at > 0 AND deleted_at < ?
LIMIT ?
### Cause: org.postgresql.util.PSQLException: ERROR: syntax error at or near
"LIMIT"
Position: 69
at
org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:199)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:212)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:72)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:152)
~[mybatis-3.5.6.jar:3.5.6]
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:85)
~[mybatis-3.5.6.jar:3.5.6]
at
jdk.proxy2/jdk.proxy2.$Proxy105.deleteStatisticsByLegacyTimeline(Unknown
Source) ~[?:?]
at
org.apache.gravitino.storage.relational.service.StatisticMetaService.lambda$deleteStatisticsByLegacyTimeline$3(StatisticMetaService.java:100)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
org.apache.gravitino.storage.relational.utils.SessionUtils.doWithCommitAndFetchResult(SessionUtils.java:56)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
org.apache.gravitino.storage.relational.service.StatisticMetaService.deleteStatisticsByLegacyTimeline(StatisticMetaService.java:98)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
org.apache.gravitino.storage.relational.JDBCBackend.hardDeleteLegacyData(JDBCBackend.java:390)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
org.apache.gravitino.storage.relational.RelationalGarbageCollector.collectAndClean(RelationalGarbageCollector.java:87)
~[gravitino-core-1.0.0-zdh15.8.1-SNAPSHOT.jar:?]
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
~[?:?]
at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
~[?:?]
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
~[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
~[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
~[?:?]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at or near
"LIMIT"
Position: 69
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2713)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2401)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498)
~[postgresql-42.6.0.jar:42.6.0]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:415)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190)
~[postgresql-42.6.0.jar:42.6.0]
at
org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:177)
~[postgresql-42.6.0.jar:42.6.0]
at
org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:95)
~[commons-dbcp2-2.11.0.jar:2.11.0]
at
org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:95)
~[commons-dbcp2-2.11.0.jar:2.11.0]
at
org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:47)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
~[mybatis-3.5.6.jar:3.5.6]
at
org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:197)
~[mybatis-3.5.6.jar:3.5.6]
... 16 more
2025-11-19 22:53:41.763 INFO [RelationalBackend-Garbage-Collector]
[org.apache.gravitino.storage.relational.RelationalGarbageCollector.collectAndClean(RelationalGarbageCollector.java:94)]
- Start to collect and delete old version data by thread 20
### How to reproduce
1、start the gravition service
2、wait for RelationalGarbageCollector.collectAndClean
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
