Hi SeungMinLee,
First of all, thank you for putting together FIP-30. The ability
Tracking tiering status is a much-needed feature, and I appreciate the
thorough
design work that went into this proposal.
After reviewing the FIP, I have a few thoughts and questions I'd like to
raise
for discussion. These are suggestions based on my understanding - I may
be
missing context, so please feel free to correct me if any of these points
have
already been considered.
1. Per-Table System Table vs Global System Table
The proposal introduces both:
- Global view: `fluss_catalog.sys.lake_tiering_status`
- Per-table view: `my_db.my_table$tiering_status`
I was wondering if we could simplify the initial implementation by
focusing on
the per-table `$tiering_status` virtual table for SQL access, while
relying on
The `listTieringStatuses()` Admin API for bulk/system-wide queries.
My reasoning:
- Consistency: The per-table pattern (`$tiering_status`) aligns with
Fluss's
existing virtual table conventions and is similar to the virtual table
approach with
`$changelog`, `$binlog`, etc.
- Scalability: A global SQL table querying thousands of tables could have
performance implications. The Admin API seems better suited for bulk
operations
with potential pagination support.
A phased approach (Phase 1: per-table SQL, Phase 2: Admin API) could ship
value to users faster with reduced initial scope.
That said, I may be underestimating the need for the global SQL table. Are
there specific use cases that would be difficult to serve with just the
Admin API?
2. Error Message Truncation Strategy
The proposal mentions truncating error messages to 2-4KB before sending
to the
Coordinator. I have a concern about simple head truncation potentially
removing
the most useful diagnostic information.
Are we considering an extraction strategy to deal with it, in my mind,
something like this?
- Smart extraction: Parse and extract all "Caused by:" lines, which
typically
contain the most actionable information
I understand this adds complexity, so it's a trade-off. Curious to hear
others'
thoughts on whether this is worth addressing.
3. Consolidating State Maps in LakeTableTieringManager
The proposal adds `tieringFailMessages` and `tieringFailTimes` maps to
`LakeTableTieringManager`. Looking at the current implementation, the
manager
already maintains 6+ separate maps keyed by `tableId`:
```java
Map<Long, TieringState> tieringStates;
Map<Long, TablePath> tablePaths;
Map<Long, Long> tableLakeFreshness;
Map<Long, Long> tableTierEpoch;
Map<Long, Long> tableLastTieredTime;
Map<Long, Long> liveTieringTableIds;
// Proposed additions:
Map<Long, String> tieringFailMessages;
Map<Long, Long> tieringFailTimes;
One thought: would it be cleaner to consolidate these into a single
TableTieringInfo object?
Map<Long, TableTieringInfo> tableInfos;
class TableTieringInfo {
TablePath tablePath;
long lakeFreshness;
TieringState state;
long tieringEpoch;
long lastTieredTime;
@Nullable String lastError;
@Nullable Long lastErrorTime;
}
Potential benefits:
- Single map lookup instead of multiple
- Related state updated together naturally
- Cleaner cleanup in removeLakeTable() (one removal vs. 8)
This could be a separate preparatory refactoring PR or part of FIP-30.
However,
I understand this might be out of scope for this FIP, and I don't want to
expand
the scope unnecessarily. Just raising it as a thought for the authors to
consider.
These are just suggestions based on my reading of the proposal. I'm happy
to be
corrected if I've misunderstood anything. Also happy to help with
implementation or further discussion if useful.
Thanks again for driving this important feature!
Best regards,
Mehul Batra
On Thu, Feb 12, 2026 at 5:53 PM SeungMin Lee <[email protected]> wrote:
> Hi dev,
>
> Just a quick update.
>
> I have migrated the design google docs to the cwiki and registered it as
> *FIP-30*. Please refer to the link below for the formal proposal:
>
>
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-30%3A+Support+tracking+the+tiering+status+of+a+tiering+table
>
> The content remains consistent with the previous Google Doc.
>
> Best regards,
> SeungMin Lee
>
> 2026년 2월 12일 (목) PM 5:37, SeungMin Lee <[email protected]>님이 작성:
> >
> > Hi, dev
> >
> > Currently, there is no way for users to check the status of lake tiering.
> Users cannot be aware if tiering fails, and they have to manually parse the
> Tiering Service logs to identify the cause.
> >
> > So, I'd like to propose Issue-2362: Allow users to track the tiering
> status of a tiering table to address this visibility issue.
> >
> > I have drafted a design docs [2]. Please feel free to review and share
> your feed.
> >
> > Considering the upcoming holidays in some regions, I'll wait for feedback
> and give a ping on this thread around Feb 23rd.
> >
> > Looking forward to your thoughts.
> >
> > Best regards,
> > SeungMin Lee
> >
> > [1] https://github.com/apache/fluss/issues/2362
> > [2]
>
> https://docs.google.com/document/d/1eJbRCwzAbeJLA97zQQ0I3JM1jerBXXhq69Dn8r4xWV0/edit?usp=sharing
>
