gavinchou commented on code in PR #59754:
URL: https://github.com/apache/doris/pull/59754#discussion_r2705542656
##########
be/src/cloud/cloud_tablet.h:
##########
@@ -463,6 +475,23 @@ class CloudTablet final : public BaseTablet {
mutable std::shared_mutex _warmed_up_rowsets_mutex;
std::unordered_set<RowsetId> _warmed_up_rowsets;
+
+ // Map: version -> <rowset_meta, expiration_time>
+ // Stores rowsets that have been notified by FE but not yet added to
tablet meta
+ // due to out-of-order notification or version discontinuity
+ struct VisiblePendingRowset {
+ const bool is_empty_rowset;
+ const int64_t expiration_time; // seconds since epoch
+ RowsetMetaSharedPtr rowset_meta;
+
+ VisiblePendingRowset(RowsetMetaSharedPtr rowset_meta_, int64_t
expiration_time_,
+ bool is_empty_rowset_ = false)
+ : is_empty_rowset(is_empty_rowset_),
+ expiration_time(expiration_time_),
+ rowset_meta(std::move(rowset_meta_)) {}
+ };
+ mutable std::mutex _visible_pending_rs_lock;
+ std::map<int64_t, VisiblePendingRowset> _visible_pending_rs_map;
Review Comment:
consider using a global LRU cache to store the value `VisiblePendingRowset`
to get the total memory consumption of pending rowsets under control
, with which the memory is not related to number of total tabelts, and the
consumption can be limited by LRU itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
