This is an automated email from the ASF dual-hosted git repository.
karan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 03a092f4ece Document Nuances in SegmentMetadataCache Behaviour (#15367)
03a092f4ece is described below
commit 03a092f4ecedf1950321b12046c7f35fa52d91a9
Author: Rishabh Singh <[email protected]>
AuthorDate: Wed Nov 15 18:41:20 2023 +0530
Document Nuances in SegmentMetadataCache Behaviour (#15367)
* Document segment metadata cache behaviour
* Fix typo
* Minor update
* Minor change
---
.../segment/metadata/AbstractSegmentMetadataCache.java | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git
a/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
b/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
index bcc08f54274..512f247c4fc 100644
---
a/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
+++
b/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
@@ -90,9 +90,18 @@ import java.util.stream.StreamSupport;
* An abstract class that listens for segment change events and caches segment
metadata. It periodically refreshes
* the segments, by fetching their metadata which includes schema information
from sources like
* data nodes, tasks, metadata database and builds table schema.
- *
- * <p>This class has an abstract method {@link #refresh(Set, Set)} which the
child class must override
- * with the logic to build and cache table schema.</p>
+ * <p>
+ * At startup, the cache awaits the initialization of the timeline.
+ * If the cache employs a segment metadata query to retrieve segment schema,
it attempts to refresh a maximum
+ * of {@code MAX_SEGMENTS_PER_QUERY} segments for each datasource in each
refresh cycle.
+ * Once all datasources have undergone this process, the initial schema of
each datasource is constructed,
+ * and the cache is marked as initialized.
+ * Subsequently, the cache continues to periodically refresh segments and
update the datasource schema.
+ * It is also important to note that a failure in segment refresh results in
pausing the refresh work,
+ * and the process is resumed in the next refresh cycle.
+ * <p>
+ * This class has an abstract method {@link #refresh(Set, Set)} which the
child class must override
+ * with the logic to build and cache table schema.
*
* @param <T> The type of information associated with the data source, which
must extend {@link DataSourceInformation}.
*/
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
