JingsongLi commented on code in PR #1401:
URL: https://github.com/apache/paimon-rust/pull/1401#discussion_r4656596376
##########
crates/paimon/src/table/compact/primary_key/planner.rs:
##########
@@ -227,10 +241,71 @@ pub(in crate::table::compact) fn
select_with_force_level_zero(
Ok(Some((selected, output_level)))
}
+/// Incremental run selection used by execute/commit validation.
+///
+/// Matches planning: prefer UniversalCompaction with the same force-L0 policy,
+/// then fall back to an expire-only max-level rewrite when value stats prove
+/// every record expired at `expire_now_secs` (or the current clock when
unset).
+pub(in crate::table::compact) fn select_or_expire(
+ files: Vec<DataFileMeta>,
+ schema: &TableSchema,
+ force_level_zero: bool,
+ expire_now_secs: Option<i64>,
+) -> Result<Option<(Vec<DataFileMeta>, i32)>> {
+ if let Some(selected) =
+ select_with_force_level_zero(files.clone(), schema.options(),
force_level_zero)?
+ {
+ return Ok(Some(selected));
+ }
+ let now =
expire_now_secs.unwrap_or_else(crate::table::record_level_expire::current_epoch_secs);
+ select_expired_max_level(&files, schema, now)
+}
+
+/// When UniversalCompaction picks nothing, rewrite max-level files whose value
+/// stats prove every record is already past `record-level.expire-time`.
+///
+/// Unknown / incomplete stats never force a rewrite. `now_secs` must match the
+/// value frozen on the compaction task so selection does not drift.
+pub(in crate::table::compact) fn select_expired_max_level(
+ files: &[DataFileMeta],
+ schema: &TableSchema,
+ now_secs: i64,
+) -> Result<Option<(Vec<DataFileMeta>, i32)>> {
+ let Some(expire) =
crate::table::record_level_expire::RecordLevelExpire::create(schema)? else {
+ return Ok(None);
+ };
+ let properties = schema.options();
+ let strategy = UniversalCompaction::from_options(properties, "")?;
+ let configured_levels = integer_option(
+ properties,
+ "num-levels",
+ strategy.trigger.saturating_add(1),
+ 2,
+ )?;
+ let levels = configured_levels.max(
+ files
+ .iter()
+ .map(|file| file.level.saturating_add(1))
+ .max()
+ .unwrap_or(2),
+ );
+ let max_level = levels - 1;
+ let expired = files
+ .iter()
+ .filter(|file| file.level == max_level && expire.is_expire_file(file,
schema, now_secs))
+ .cloned()
+ .collect::<Vec<_>>();
Review Comment:
**[P1] Make expire-only input ordering deterministic**
The planner preserves the incoming file order here, but
`CompactTask::validate_inputs` replans from a fresh `HashMap` and compares the
selected vectors in order. With two expired top-level files in one bucket
(`target-file-row-num=1`), the exact same task on an unchanged snapshot was
rejected in 7 of 16 local executions with `PK compaction inputs do not match
incremental run selection`. Canonicalize expire-only inputs or compare their
sets while still validating every file's metadata.
##########
crates/paimon/src/table/compact/primary_key.rs:
##########
@@ -86,6 +92,29 @@ impl PrimaryKeyCompactPlanner {
total_buckets,
files,
output_level: Some(output_level),
+ force_level_zero,
+ expire_now_secs,
+ blob_field_id: None,
+ vector_field_ids: None,
+ });
+ } else if let Some((files, output_level)) =
planner::select_expired_max_level(
+ &bucket_files,
+ &context.schema,
+ expire_now_secs
+
.unwrap_or_else(crate::table::record_level_expire::current_epoch_secs),
+ )? {
+ // Max-level files that UniversalCompaction would skip
still
+ // need a rewrite when value stats prove every record
expired.
+ tasks.push(CompactTask {
+ context: context.clone(),
+ task_id: tasks.len(),
+ partition,
+ bucket,
+ total_buckets,
+ files,
Review Comment:
**[P1] Preserve the key ranges of nonadjacent TTL rewrite inputs**
The minimum timestamp lets selected files contain live rows. I seeded a
single max-level run with `[1,2]`, `[3,4]`, and `[5,6]`, then expired only `1`
and `5`. The outer inputs were selected, the writer combined live keys `2` and
`6` into one file, and COMPACT failed with `LSM conflicts detected: overlapping
or unverifiable key ranges in a positive level` because `[3,4]` was retained.
Once input validation passes, this still blocks expiration. Rewrite each
selected file or contiguous range separately, or include the intervening files.
##########
crates/paimon/src/table/sort_merge.rs:
##########
@@ -1879,6 +1894,37 @@ fn sort_merge_stream(
tree.update(|a, b| compare_cursors(&cursors, a,
b).then_with(|| a.cmp(&b)).is_gt());
}
+ // Filter expired compact-input rows before lookup inject / merge
+ // (Java RecordLevelExpire wraps the compact reader so Lookup never
+ // sees expired L0). Historical LevelLookup rows are filtered by
+ // ExpiringLookup when a surviving L0 key still needs before-state.
+ if let Some((expire, now_secs)) = &input_expire {
+ let mut kept = Vec::with_capacity(same_key_rows.len());
+ for row in same_key_rows.drain(..) {
+ if expire.keeps_merge_row(
+ &batch_buffer,
+ &row,
+ &source_output_col_indices,
+ &output_schema,
+ *now_secs,
+ )? {
+ kept.push(row);
+ }
+ }
+ same_key_rows = kept;
+ }
+
+ // Java reader wrap means a fully-expired key never enters merge,
so
+ // Lookup is not consulted and history is left untouched.
+ if same_key_rows.is_empty() {
+ // Same cleanup as omit/no-output keys: release batches that
are
+ // no longer referenced by cursors or pending output indices.
+ if output_indices.is_empty() {
Review Comment:
**[P1] Release expired input batches while a live row is pending**
I extended the batch-buffer test to keep only the first and last of 64
sorted one-row batches, with output batch size 2. The peak retained buffer was
all 64 input batches. A single pending live row prevents this branch from
reclaiming unrelated consumed inputs until enough later live keys fill the
output, so a long expired range can retain the entire range's decoded data and
exhaust compaction memory. Keep the sources referenced by pending rows while
releasing the other consumed batches.
##########
crates/paimon/src/table/compact/primary_key.rs:
##########
@@ -390,13 +421,42 @@ pub(super) async fn execute(
let lookup_changelog = options.try_changelog_producer()? ==
ChangelogProducer::Lookup;
let output_level = task.output_level();
let drop_delete = options.deletion_vectors_enabled() && output_level > 0;
- let reader = new_reader(table, resources.clone())?.with_lookup(lookup);
+ // Java wraps compact input readers with RecordLevelExpire. For first-row
+ // Lookup, history uses an unwrapped existence-only reader (PersistEmpty /
+ // FirstRowMergeFunctionWrapper), so expired retained history must still be
+ // visible until that key is retired. Other Lookup engines filter history.
+ let expire =
crate::table::record_level_expire::RecordLevelExpire::create(schema)?;
+ let now_secs = task
+ .expire_now_secs()
+ .unwrap_or_else(crate::table::record_level_expire::current_epoch_secs);
+ let first_row = options.merge_engine()? == MergeEngine::FirstRow;
+ let lookup = match (lookup, &expire) {
+ (Some(provider), Some(expire)) if !first_row => {
+ Some(crate::table::record_level_expire::ExpiringLookup::wrap(
+ provider,
+ expire.clone(),
+ now_secs,
+ ))
+ }
+ (provider, _) => provider,
+ };
+ let reader = new_reader(table, resources.clone())?
+ .with_lookup(lookup)
+ .with_input_expire(expire.clone().map(|expire| (expire, now_secs)));
let mut batches =
reader.read_with_merge_function(std::slice::from_ref(split), move
|config, fields| {
+ let engine = new_merge_function(config, fields, output_level)?;
if lookup_changelog {
- lookup::new_merge_function(config, fields, output_level)
+ // Input expire filtering happens before Lookup.add; first-row
+ // history is intentionally not expire-wrapped above.
+ lookup::wrap(engine, config, fields)
} else {
- new_merge_function(config, fields, output_level)
+ Ok(match expire.clone() {
+ Some(expire) =>
crate::table::record_level_expire::wrap_expire_with_now(
Review Comment:
**[P1] Preserve retained first-row history with `changelog-producer=none`**
For first-row the lookup provider deliberately keeps historical hits while
their files are still retained. This second expiration wrapper filters that
injected hit out again for `changelog-producer=none`. With an expired retained
first row and a live L0 row for the same key, I verified an L0-only compact
writes one duplicate, although the query still returns the retained first row
and compaction should write zero. Retiring the original can then expose the
value that first-row should have discarded. Apply expiration to the compact
inputs without re-filtering injected first-row history.
##########
crates/paimon/src/table/record_level_expire.rs:
##########
@@ -0,0 +1,598 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Record-level expiration for primary-key compaction.
+//!
+//! Mirrors Java `org.apache.paimon.io.RecordLevelExpire`: versions whose
+//! configured time field is older than `now - expire` are dropped **before**
+//! merge during compaction. Query reads are unaffected.
+
+use crate::spec::{
+ extract_datum_from_array, parse_duration_millis, DataField, DataFileMeta,
DataType, Datum,
+ TableSchema,
+};
+use crate::table::sort_merge::{
+ BufferedBatch, LookupProvider, MergeFunction, MergeResult, MergeRow,
+};
+use crate::{Error, Result};
+use arrow_array::RecordBatch;
+use arrow_schema::SchemaRef;
+use async_trait::async_trait;
+use std::sync::Arc;
+use std::time::{SystemTime, UNIX_EPOCH};
+
+const RECORD_LEVEL_EXPIRE_TIME: &str = "record-level.expire-time";
+const RECORD_LEVEL_TIME_FIELD: &str = "record-level.time-field";
+/// Java treats BIGINT values at or above this threshold as milliseconds.
+const BIGINT_MILLIS_THRESHOLD: i64 = 1_000_000_000_000;
+
+/// Compaction-time filter for expired primary-key versions.
+#[derive(Debug, Clone)]
+pub(crate) struct RecordLevelExpire {
+ expire_secs: i64,
+ time_field: DataField,
+}
+
+impl RecordLevelExpire {
+ /// Build from table options, or `Ok(None)` when expire-time is unset.
+ pub(crate) fn create(schema: &TableSchema) -> Result<Option<Self>> {
+ let options = schema.options();
+ let Some(raw) = options.get(RECORD_LEVEL_EXPIRE_TIME) else {
+ return Ok(None);
+ };
+ if options
+ .get("pk-clustering-override")
+ .is_some_and(|value| value.eq_ignore_ascii_case("true"))
+ {
+ return Err(Error::Unsupported {
+ message: "Cannot support record-level.expire-time with
pk-clustering-override"
+ .into(),
+ });
+ }
+ let expire_millis = parse_duration_millis(raw).ok_or_else(||
Error::ConfigInvalid {
+ message: format!("Invalid record-level.expire-time '{raw}'"),
+ })?;
+ let expire_secs =
+ i64::try_from(expire_millis / 1000).map_err(|_|
Error::ConfigInvalid {
+ message: format!("record-level.expire-time '{raw}' is out of
range"),
+ })?;
+ let time_field_name =
+ options
+ .get(RECORD_LEVEL_TIME_FIELD)
+ .ok_or_else(|| Error::ConfigInvalid {
+ message: "You should set time field for record-level
expire.".into(),
+ })?;
+ let time_field = schema
+ .fields()
+ .iter()
+ .find(|field| field.name() == time_field_name)
+ .cloned()
+ .ok_or_else(|| Error::ConfigInvalid {
+ message: format!(
+ "Can not find time field {time_field_name} for record
level expire."
+ ),
+ })?;
+ validate_time_field_type(time_field.data_type())?;
+ Ok(Some(Self {
+ expire_secs,
+ time_field,
+ }))
+ }
+
+ /// Convert a datum to epoch seconds, matching Java
+ /// `createFieldGetterAndConvertToSecond`. `None` means NULL (keep).
+ pub(crate) fn datum_to_seconds(&self, datum: &Datum) ->
Result<Option<i64>> {
+ datum_to_seconds(self.time_field.data_type(), datum)
+ }
+
+ /// Keep when `now_sec <= time_sec + expire_secs`, or when time is NULL.
+ pub(crate) fn is_live(&self, time_secs: Option<i64>, now_secs: i64) ->
bool {
+ match time_secs {
+ None => true,
+ Some(time) => now_secs <= time.saturating_add(self.expire_secs),
+ }
+ }
+
+ /// Whole-file expire only when min(time) is known and already past the
+ /// expire watermark. Unknown stats return `false` (never force rewrite).
+ ///
+ /// Callers must pass the same `now_secs` used when the compaction task was
+ /// planned so selection stays stable through execute and validation.
+ pub(crate) fn is_expire_file(
+ &self,
+ file: &DataFileMeta,
+ schema: &TableSchema,
+ now_secs: i64,
+ ) -> bool {
+ let Some(stats) =
+ file.value_stats_for_field(schema.id(), schema.fields(),
&self.time_field)
+ else {
+ return false;
+ };
+ let Some(min) = stats.min_value.as_ref() else {
+ return false;
+ };
+ let Ok(Some(min_secs)) = self.datum_to_seconds(min) else {
+ return false;
+ };
+ now_secs.saturating_sub(self.expire_secs) > min_secs
+ }
+
+ /// Whether a sort-merge row should be kept (NULL time or within expire
window).
+ pub(crate) fn keeps_merge_row(
+ &self,
+ batches: &[BufferedBatch],
+ row: &MergeRow,
+ columns: &[usize],
+ schema: &SchemaRef,
+ now_secs: i64,
+ ) -> Result<bool> {
+ let time = self.row_time_secs(batches, row, columns, schema)?;
+ Ok(self.is_live(time, now_secs))
+ }
+
+ /// Whether a single-row KV batch (e.g. LevelLookup hit) should be kept.
+ pub(crate) fn keeps_record_batch(
+ &self,
+ batch: &RecordBatch,
+ row_idx: usize,
+ now_secs: i64,
+ ) -> Result<bool> {
+ let output_idx = batch
+ .schema()
+ .index_of(self.time_field.name())
+ .map_err(|_| Error::DataInvalid {
+ message: format!(
+ "Compaction batch is missing record-level time field '{}'",
+ self.time_field.name()
+ ),
+ source: None,
+ })?;
+ let datum = extract_datum_from_array(
+ batch.column(output_idx),
+ row_idx,
+ output_idx,
+ self.time_field.data_type(),
+ )?;
+ let time = match datum {
+ None => None,
+ Some(value) => self.datum_to_seconds(&value)?,
+ };
+ Ok(self.is_live(time, now_secs))
+ }
+
+ fn row_time_secs(
+ &self,
+ batches: &[BufferedBatch],
+ row: &MergeRow,
+ columns: &[usize],
+ schema: &SchemaRef,
+ ) -> Result<Option<i64>> {
+ let output_idx = schema
+ .fields()
+ .iter()
+ .position(|field| field.name() == self.time_field.name())
+ .ok_or_else(|| Error::DataInvalid {
+ message: format!(
+ "Compaction batch is missing record-level time field '{}'",
+ self.time_field.name()
+ ),
+ source: None,
+ })?;
+ let array = batches[row.batch_idx].column_ref_for_output(output_idx,
columns);
+ let datum =
+ extract_datum_from_array(array, row.row_idx, output_idx,
self.time_field.data_type())?;
+ match datum {
+ None => Ok(None),
+ Some(value) => self.datum_to_seconds(&value),
+ }
+ }
+}
+
+/// Drops expired LevelLookup hits before sort-merge injects them as
before-state.
+pub(crate) struct ExpiringLookup {
+ inner: Arc<dyn LookupProvider>,
+ expire: RecordLevelExpire,
+ now_secs: i64,
+}
+
+impl ExpiringLookup {
+ pub(crate) fn wrap(
+ inner: Arc<dyn LookupProvider>,
+ expire: RecordLevelExpire,
+ now_secs: i64,
+ ) -> Arc<dyn LookupProvider> {
+ Arc::new(Self {
+ inner,
+ expire,
+ now_secs,
+ })
+ }
+}
+
+#[async_trait]
+impl LookupProvider for ExpiringLookup {
+ async fn lookup(&self, key: &[u8]) -> Result<Option<RecordBatch>> {
+ let Some(batch) = self.inner.lookup(key).await? else {
Review Comment:
**[P1] Continue lookup when one retained historical candidate expires**
The underlying store chooses the first hit by level before this wrapper can
check expiration. I verified an L0-only Lookup aggregation with a live L3 value
`10`, an expired L2 materialization `15`, and a live L0 delta `5`. It publishes
`5` and a `+I(5)` instead of materializing `15` and publishing `-U(10),
+U(15)`. Java filters each historical reader before collecting lookup
candidates, so the older live level can still be found. Move filtering to
candidate selection and preserve first-row and deletion-vector behavior.
##########
crates/paimon/src/table/compact/primary_key.rs:
##########
@@ -86,6 +92,29 @@ impl PrimaryKeyCompactPlanner {
total_buckets,
files,
output_level: Some(output_level),
+ force_level_zero,
+ expire_now_secs,
+ blob_field_id: None,
+ vector_field_ids: None,
+ });
+ } else if let Some((files, output_level)) =
planner::select_expired_max_level(
Review Comment:
**[P2] Schedule expiration in supported complete-bucket modes**
The TTL-only fallback is reachable only when incremental planning is
enabled. With `merge-engine=partial-update`, default `ignore-delete=false`, and
`num-levels=2`, I seeded one row, compacted it to L1, then enabled TTL without
changing the schema ID. Explicit compaction still produced zero tasks for the
fully expired row. The newly advertised PK TTL behavior needs a complete-bucket
rewrite trigger in this mode, or a clearly defined support restriction.
##########
crates/paimon/src/table/compact.rs:
##########
@@ -1248,18 +1290,24 @@ pub(super) fn ensure_supported(table: &Table) ->
Result<()> {
message: "Compaction of ordered bucketed append tables is not
supported; bucket-append-ordered=false is required".into(),
});
}
- if properties
+ if properties.contains_key("record-level.expire-time") {
+ if table.schema().primary_keys().is_empty() {
+ return Err(Error::Unsupported {
+ message: "record-level.expire-time is only supported for
primary-key compaction"
+ .into(),
+ });
+ }
+ // Validate time-field / duration and reject expire + clustering.
+ let _ =
crate::table::record_level_expire::RecordLevelExpire::create(table.schema())?;
Review Comment:
**[P2] Validate the active TTL time field at CREATE and ALTER**
Using the filesystem catalog, a valid PK table with TTL accepts `ALTER
RENAME COLUMN ts TO observed_at` and keeps `record-level.time-field=ts`. After
reloading, a write with the correct renamed Arrow field fails at commit with
`Can not find time field ts for record level expire`. Reject DDL that leaves an
invalid TTL reference or update the reference on rename in shared schema
validation, before persisting a schema that blocks writes.
##########
crates/paimon/src/table/record_level_expire.rs:
##########
@@ -0,0 +1,598 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Record-level expiration for primary-key compaction.
+//!
+//! Mirrors Java `org.apache.paimon.io.RecordLevelExpire`: versions whose
+//! configured time field is older than `now - expire` are dropped **before**
+//! merge during compaction. Query reads are unaffected.
+
+use crate::spec::{
+ extract_datum_from_array, parse_duration_millis, DataField, DataFileMeta,
DataType, Datum,
+ TableSchema,
+};
+use crate::table::sort_merge::{
+ BufferedBatch, LookupProvider, MergeFunction, MergeResult, MergeRow,
+};
+use crate::{Error, Result};
+use arrow_array::RecordBatch;
+use arrow_schema::SchemaRef;
+use async_trait::async_trait;
+use std::sync::Arc;
+use std::time::{SystemTime, UNIX_EPOCH};
+
+const RECORD_LEVEL_EXPIRE_TIME: &str = "record-level.expire-time";
+const RECORD_LEVEL_TIME_FIELD: &str = "record-level.time-field";
+/// Java treats BIGINT values at or above this threshold as milliseconds.
+const BIGINT_MILLIS_THRESHOLD: i64 = 1_000_000_000_000;
+
+/// Compaction-time filter for expired primary-key versions.
+#[derive(Debug, Clone)]
+pub(crate) struct RecordLevelExpire {
+ expire_secs: i64,
+ time_field: DataField,
+}
+
+impl RecordLevelExpire {
+ /// Build from table options, or `Ok(None)` when expire-time is unset.
+ pub(crate) fn create(schema: &TableSchema) -> Result<Option<Self>> {
+ let options = schema.options();
+ let Some(raw) = options.get(RECORD_LEVEL_EXPIRE_TIME) else {
+ return Ok(None);
+ };
+ if options
+ .get("pk-clustering-override")
+ .is_some_and(|value| value.eq_ignore_ascii_case("true"))
+ {
+ return Err(Error::Unsupported {
+ message: "Cannot support record-level.expire-time with
pk-clustering-override"
+ .into(),
+ });
+ }
+ let expire_millis = parse_duration_millis(raw).ok_or_else(||
Error::ConfigInvalid {
+ message: format!("Invalid record-level.expire-time '{raw}'"),
+ })?;
+ let expire_secs =
+ i64::try_from(expire_millis / 1000).map_err(|_|
Error::ConfigInvalid {
+ message: format!("record-level.expire-time '{raw}' is out of
range"),
+ })?;
+ let time_field_name =
+ options
+ .get(RECORD_LEVEL_TIME_FIELD)
+ .ok_or_else(|| Error::ConfigInvalid {
+ message: "You should set time field for record-level
expire.".into(),
+ })?;
+ let time_field = schema
+ .fields()
+ .iter()
+ .find(|field| field.name() == time_field_name)
+ .cloned()
+ .ok_or_else(|| Error::ConfigInvalid {
+ message: format!(
+ "Can not find time field {time_field_name} for record
level expire."
+ ),
+ })?;
+ validate_time_field_type(time_field.data_type())?;
+ Ok(Some(Self {
+ expire_secs,
+ time_field,
+ }))
+ }
+
+ /// Convert a datum to epoch seconds, matching Java
+ /// `createFieldGetterAndConvertToSecond`. `None` means NULL (keep).
+ pub(crate) fn datum_to_seconds(&self, datum: &Datum) ->
Result<Option<i64>> {
+ datum_to_seconds(self.time_field.data_type(), datum)
+ }
+
+ /// Keep when `now_sec <= time_sec + expire_secs`, or when time is NULL.
+ pub(crate) fn is_live(&self, time_secs: Option<i64>, now_secs: i64) ->
bool {
+ match time_secs {
+ None => true,
+ Some(time) => now_secs <= time.saturating_add(self.expire_secs),
+ }
+ }
+
+ /// Whole-file expire only when min(time) is known and already past the
+ /// expire watermark. Unknown stats return `false` (never force rewrite).
+ ///
+ /// Callers must pass the same `now_secs` used when the compaction task was
+ /// planned so selection stays stable through execute and validation.
+ pub(crate) fn is_expire_file(
+ &self,
+ file: &DataFileMeta,
+ schema: &TableSchema,
+ now_secs: i64,
+ ) -> bool {
+ let Some(stats) =
+ file.value_stats_for_field(schema.id(), schema.fields(),
&self.time_field)
Review Comment:
**[P2] Resolve compatible TTL statistics across schema IDs**
I seeded an already compacted bucket with one expired top-level row, enabled
TTL through filesystem catalog `alter_table`, then reloaded. The ALTER only
changes options and increments the schema ID. Manual compaction plans zero
tasks and the expired row remains readable. The same data with the existing
test's `copy_with_options` control plans one task because that API preserves
the schema ID. Resolve the historical field by ID and interpret compatible
stats using that file's schema, as Java does. This also matters after an
unrelated column change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]