tarun11Mavani commented on PR #19776:
URL: https://github.com/apache/pinot/pull/19776#issuecomment-6941662007

   > [@tarun11Mavani](https://github.com/tarun11Mavani) could you take a look 
at these two remaining cases and address them in a follow-up PR? I reviewed the 
final revision `d9338330` after merge. The failed-invalidation fix and the 
segment-name collision fix are addressed; the first issue below is a different 
collision introduced by the new root.
   > 
   > 1. **[P1] The new metadata archive root can collide with a live table and 
delete its full segments.** `Deleted_Segments_Metadata` is a valid raw table 
name, so its live deep-store directory is also the new archive root. The 
[unconditional 
sweep](https://github.com/apache/pinot/blob/d9338330bf12f45d07b985053ac16a721e5d13dd/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java#L466-L471)
 runs even with preservation disabled. On S3/GCS, listing that root returns 
live segment objects as supposed table directories. Listing each object as a 
directory searches `<segment>/` and returns empty; [the empty-directory 
cleanup](https://github.com/apache/pinot/blob/d9338330bf12f45d07b985053ac16a721e5d13dd/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java#L528-L535)
 then calls `delete(<segment>, false)`, which deletes the actual file object. 
For example, [S3PinotFS's file branch](https://github.com/apach
 
e/pinot/blob/d9338330bf12f45d07b985053ac16a721e5d13dd/pinot-plugins/pinot-file-system/pinot-s3/src/main/java/org/apache/pinot/plugin/filesystem/S3PinotFS.java#L603-L609)
 deletes that exact key. This affects both OFFLINE and REALTIME and does not 
wait for segment retention. Please protect existing table-name collisions and 
reject non-directory entries before directory cleanup; add a regression for a 
**table** named `Deleted_Segments_Metadata`, not only a segment with that name.
   > 2. **[P2] Matching sidecar paths does not establish that the sidecar 
describes the incoming build.** Start with `seg` and its old 
`seg.metadata.tar.gz`, then stage a changed `seg.tar.gz` in the same directory. 
A metadata push with `preferMetadataTarGz=false` and 
`copyToDeepStoreForMetadataPush=true` extracts fresh request metadata from the 
tar. However, [this 
exemption](https://github.com/apache/pinot/blob/d9338330bf12f45d07b985053ac16a721e5d13dd/pinot-controller/src/main/java/org/apache/pinot/controller/api/upload/ZKOperator.java#L695-L698)
 skips invalidation solely because the computed sidecar paths match. The 
controller replaces the full segment while retaining its old sidecar. A later 
metadata push using the default preference can consume the old CRC/document 
counts, and deletion preservation archives a mismatched pair. Please validate 
the sidecar against the incoming metadata, or invalidate/rewrite it from the 
request. A same-directory test with an old sidecar and independe
 ntly supplied fresh request metadata would cover this.
   > 
   > I would prioritize the first case because it can remove live deep-store 
segment files with the new feature flags off. These findings are based on 
source/control-flow inspection; I did not run a local reproducer.
   
   @Jackie-Jiang I have created a new PR to address 1. 
https://github.com/apache/pinot/pull/20284
   Will share the PR for 2 soon.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to