Umeshkumar9414 opened a new pull request, #8877:
URL: https://github.com/apache/hbase/pull/8877

   JIRA: https://issues.apache.org/jira/browse/HBASE-30539
   
   ### Problem
   `FSTableDescriptors.writeTableDescriptor` retries 10 times, logs each 
failure at DEBUG, then returns `null`. `createTableDescriptorForTableDirectory` 
turns that into `false`, which is the same value it returns for "an equal 
descriptor already exists". `CreateTableProcedure.createFsLayout` ignores the 
return value, so when HDFS refuses every `.tableinfo` write the procedure still 
moves on to `CREATE_TABLE_ADD_TO_META` and `CREATE_TABLE_ASSIGN_REGIONS`.
   
   The table then exists in `hbase:meta` and in the master's descriptor cache, 
but not on the file system. The RegionServer has no descriptor cache, so every 
region open fails with `Missing table descriptor for ...`. The child 
`TransitRegionStateProcedure` retries forever and the parent create procedure 
stays in `ASSIGN_REGIONS` holding the table lock.
   
   ### Fix
   `writeTableDescriptor` now throws `IOException` (last failure as the cause) 
after all attempts fail, and logs failed attempts at WARN. `createFsLayout` 
therefore fails the step with an `IOException`; because 
`CREATE_TABLE_WRITE_FS_LAYOUT` begins with the pre-clean of the table dir, the 
procedure's retry with backoff is idempotent. Once HDFS recovers, the same 
procedure completes.
   
   `createTableDescriptorForTableDirectory` now returns `false` only when an 
equal descriptor already exists. The two callers that checked for `null` 
(`updateTableDescriptor`, `tryUpdateAndGetMetaTableDescriptor`) lose their dead 
checks. Other callers that ignored the result (`CloneSnapshotProcedure`, 
`SnapshotManifest`, `MasterRegion`, `BackupUtils`) now see the failure instead 
of continuing silently.
   
   ### Test
   `TestCreateTableProcedureDescriptorWriteFailure` injects `.tabledesc` create 
failures through a `DistributedFileSystem` subclass. While writes fail, the 
create procedure keeps retrying and does not fail, finish, add rows to 
`hbase:meta`, or start a child assign procedure. After the injection is 
cleared, the same procedure completes and the region server can load the 
descriptor. The same test, written against the unfixed code, reproduced the bug 
(rows in meta, `Missing table descriptor`, child TRSP looping in "Failed 
transition, suspend").
   
   Also run locally and passing: `TestFSTableDescriptors`, 
`TestCreateTableProcedure`, `TestCloneSnapshotProcedure`, 
`TestSnapshotManifest`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to