[
https://issues.apache.org/jira/browse/HDDS-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang resolved HDDS-10670.
------------------------------------
Resolution: Invalid
HDDS-10670 is invalidated by HDDS-10770.
> [Hsync] OverWrite using "sh key put" shouldn't be allowed on hsynced files.
> ---------------------------------------------------------------------------
>
> Key: HDDS-10670
> URL: https://issues.apache.org/jira/browse/HDDS-10670
> Project: Apache Ozone
> Issue Type: Bug
> Components: OM
> Reporter: Pratyush Bhatt
> Priority: Major
>
> *Scenario:* Do a _sh key put_ on the same key that has been hsynced and is
> open.
> *Observations:*
> Open a file, write data and do hsync
> {code:java}
> Data written on iteration : 1
> Data written on iteration : 2
> Data written on iteration : 3
> Data written on iteration : 4
> Data written on iteration : 5
> Data written on iteration : 6
> Data written on iteration : 7
> Data written on iteration : 8
> Data written on iteration : 9
> Data written on iteration : 10
> Hsync completed on file, counter: 10 {code}
> ----Here the code sleeps of 60 seconds-----
> Checking the file is Open and hsynced
> {code:java}
> ozone admin om lof --service-id=ozone1712556312 --prefix=/hsyncvol/hsyncbuck/
> 1151 total open files (est.). Showing 1 open files (limit 100) under path
> prefix:
> /hsyncvol/hsyncbuck/Client ID Creation time Hsync'ed Open File
> Path
> 112241980463843099 1712676703855 Yes
> /hsyncvol/hsyncbuck/-9223372036849133055/File_new_11.txtReached the end of
> the list. {code}
> Doing a key info till now:
> {code:java}
> ozone sh key info hsyncvol/hsyncbuck/hsync/File_new_11.txt
> 24/04/09 15:31:48 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/04/09 15:31:49 INFO client.ClientTrustManager: Loading certificates for
> client.
> {
> "volumeName" : "hsyncvol",
> "bucketName" : "hsyncbuck",
> "name" : "hsync/File_new_11.txt",
> "dataSize" : 51200,
> "creationTime" : "2024-04-09T15:31:43.855Z",
> "modificationTime" : "2024-04-09T15:31:45.145Z",
> "replicationConfig" : {
> "replicationFactor" : "THREE",
> "requiredNodes" : 3,
> "replicationType" : "RATIS"
> },
> "metadata" : {
> "hsyncClientId" : "112241980463843099"
> },
> "ozoneKeyLocations" : [ {
> "containerID" : 2,
> "localID" : 113750153625604824,
> "length" : 51200,
> "offset" : 0,
> "keyOffset" : 0
> } ],
> "file" : true
> } {code}
> Now OverWrite the same key using _sh key put_
> {code:java}
> ozone sh key put hsyncvol/hsyncbuck/hsync/File_new_11.txt /etc/passwd
> 24/04/09 15:32:08 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/04/09 15:32:09 INFO client.ClientTrustManager: Loading certificates for
> client.
> 24/04/09 15:32:10 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
> 24/04/09 15:32:10 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/04/09 15:32:10 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics
> system started
> 24/04/09 15:32:10 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.ratis.metrics.dropwizard3.Dm3MetricRegistriesImpl {code}
> Key info, successfully overwritten:
> {code:java}
> ozone sh key info hsyncvol/hsyncbuck/hsync/File_new_11.txt
> 24/04/09 15:32:17 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/04/09 15:32:17 INFO client.ClientTrustManager: Loading certificates for
> client.
> {
> "volumeName" : "hsyncvol",
> "bucketName" : "hsyncbuck",
> "name" : "hsync/File_new_11.txt",
> "dataSize" : 6644,
> "creationTime" : "2024-04-09T15:31:43.855Z",
> "modificationTime" : "2024-04-09T15:32:11.283Z",
> "replicationConfig" : {
> "replicationFactor" : "THREE",
> "requiredNodes" : 3,
> "replicationType" : "RATIS"
> },
> "metadata" : { },
> "ozoneKeyLocations" : [ {
> "containerID" : 1,
> "localID" : 113750153625604825,
> "length" : 6644,
> "offset" : 0,
> "keyOffset" : 0
> } ],
> "file" : true
> } {code}
> But key is still open in the OpenKeyTable:
> {code:java}
> ozone admin om lof --service-id=ozone1712556312 --prefix=/hsyncvol/hsyncbuck/
> 1151 total open files (est.). Showing 1 open files (limit 100) under path
> prefix:
> /hsyncvol/hsyncbuck/Client ID Creation time Hsync'ed Open File
> Path
> 112241980463843099 1712676703855 Yes
> /hsyncvol/hsyncbuck/-9223372036849133055/File_new_11.txtReached the end of
> the list. {code}
> Perform LeaseRecovery:
> {code:java}
> ozone debug recover
> --path=ofs://ozone1712556312//hsyncvol/hsyncbuck/hsync/File_new_11.txt
> 24/04/09 15:32:32 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/04/09 15:32:32 INFO client.ClientTrustManager: Loading certificates for
> client.
> Lease recovery SUCCEEDED on
> ofs://ozone1712556312//hsyncvol/hsyncbuck/hsync/File_new_11.txt {code}
> Still the key is open:
> {code:java}
> ozone admin om lof --service-id=ozone1712556312 --prefix=/hsyncvol/hsyncbuck/
> 1151 total open files (est.). Showing 1 open files (limit 100) under path
> prefix:
> /hsyncvol/hsyncbuck/Client ID Creation time Hsync'ed Open File
> Path
> 112241980463843099 1712676703855 Yes
> /hsyncvol/hsyncbuck/-9223372036849133055/File_new_11.txtReached the end of
> the list. {code}
> -- 60 seconds of code pause is resumed now--
> Still able to write the data into the file and hsync
> {code:java}
> Data written on iteration : 11
> Data written on iteration : 12
> Data written on iteration : 13
> Data written on iteration : 14
> Data written on iteration : 15
> Data written on iteration : 16
> Data written on iteration : 17
> Data written on iteration : 18
> Data written on iteration : 19
> Data written on iteration : 20
> Hsync completed on file, counter: 20 {code}
> Basically looks like lease recovery gets broken if we allow such overwriting.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
