[
https://issues.apache.org/jira/browse/HDDS-10675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang resolved HDDS-10675.
------------------------------------
Resolution: Duplicate
It duplicates HDDS-10670. Will continue the work over there.
> [Hsync] OverWrite using "sh key cp" shouldn't be allowed on hsynced files.
> --------------------------------------------------------------------------
>
> Key: HDDS-10675
> URL: https://issues.apache.org/jira/browse/HDDS-10675
> Project: Apache Ozone
> Issue Type: Bug
> Components: OM
> Reporter: Pratyush Bhatt
> Priority: Major
>
> Similar Scenario on the lines of HDDS-10670
> *Scenario:* Do a _sh key cp_ on the same key that has been hsynced and is
> open.
> *Observations:*
> Open a file, write data and do hsync
> {code:java}
> Data written on iteration : 1
> Data written on iteration : 2
> Data written on iteration : 3
> Data written on iteration : 4
> Data written on iteration : 5
> Data written on iteration : 6
> Data written on iteration : 7
> Data written on iteration : 8
> Data written on iteration : 9
> Data written on iteration : 10
> Hsync completed on file, counter: 10 {code}
> --Here the code sleeps of 60 seconds--
> Checking the file is Open and hsynced
> {code:java}
> ozone admin om lof --service-id=ozone1712556312 --prefix=/hsyncvol/hsyncbuck/
> 1809 total open files (est.). Showing 1 open files (limit 100) under path
> prefix:
> /hsyncvol/hsyncbuck/Client ID Creation time Hsync'ed Open File
> Path
> 112246067997904292 1712739074675 Yes
> /hsyncvol/hsyncbuck/-9223372036845049599/File_new_12.txtReached the end of
> the list.{code}
> Doing a key info till now:
> {code:java}
> ozone sh key info hsyncvol/hsyncbuck/hsync/File_new_12.txt
> 24/04/10 08:51:18 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/04/10 08:51:18 INFO client.ClientTrustManager: Loading certificates for
> client.
> {
> "volumeName" : "hsyncvol",
> "bucketName" : "hsyncbuck",
> "name" : "hsync/File_new_12.txt",
> "dataSize" : 51200,
> "creationTime" : "2024-04-10T08:51:14.675Z",
> "modificationTime" : "2024-04-10T08:51:16.008Z",
> "replicationConfig" : {
> "replicationFactor" : "THREE",
> "requiredNodes" : 3,
> "replicationType" : "RATIS"
> },
> "metadata" : {
> "hsyncClientId" : "112246067997904292"
> },
> "ozoneKeyLocations" : [ {
> "containerID" : 6,
> "localID" : 113750153625606497,
> "length" : 51200,
> "offset" : 0,
> "keyOffset" : 0
> } ],
> "file" : true
> }{code}
> Now OverWrite the same key using _sh key cp_
> {code:java}
> >> ozone sh key put hsyncvol/hsyncbuck/dummy_file.txt /etc/passwd
> 24/04/10 08:51:32 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/04/10 08:51:32 INFO client.ClientTrustManager: Loading certificates for
> client.
> 24/04/10 08:51:33 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
> 24/04/10 08:51:33 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/04/10 08:51:33 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics
> system started
> 24/04/10 08:51:34 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.ratis.metrics.dropwizard3.Dm3MetricRegistriesImpl
> >> ozone sh key cp /hsyncvol/hsyncbuck/ dummy_file.txt hsync/File_new_12.txt
> 24/04/10 08:51:48 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/04/10 08:51:49 INFO client.ClientTrustManager: Loading certificates for
> client.
> 24/04/10 08:51:50 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
> 24/04/10 08:51:50 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/04/10 08:51:50 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics
> system started
> 24/04/10 08:51:51 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.ratis.metrics.dropwizard3.Dm3MetricRegistriesImpl{code}
> Key info, successfully overwritten:
> {code:java}
> ozone sh key info hsyncvol/hsyncbuck/hsync/File_new_12.txt
> 24/04/10 08:51:56 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/04/10 08:51:57 INFO client.ClientTrustManager: Loading certificates for
> client.
> {
> "volumeName" : "hsyncvol",
> "bucketName" : "hsyncbuck",
> "name" : "hsync/File_new_12.txt",
> "dataSize" : 6644,
> "creationTime" : "2024-04-10T08:51:14.675Z",
> "modificationTime" : "2024-04-10T08:51:51.673Z",
> "replicationConfig" : {
> "replicationFactor" : "THREE",
> "requiredNodes" : 3,
> "replicationType" : "RATIS"
> },
> "metadata" : { },
> "ozoneKeyLocations" : [ {
> "containerID" : 3,
> "localID" : 113750153625606500,
> "length" : 6644,
> "offset" : 0,
> "keyOffset" : 0
> } ],
> "file" : true
> }{code}
> But key is still open in the OpenKeyTable:
> {code:java}
> ozone admin om lof --service-id=ozone1712556312 --prefix=/hsyncvol/hsyncbuck/
> 1809 total open files (est.). Showing 1 open files (limit 100) under path
> prefix:
> /hsyncvol/hsyncbuck/Client ID Creation time Hsync'ed Open File
> Path
> 112246067997904292 1712739074675 Yes
> /hsyncvol/hsyncbuck/-9223372036845049599/File_new_12.txtReached the end of
> the list.{code}
> Perform LeaseRecovery:
> {code:java}
> ozone debug recover
> --path=ofs://ozone1712556312//hsyncvol/hsyncbuck/hsync/File_new_12.txt
> 24/04/10 08:52:13 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/04/10 08:52:13 INFO client.ClientTrustManager: Loading certificates for
> client.
> Lease recovery SUCCEEDED on
> ofs://ozone1712556312//hsyncvol/hsyncbuck/hsync/File_new_12.txt{code}
> Still the key is open:
> {code:java}
> ozone admin om lof --service-id=ozone1712556312 --prefix=/hsyncvol/hsyncbuck/
> 1809 total open files (est.). Showing 1 open files (limit 100) under path
> prefix:
> /hsyncvol/hsyncbuck/Client ID Creation time Hsync'ed Open File
> Path
> 112246067997904292 1712739074675 Yes
> /hsyncvol/hsyncbuck/-9223372036845049599/File_new_12.txtReached the end of
> the list.{code}
> – 60 seconds of code pause is resumed now--
> Still able to write the data into the file and hsync
> {code:java}
> Data written on iteration : 11
> Data written on iteration : 12
> Data written on iteration : 13
> Data written on iteration : 14
> Data written on iteration : 15
> Data written on iteration : 16
> Data written on iteration : 17
> Data written on iteration : 18
> Data written on iteration : 19
> Data written on iteration : 20
> Hsync completed on file, counter: 20 {code}
> Basically looks like lease recovery gets broken if we allow such overwriting.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
