[
https://issues.apache.org/jira/browse/RATIS-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChenXi updated RATIS-1705:
--------------------------
Description:
How to reproduce bugs:
# an ozone cluster with s3g components.
# mutliupload an object to s3g.
It is very important that you use an s3g that is not receiving any other
requests before doing a mutliupload. In other words, *the mutliupload must be
the first request received by the s3g after it starts.*
there is a mutliupload example code:
{code:python}
import boto3
from boto3.s3.transfer import TransferConfig
# boto3 maybe need install
MB = 1024 * 1024
s3 = boto3.resource('s3')
MultipartChunkSize = 5 * MB
MaxConcurrency = 10
AccessKeyID = "user_id"
SecretAccessKey = "user_key"
BucketRegion = "dummyregion"
BucketName = "testbucket"
EndpointUrl = "http://localhost:9878";
LocalFilePath = "/root/50M.img"
ObjectKeyName = "key_from_py"
def multipart_upload(client, local_file_path, bucket_name, object_key,
multipart_chunksize,
max_concurrency):
config = TransferConfig(multipart_chunksize=multipart_chunksize,
max_concurrency=max_concurrency)
client.upload_file(local_file_path, bucket_name, object_key, Config=config)
if __name__ == '__main__':
client = boto3.client('s3',
endpoint_url=EndpointUrl,
aws_access_key_id=AccessKeyID,
aws_secret_access_key=SecretAccessKey)
multipart_upload(client, LocalFilePath, BucketName, ObjectKeyName,
MultipartChunkSize,
MaxConcurrency)
{code}
After that, every time upload(mutliupload / normal upload), the ratis grpc
metrics will increase,
You can run
{code:java}
curl -s http://{you_address}:9878/jmx | grep client_message_metrics | wc -l
{code}
to verify that
was:
How to reproduce bugs:
# an ozone cluster with s3g components.
# mutliupload an object to s3g.
It is very important that you use an s3g that is not receiving any other
requests before doing a mutliupload. In other words, *the mutliupload must be
the first request received by the s3g after it starts.*
there is a mutliupload example code:
{code:python}
import boto3
from boto3.s3.transfer import TransferConfig
# boto3 maybe need install
MB = 1024 * 1024
s3 = boto3.resource('s3')
MultipartChunkSize = 5 * MB
MaxConcurrency = 10
AccessKeyID = "user_id"
SecretAccessKey = "user_key"
BucketRegion = "dummyregion"
BucketName = "testbucket"
EndpointUrl = "http://localhost:9878";
LocalFilePath = "/root/50M.img"
ObjectKeyName = "key_from_py"
def multipart_upload(client, local_file_path, bucket_name, object_key,
multipart_chunksize,
max_concurrency):
config = TransferConfig(multipart_chunksize=multipart_chunksize,
max_concurrency=max_concurrency)
client.upload_file(local_file_path, bucket_name, object_key, Config=config)
if __name__ == '__main__':
client = boto3.client('s3',
endpoint_url=EndpointUrl,
aws_access_key_id=AccessKeyID,
aws_secret_access_key=SecretAccessKey)
multipart_upload(client, LocalFilePath, BucketName, ObjectKeyName,
MultipartChunkSize,
MaxConcurrency)
{code}
After that, every time upload(mutliupload / normal upload), the ratis grpc
metrics will increase,
You can run
"curl -s http://\{you_address}:9878/jmx | grep client_message_metrics | wc -l"
to verify that
> Ozone S3g leak ratis GRPC messages metrics
> ------------------------------------------
>
> Key: RATIS-1705
> URL: https://issues.apache.org/jira/browse/RATIS-1705
> Project: Ratis
> Issue Type: Bug
> Reporter: ChenXi
> Priority: Major
>
> How to reproduce bugs:
> # an ozone cluster with s3g components.
> # mutliupload an object to s3g.
> It is very important that you use an s3g that is not receiving any other
> requests before doing a mutliupload. In other words, *the mutliupload must be
> the first request received by the s3g after it starts.*
> there is a mutliupload example code:
> {code:python}
> import boto3
> from boto3.s3.transfer import TransferConfig
> # boto3 maybe need install
> MB = 1024 * 1024
> s3 = boto3.resource('s3')
> MultipartChunkSize = 5 * MB
> MaxConcurrency = 10
> AccessKeyID = "user_id"
> SecretAccessKey = "user_key"
> BucketRegion = "dummyregion"
> BucketName = "testbucket"
> EndpointUrl = "http://localhost:9878";
> LocalFilePath = "/root/50M.img"
> ObjectKeyName = "key_from_py"
> def multipart_upload(client, local_file_path, bucket_name, object_key,
> multipart_chunksize,
> max_concurrency):
> config = TransferConfig(multipart_chunksize=multipart_chunksize,
> max_concurrency=max_concurrency)
> client.upload_file(local_file_path, bucket_name, object_key,
> Config=config)
> if __name__ == '__main__':
> client = boto3.client('s3',
> endpoint_url=EndpointUrl,
> aws_access_key_id=AccessKeyID,
> aws_secret_access_key=SecretAccessKey)
> multipart_upload(client, LocalFilePath, BucketName, ObjectKeyName,
> MultipartChunkSize,
> MaxConcurrency)
> {code}
>
> After that, every time upload(mutliupload / normal upload), the ratis grpc
> metrics will increase,
> You can run
> {code:java}
> curl -s http://{you_address}:9878/jmx | grep client_message_metrics | wc -l
> {code}
> to verify that
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
