[
https://issues.apache.org/jira/browse/HDDS-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17533098#comment-17533098
]
Shawn commented on HDDS-6441:
-----------------------------
Hi [~erose] ,
Thanks for looking into this. Currently which aws s3cli version does ozone
support? I am using aws-cli 2.4.0, but sees got the parsing issues. aws s3
command is fine, but aws s3api seems has issues.
{code:java}
% aws --debug --ca-bundle xxx.pem --endpoint-url https://xxxxx s3api
list-buckets
.....
2022-05-06 14:32:41,689 - MainThread - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"
standalone="yes"?><ListAllMyBucketsResult
xmlns="http://s3.amazonaws.com/doc/2006-03-01/";><Buckets><Bucket><Name>2468014b-92e3-462f-91a6-fdcd0823db55</Name><CreationDate>2022-04-06T18:26:59.306Z</CreationDate></Bucket><Bucket><Name>5914d5ed-9a1b-44b7-b2d5-900ad42ed970</Name><CreationDate>2022-03-16T21:37:32.568Z</CreationDate></Bucket><Bucket><Name>a5139323-e310-4405-be9f-d03698cf9be5</Name><CreationDate>2022-03-16T21:32:41.278Z</CreationDate></Bucket><Bucket><Name>shawn-test</Name><CreationDate>2022-03-16T20:42:56.623Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>'
2022-05-06 14:32:41,690 - MainThread - botocore.hooks - DEBUG - Event
needs-retry.s3.ListBuckets: calling handler <bound method
RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at
0x7fcaa8902ac0>>
2022-05-06 14:32:41,690 - MainThread - botocore.retries.standard - DEBUG - Not
retrying request.
2022-05-06 14:32:41,690 - MainThread - botocore.hooks - DEBUG - Event
needs-retry.s3.ListBuckets: calling handler <bound method
S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector
object at 0x7fcaa8902b20>>
2022-05-06 14:32:41,691 - MainThread - botocore.hooks - DEBUG - Event
after-call.s3.ListBuckets: calling handler <function enhance_error_msg at
0x7fca6824eb80>
2022-05-06 14:32:41,691 - MainThread - botocore.hooks - DEBUG - Event
after-call.s3.ListBuckets: calling handler <bound method
RetryQuotaChecker.release_retry_quota of
<botocore.retries.standard.RetryQuotaChecker object at 0x7fcaa89026d0>>
2022-05-06 14:32:41,691 - MainThread - awscli.clidriver - DEBUG - Exception
caught in main()
Traceback (most recent call last):
File "awscli/clidriver.py", line 459, in main
File "awscli/clidriver.py", line 594, in __call__
File "awscli/clidriver.py", line 770, in __call__
File "awscli/clidriver.py", line 903, in invoke
File "awscli/clidriver.py", line 923, in _display_response
File "awscli/formatter.py", line 376, in get_formatter
ValueError: Unknown output type: NoneUnknown output type: None{code}
> Ozone metadata does not align with underlying blocks when there are many
> incomplete uploads happens
> ---------------------------------------------------------------------------------------------------
>
> Key: HDDS-6441
> URL: https://issues.apache.org/jira/browse/HDDS-6441
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode
> Affects Versions: 1.2.0
> Reporter: Shawn
> Assignee: Ethan Rose
> Priority: Major
>
> Ozone metadata does not align with underlying blocks when there are many
> incomplete uploads happens. I have a cluster which has a very few objects.
> But the datanode usage tells me I almost run out of space.
> ????
> Usage info for datanode with UUID f50108f1-d8bf-44e3-abed-6e77c91f994d:
> Capacity : 8802545958912B
> SCMUsed : 8802128257024B (99.99525%)
> Remaining : 74715136B (0.00085%)
> Usage info for datanode with UUID 2bdb3198-b71f-4153-9663-e3b349c6f82a:
> Capacity : 8802545958912B
> SCMUsed : 8802133102592B (99.99531%)
> Remaining : 76824576B (0.00087%)
> Usage info for datanode with UUID d5644a36-b967-44a6-a736-4bd2013c2b86:
> Capacity : 8793955991552B
> SCMUsed : 8793311227904B (99.99267%)
> Remaining : 291676160B (0.00332%)
> ...
>
> Also I see there are lots of errors in logs, complaining out of disk space
> and also report missing .container files as below:
> ????
> 2022-03-10 03:56:02 ERROR Thread-6 ContainerReader:159 - Missing .container
> file for ContainerID: 15221
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
