[
https://issues.apache.org/jira/browse/HADOOP-16875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042045#comment-17042045
]
Rafael Acevedo commented on HADOOP-16875:
-----------------------------------------
Hi [[email protected]], just some additional info: this is the chart for
"query returned item count". I guess it's because S3Guard soft-deletes the
metadata, and directory listing filters apply the filter for `is_deleted` are
applied on the code itself, instead of on the database itself. Any reasons why
this is not done in the db? This causes read capacity units to keep getting
higher, which elevates costs.
!image-2020-02-21-14-02-23-866.png|width=403,height=171!
> S3Guard: add support for other MetadataStores
> ---------------------------------------------
>
> Key: HADOOP-16875
> URL: https://issues.apache.org/jira/browse/HADOOP-16875
> Project: Hadoop Common
> Issue Type: Wish
> Affects Versions: 3.2.1
> Reporter: Rafael Acevedo
> Priority: Major
> Attachments: image-2020-02-21-14-02-23-866.png,
> image-2020-02-21-14-03-28-327.png
>
>
> Hi all,
>
> Are there any plans to add other MetadataStore implementations for S3Guard?
> DynamoDB costs are too high when the read capacity/write capacity are high.
>
> Maybe a Postgres/MySQL implementation is simple enough to implement and offer
> strong consistency.
> Another idea is to implement a Cassandra/Scylla MetadataStore(for better
> write scalability), but we should pay attention to consistency.
>
> Any thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
