Hi, I have almost the same problem with Ceph RGW, and currently do research about Apache Iceberg and Databricks Delta(opensource version). I think these libraries can address the problem.
2021년 2월 3일 (수) 오후 3:44, Boris Litvak <boris.lit...@skf.com>님이 작성: > This December AWS announced https://aws.amazon.com/s3/consistency/, are > you sure this is your problem? > > I think all these s3guard like wrappers are irrelevant right now. Please > correct me if I am wrong. > > > > *From:* David Morin <morin.david....@gmail.com> > *Sent:* Tuesday, 2 February 2021 22:26 > *To:* user@spark.apache.org > *Subject:* S3a Committer > > > > Hi, > > > > I have some issues at the moment with S3 API of Openstack Swift (S3a). > > This one is eventually consistent and it causes lots of issues with my > distributed jobs in Spark. > > Is the S3A committer able to fix that ? Or an "S3guard like" > implementation is the only way ? > > > > David >
