def downloadS3ObjectFlow(s3Client: S3Client, bucket: String): Flow[String,
ByteString, NotUsed] = {
Flow[String].flatMapConcat { key =>
log.debug(s"Downloading $key")
val data: Source[ByteString, NotUsed] = s3Client.download(bucket, key)
data.recoverWithRetries(3, {
case SomeExpectedException => data
})
}
}
On Thu, Feb 23, 2017 at 5:20 PM, Sean Callahan <[email protected]>
wrote:
> Hey all, Currently I have the following flow to take in an S3 key and
> download that that file from S3.
>
> def downloadS3ObjectFlow(s3Client: S3Client, bucket: String): Flow[String,
> ByteString, NotUsed] = {
> Flow[String].map { key =>
> log.debug(s"Downloading $key")
> val data: Source[ByteString, NotUsed] = s3Client.download(bucket,
> key)
> data
> }.flatMapConcat(identity)
> }
>
>
>
> In 99% of cases, this works great, but as many of us know, S3 fails a fair
> amount for random unknown reasons the proper way to resolve it is to just
> retry the request. After a few hours of googling and trying various
> different solutions out there I have yet to find the proper solution here.
> Is there anything built into Akka that can accomplish this, or should I
> write some custom logic here to check the value of "data" and setup some
> type of stateful retries?
>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
--
Cheers,
√
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
