Hi Roland,
On 14 Aug 2015, at 12:21, Akka Team <[email protected]> wrote:
> Hi Jan,
>
> stream-compatible data sources are starting to appear, see for example Slick
> 3.0 with its DBIO model that can either return a Future[] or a Publisher[] of
> the query result.
Maybe you can help me wrap my head around a follow up question:
I see how I can define such Actions that provide access to a publisher on which
I can wire up a predefined flow (sorry if I am am not using the terms correctly
yet).
In such cases there already is an underlying 'driver' that "manages" the hooked
up streams - at least this is how I understand it.
Two questions:
1. can you roughly point me to the package or class that acts as this driver in
the akka-http server side implementation?
2. suppose I want to use streams to read from an object store (Swift, S3) that
requires
periodically akquiring an access token. How would you implement a 'driver'
in this case?
It would nee to maintain the access token, add it to the upstream requests
and
re-authenticate if the token expires.
In addition many parallel requests (that is: running streams) should be
supported.
In the past I have implemented this with parallel actors (and a router), with
each actor maintaining
its own access token and an akka agent to provide concurrent read access to the
token.
Would you suggest to do something like this to provide a 'driver' on which to
call such above 'actions'
that produce publishers?
IOW, would you in general suggest to use (some group of) actors to encapsulate
the external resource and all connection
management and hook in streams for each 'query'? Is this the correct way to
think about the balance bewteen
actors and streams?
Jan
> The former would be incorporated using mapAsync and the latter can be wrapped
> in a Source and (with some care, usually) integrated into a stream using
> `.flatten(FlattenStrategy.concat)`. Rate limiting is described in the
> documentation (see the Cookbook), for retries you’ll have to construct a
> feedback c ycle in the stream graph (e.g. like in the HTTP client; it is a
> bit involved but we might later condense this pattern into a readily reusable
> flow element).
>
> When it comes to streaming data sinks there also is movement e.g. in the form
> of reactive Kafka drivers, we should see Cassandra and Spark as well soon,
> but this movement has just begun.
>
> I am not aware of an FTP implementation for Akka Streams yet, maybe someone
> has done that and would speak up?
>
> Regards,
>
> Roland
>
>
> On Wed, Aug 12, 2015 at 12:23 PM, algermissen1971 <[email protected]>
> wrote:
> Hi all,
>
> I am looking for an akka-streams example that shows how one would approach
> talking to databases (or an FTP server in my case).
>
> Besides the basic question of how to create a Source or Sink on top of a
> database driver (or SFTP connection for that matter) I am looking for best
> practices on rate limiting and retries, or re-establishing a connection if it
> gets lost.
>
> Can somone point me to an example to get started?
>
> Jan
>
>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ:
> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Akka Team
> Typesafe - Reactive apps on the JVM
> Blog: letitcrash.com
> Twitter: @akkateam
>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ:
> >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
