[akka-user] Re: [ANNOUNCE] Akka 2.4.9-RC1 Released! (Akka HTTP Performance and Entity Streaming)
Some quick feedback on raw iron performance, on my local workstation. Your mileage my vary, but I'm pretty happy with these numbers. Minimal example Webserver code from: http://doc.akka.io/docs/akka/2.4/scala/http/routing-dsl/index.html wrk -d30 -c100 -t12 http://localhost:8080/hello Running 30s test @ http://localhost:8080/hello 12 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 2.15ms 12.36ms 309.76ms 97.72% Req/Sec21.98k 4.54k 39.19k75.18% 7827755 requests in 30.08s, 1.28GB read Requests/sec: 260269.56 Transfer/sec: 43.69MB Thanks Akka Team! Been looking forward to this for a long time. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Can I Pause some Flow In the the akka-stream?
Isn't a valve just a specialization of a throttle? Throttle at infinite = open valve Throttle at 0 = closed valve As long as you have an out of band way to adjust this throttle (via materialized value?), you should have your valve already. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Proper way to shut down an outgoing http request stream?
I have this basic client test. It works way better in 2.4.2-RC2 than ever
before, however I still get an error during cleanup:
[akka.actor.ActorSystemImpl(HttpClient)] Outgoing request stream error (akka
.stream.AbruptTerminationException)
Seems like there's a akka-http/streams bug here, or a user error on my part.
object StreamClientTest extends App {
import scala.concurrent.ExecutionContext.Implicits.global
implicit val system = ActorSystem("HttpClient", ConfigFactory.empty())
implicit val materializer = ActorMaterializer()
val host = "akka.io"
// val host = "www.google.com"
val uri = s"http://$host/;
val eventuallyResponse = Http().singleRequest(Get(uri = uri))
for(response <- eventuallyResponse)
{
println(s"Status: ${response.status}")
response.entity.dataBytes
.runForeach {bs => print(bs.utf8String)}
.onComplete {_ => materializer.shutdown(); system.terminate()}
}
}
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: Akka HTTP Performance (2.4.2-RC1)
+1 here as well. I've been waiting in anger for akka-http to catch up to spray in terms of performance and have verified some great leaps in recent builds. Thanks guys! If there really is a 6-7x speedup waiting to fall into place (after bug fixing), that would be amazing (and worth the wait). On Friday, February 5, 2016 at 3:18:17 PM UTC-8, Gavin Baumanis wrote: > > Please let me add my +1 to this thread. > it has been great to get an understanding of the current state of play for > akka HTTP and the how and whys of current performance limits and the soon > to be implemented updates! > > Great work everyone - as a consumer of all your hard work, I must say > Thank-you! > > Gavin. > > On Friday, 5 February 2016 09:46:08 UTC+11, tigerfoot wrote: >> >> Endre, >>> >> >> Thanks for this great explanation--makes complete sense. It's really >> exciting to see this generation of Akka functionality come together! >> >> Greg >> > -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: At present, which one is better to use Akka-HTTP or Spray.io for future based on a new learner?
At present, for a new learner, go with Spray. Spray is mature and fast and all around, "just works". Akka-HTTP is the future of stream based HTTP processing for Akka, but he current 1.0 release lacks the feature parity or performance of Spray. That will hopefully change with the 2.0 release. In the mean time, learn and use Spray. Akka-HTTP will feel familiar, and porting to it shouldn't be difficult once it's ready. So, in short, while learning, start with Spray. You'll have a good experience. Once Akka-HTTP matures, port. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: CPU-usage and a lot of actors with cluster-systems
But! It is important for me that the work of message 1 is done before > message 2 is handled. And isn't one of the benefits for the actor model and > akka itself, that I have the garantee to that message 1 is done before > message 2? > > Yes. And you might be in trouble there. In which case, at least on a per-actor basis, you wouldn't be able to use actor pooling or my Future dispatching solution. You have essentially 20k sequential processors. If having 20k concurrent actors doesn't provide enough concurrency to keep your system busy, I'm not sure what else you can do. Optimize the IO and the sequential time best you can. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: CPU-usage and a lot of actors with cluster-systems
I'm not saying use a lot of dispatchers. I'm saying that you should delegate to maybe 1 more dispatcher for your heavy work to unblock your actor's dispatching thread (and definitely another for blocking IO) allowing it to put more of your Actor's mailbox entries to work concurrently. In one of your posts you were afraid that actors "doing too much" could be causing things to slow down. If your actors are blocking on IO or some other lengthy task, a single actor won't be able to process message 2 until the receive block from message 1 has completed. By delegating heavy work to a different "heavy work" dispatcher, your Actor's dispatcher can start to process more requests from the mailbox. This allows more heavy/blocking tasks to be inflight. It doesn't make long tasks take less time, but hopefully will allow you to maximize your remaining resources either queuing up blocking work or processing the results as they return. To Patrik's point. 2 actors is already a lot of concurrency. Unless you broadcast 5 messages to each while the first message waits for 10s on a remote Geo call. Then you've spent some CPU queuing 100k requests, started the blocking IO for 20k of those and then wait for 10s. This is an exaggeration, but the idea here is that the time you spend waiting for IO drives down your overall CPU utilization. If you're looking to maximize CPU utilization you want all of your 100k requests in flight, saturating your IO pipe, and then processing results as they return. After a window of delay you should be processing results at the max speed of your network, which ought to keep your CPU busier than it is now. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: Cluster aware consistent hashing pool - works only on leader
Wait, what? So cluster sharding depends on shared mutable state across your cluster to work? AFAIknew the independent local nodes managed their state internally, communicated/coordinated via network protocolling and delegated to a master when it needed to determine a shard owner the first time. All of this allowing for local state, mutated locally with discovered cluster information. Is this not the case? If so, why, this seems contra to the Akka/Actor model and many other clustering strategies. My product is using a distributed journal, since we also use persistence along with cluster sharding. However we're plagued with clustering issues when rolling new code and ripple restarting nodes in the cluster. I was hoping this would go away in 2.4 when I could go back to a local journal for sharding state. The assumption being that an inter-cluster network partition was corrupting the shared state on the distributed journal. Currently the only way to recover my cluster in these situations is to shut down all nodes, remove all shard entries from dynamo and restart the cluster nodes, 1 by 1. This is on akka 2.3.10. java.lang.IllegalArgumentException: requirement failed: Region Actor[akka://User/user/sharding/TokenOAuthState#1273636297] not registered: State(Map(-63 -> Actor[akka://User/user/sharding/TokenOAuthState#-1778736418], 23 -> Actor[akka://User/user/sharding/TokenOAuthState#-1778736418], 40 -> Actor[akka.tcp://User@172.31.13.56:8115/user/sharding/TokenOAuthState#1939420601], 33 -> Actor[akka.tcp://User@172.31.13.59:8115/user/sharding/TokenOAuthState#-1679280759], 50 -> Actor[akka://User/user/sharding/TokenOAuthState#-1778736418], -58 -> Actor[akka.tcp://User@172.31.13.59:8115/user/sharding/TokenOAuthState#-1679280759], 35 -> Actor[akka.tcp://User@172.31.13.59:8115/user/sharding/TokenOAuthState#-1679280759], -66 -> Actor[akka.tcp://User@172.31.13.59:8115/user/sharding/TokenOAuthState#-1679280759], -23 -> Actor[akka.tcp://User@172.31.13.56:8115/user/sharding/TokenOAuthState#1939420601], -11 -> Actor[akka.tcp://User@172.31.13.56:8115/user/sharding/TokenOAuthState#1939420601]),Map(Actor[akka.tcp://User@172.31.13.59:8115/user/sharding/TokenOAuthState#-1679280759] -> Vector(35, -58, -66, 33), Actor[akka://User/user/sharding/TokenOAuthState#-1778736418] -> Vector(-63, 23, 50), Actor[akka.tcp://User@172.31.13.56:8115/user/sharding/TokenOAuthState#1939420601] -> Vector(-23, 40, -11)),Set()) On Wednesday, November 18, 2015 at 12:23:28 PM UTC-8, Patrik Nordwall wrote: > > Leveldb can't be used for cluster sharding, since that is a local journal. > The documentation of persistence has links to distributed journals. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: Cluster aware consistent hashing pool - works only on leader
On Thursday, November 19, 2015 at 12:32:38 PM UTC-8, Patrik Nordwall wrote: > > > > On Thu, Nov 19, 2015 at 8:06 PM, Jim Hazen <jimhaz...@gmail.com > > wrote: > >> Wait, what? So cluster sharding depends on shared mutable state across >> your cluster to work? AFAIknew the independent local nodes managed their >> state internally, communicated/coordinated via network protocolling and >> delegated to a master when it needed to determine a shard owner the first >> time. All of this allowing for local state, mutated locally with >> discovered cluster information. Is this not the case? If so, why, this >> seems contra to the Akka/Actor model and many other clustering strategies. >> > > That is correct, but the decisions taken by the coordinator must be > consistent, also when the coordinator crashes and fails over to another > node. Therefore, the state of the new coordinator instance must be > recovered to the exact same state as the previous coordinator instance. By > default the coordinator is using Akka Persistence to store and recover this > state. Distributed Data can be used as an alternative for storing this > state. > > The coordinator could broadcast or gossip its state to the cluster as it builds it. Or since the state the coordinator is really managing is the node owner of a particular shard, it could simply request this information from live cluster participants as part of coordinator election (or rather immediately after). This should rebuild an accurate shard topology. Coordination then proceeds normally. > >> My product is using a distributed journal, since we also use persistence >> along with cluster sharding. However we're plagued with clustering issues >> when rolling new code and ripple restarting nodes in the cluster. I was >> hoping this would go away in 2.4 when I could go back to a local journal >> for sharding state. >> > > That will not work, becuase when the coordinator fails over to another > node it will not have access to the leveldb journal used by the previous > coordinator, and it will then recover to wrong state and you will end up > with entity actors with same id running on several nodes. Not an option. > Are you talking about duplicate IDs within the new active cluster, or across both brains in case of a split? Dupes within a cluster can be prevented from happening if you rebuild the coordinator from the remaining nodes. Once the new coordinator takes over, it could easily broadcast a new authoritative ownership table to each node. If the node notices it had running IDs for shards it no longer owns, it can shut them down. This shouldn't happen much if the coordinator built its state from the active nodes in the first place. In the case of a split, the active brain doesn't have much control over the IDs on the other side. I needs to rely on a resolution strategy shared by both sides to ensure consistency (which is why you have the split brain resolver). > > >> The assumption being that an inter-cluster network partition was >> corrupting the shared state on the distributed journal. Currently the only >> way to recover my cluster in these situations is to shut down all nodes, >> remove all shard entries from dynamo and restart the cluster nodes, 1 by >> 1. This is on akka 2.3.10. >> > > We have seen two reasons for this issue. > 1) Bugs in the journals, e.g. replaying events in wrong order. > 2) Split brain scenarios (including network partitions, long GC, and > system overload) causing split of the cluster into two separate clusters > when using auto-downing. That will result in two coordinators (one in each > cluster) writing to the same database and thereby making the event sequence > corrupt. We recommend manual downing or Split Brain Resolver > <http://www.google.com/url?q=http%3A%2F%2Fdoc.akka.io%2Fdocs%2Fakka%2Frp-15v09p02%2Fscala%2Fsplit-brain-resolver.html=D=1=AFQjCNEASVmdwI1fAGRPkHdMwFR92lZYhQ> > > instead of auto-downing. > > Splits are inevitable and this implementation feels like it amplifies the problem. Once the inevitable happens you corrupt your state because the OSS cluster sharding impl lacks a Split Brain Resolver. What's worse is that this situation doesn't appear to be detected immediately. The split silently continues with both coordinators blindly writing state and sharding getting more and more corrupt. But you don't know how bad things are until you attempt to recover shard state from the journal, then blamo, you end up with no brains. It would be nice if the OSS product at least supported the static-quorum resolver. Or honored the akka.cluster.min-nr-of-members value when cluster membership changes. It would appear th
[akka-user] Re: CPU-usage and a lot of actors with cluster-systems
That's possible. Your actor won't be able to get more work until it completes its receive. You could: A: look into actor pool/routers. This will give you more receive blocks to work with, increasing concurrency. B: look into spending less time within your receive block, maybe by delegating the real work to a dispatched Future. If you go this route you'll have the ability to use a thread-pool-dispatcher for the longer/blocking work and your fork-join-dispatcher for your very fast receive executions. I tend to end up with option B. With my Spray services, using the dispatch directive takes the real work out of http dispatching thread. With actors that are mostly IO, since Spray IO is async, you get essentially the same thing. So when I find I have dense cpu intensive code, I offload that work into another dispatcher and let the actor drive work into this ecexutor as quickly as it can. At the very least, option B should help you drive up cpu utilization. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Cluster aware consistent hashing pool - works only on leader
You don't need your clustered actors to be persistence aware. So you are absolutely free to have sharded stateless actors, or manage state in some other way. The confusing part is that the cluster sharding internals requires akka persistence to be configured for at least one journal. The internals use this journal to persist internal shard state. Without this journal, cluster sharding simply silently fails to work (or used to). Follow the directions for setting up a local leveldb journal. It is well supported by akka and easy to set up. You can then stop worrying about persistence and just use the sharding functionality. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [akka-stream] how to control the throughput of a certain flow
Might want to look into mapAsync and mapAsyncUnordered for the linear parts of your slow graph. You can specify the desired parallelism as part of those calls. I was about to write that in general there isn't a "make fast" button for arbitrary graphs. However, that may not be strictly true. Just as there are SQL planners, there could be a materialization planner that applied some well understood optimization patterns under the hood. Even could have the ability to self tune the thread polling and throughput levels of a running flow based upon realtime metrics. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: CPU-usage and a lot of actors with cluster-systems
Also, the default akka-remoting utilizing Java serialization is dog slow. There are other threads that discuss swapping out the serializer with much faster ones. So if you're doing a lot of cluster sharding, your throughput may be bottlenecked on the remote inter-node IO. -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Akka-streams TLS-PSK support
Hmm. I'm not an expert here (just use BC for CloudFront). Have you tried
keeping your Java 8 SSLContext while combining that with the BC cipher
support?
Something more like your original, but with the BC ciphers.
Security.addProvider(new
org.bouncycastle.jce.provider.BouncyCastleProvider())
val context = SSLContext.getInstance("TLS")
context.init(null, null, null)
val cipherSuites =
NegotiateNewSession.withCipherSuites("TLS_PSK_WITH_AES_128_CBC_SHA")
val clientTls = SslTls(context, cipherSuites, Client)
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Akka-streams TLS-PSK support
Have you tried adding the BouncyCastleProvider as a security provider? Then making your normal JSSE calls? Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider()) -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Akka Stream and Http 2.0-M1 Released!
With these latest foundational changes, is akka-http 2.0 intended to be the 'spray parity' release? If not, is there a roadmap for an akka-http client as rich as Spray's and akka-http server as performant as Spray's? -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Ask Pattern vs Micro Actors?
http://letitcrash.com/post/96078609472/ask-tell-and-per-request-actors Maybe this will help answer your question. Actors are cheap, but not free. There are cases where using an actor-per-request pattern has more benefits than performance detractors. Per method call? I think that would depend on how often that method is called? Per-request may be a better granularity. If that request generates 100 method calls, then maybe you would notice actor overhead. That said, the ask pattern uses an optimized actor per call as well, and is often used wherever a synchronous callback is needed. What would be the benefit of your Micro Actor over Akka's built in support for 'ask'? I have a feeling that ask will be even more micro, due to internal optimizations not available to normal actors. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] memory issue in akka application
If you'd like to handle this in logback land and not deal with altering akka's logging strategy, you can also configure logback to work in bounded space and drop overflows as well. Take a look at logback's AsyncAppender. http://logback.qos.ch/manual/appenders.html You'll configure your logback appender normally and then wrap it in your async appender. For example: appender name=ASYNC_CONSOLE class=ch.qos.logback.classic.AsyncAppender queueSize=512 appender-ref ref=CONSOLE/ /appender -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Sharding problem when restarting Cluster
I see this issue happen whenever AWS has a network hiccup. I have a multi-node cluster behind a LB and akka cluster sharding along with akka persistence writing to a Dynamo journal. I'm currently on akka 2.3.11, which means the same shared Dynamo table used to store my persistent actors is also being used to store cluster information. I know of no way to prevent this until akka 2.4. I have my min-nr-of-members set to (nodes / 2) + 1. Things seem to work fine during clean node restarts and code deploys. However, I run into the same problem as the OP when AWS suffers an intermittent network partition. The nodes within the akka cluster can't fully communicate, yet the LB is able to reach all nodes. Cluster state is persisted into the same location, because that's unavoidable while using akka persistence for other things. Eventually cluster sharding gets upset and panics. Causing the error below to be repeated constantly until the full cluster is shut down and started back up cleanly. What should a developer do to isolate from split brained issues when using cluster sharding? min-nr-of-members appears to only be checked during cluster startup. However, once started and participating, what happens automatically when the cluster detects that the cluster has dropped below min-nr-of-members? I can attempt to guard against possible issues in application land by subscribing to the cluster events and taking some action. I'm not sure if there's anything I can do to prevent the cluster sharding internals from running into state however, since writing cluster state to a shared journal is unavoidable and network issues are unavoidable. 2015-08-02 05:33:30.138 05:33:30.138UTC [Device] ERROR akka.actor.OneForOneStrategy DeviceSvc-akka.actor.default-dispatcher-3 akka://DeviceSvc/user/sharding/UserDeviceIndexCoordinator/singleton/coordinator - requirement failed: Shard [2] already allocated: State(Map(-2 - Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203], 0 - Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203], 2 - Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203], -1 - Actor[akka.tcp://DeviceSvc@172.31.13.57:8108/user/sharding/UserDeviceIndex#855444773], 3 - Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203]),Map(Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203] - Vector(2, 3, -2, 0), Actor[akka.tcp://DeviceSvc@172.31.13.57:8108/user/sharding/UserDeviceIndex#855444773] - Vector(-1)),Set()) java.lang.IllegalArgumentException: requirement failed: Shard [2] already allocated: State(Map(-2 - Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203], 0 - Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203], 2 - Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203], -1 - Actor[akka.tcp://DeviceSvc@172.31.13.57:8108/user/sharding/UserDeviceIndex#855444773], 3 - Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203]),Map(Actor[akka.tcp://DeviceSvc@172.31.4.174:8108/user/sharding/UserDeviceIndex#-360404203] - Vector(2, 3, -2, 0), Actor[akka.tcp://DeviceSvc@172.31.13.57:8108/user/sharding/UserDeviceIndex#855444773] - Vector(-1)),Set()) at scala.Predef$.require(Predef.scala:219) ~[org.scala-lang.scala-library-2.11.6.jar:na] at akka.contrib.pattern.ShardCoordinator$Internal$State.updated(ClusterSharding.scala:1119) ~[com.typesafe.akka.akka-contrib_2.11-2.3.11.jar:2.3.11] at akka.contrib.pattern.ShardCoordinator$$anonfun$receiveRecover$1.applyOrElse(ClusterSharding.scala:1242) ~[com.typesafe.akka.akka-contrib_2.11-2.3.11.jar:2.3.11] at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36) ~[org.scala-lang.scala-library-2.11.6.jar:na] at akka.persistence.Eventsourced$$anonfun$akka$persistence$Eventsourced$$recoveryBehavior$1.applyOrElse(Eventsourced.scala:168) ~[com.typesafe.akka.akka-persistence-experimental_2.11-2.3.11.jar:na] at akka.persistence.Recovery$class.runReceive(Recovery.scala:48) ~[com.typesafe.akka.akka-persistence-experimental_2.11-2.3.11.jar:na] at akka.contrib.pattern.ShardCoordinator.runReceive(ClusterSharding.scala:1195) ~[com.typesafe.akka.akka-contrib_2.11-2.3.11.jar:2.3.11] at akka.persistence.Recovery$State$$anonfun$processPersistent$1.apply(Recovery.scala:33) ~[com.typesafe.akka.akka-persistence-experimental_2.11-2.3.11.jar:na] at akka.persistence.Recovery$State$$anonfun$processPersistent$1.apply(Recovery.scala:33) ~[com.typesafe.akka.akka-persistence-experimental_2.11-2.3.11.jar:na] at
Re: [akka-user] Re: How can I check the number of the entry actors spawned off by akka ShardRegion?
To increase capacity you'll be scaling out nodes, not shard regions. Shards are logical managers/buckets, they help organize and support the actual processing actors, but it's those actors that are distributed. You'll have one for each unique message destination, regardless of your number of shards. If you're worried about resource utilization, you're likely best off using standard JVM monitors and monitoring the Java application as a whole for resource usage. Do your metrics at that level and add hosts as required. Also, actors are already single-threaded, please see the Actor docs about how they work. Messages sent to an actor will be handled by a single thread. Within your receive block/method you can decide to use additional threads to farm out message processing work and increase actor throughput. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: How can I check the number of the entry actors spawned off by akka ShardRegion?
There will be a new actor activated for each unique messageId destination. Messages will arrive at that actor for processing. Actors are grouped into shards, which help manage those actors. If you send 100 messages to unique destinations across your cluster, they'll be handled by 100 unique actors. They'll be managed by 10 shards, across your ?? nodes. So seeing the creation of 100 actors and not 10, is expected. To get the number of actors managed by the shardRegion I assume you'd need to discover the actors representing the shard regions and then size() their children. I believe there is an internal messaging protocol for doing this, but I don't have the docs in front of me. You probably also find actors under the /user/sharding/ namespace using normal actor selection. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: How can I check the number of the entry actors spawned off by akka ShardRegion?
It might be possible to find out, however why is this important to you? The point of Akka cluster sharding is that shard management, handler actor management and message routing, is all transparent. I use this facility in my work and I don't know what nodes are handling what requests (sure I can track the logs and find out). And I don't mind not knowing. Akka does a lot of work behind the scenes to rebalance as nodes add/drop from the cluster. From a client perspective, things just work. Are you just curious about behavior, or have specific plans for this information? On Mon, Jul 20, 2015 at 12:48 PM, Yifei sunyifei1...@gmail.com wrote: Thanks Jim, It helps a lot. Assuming I have 3 shardRegions running on 3 nodes with the same shard name. Can I know the relationship between a shard and a shardRegion? For example, can I know if shard#3 is under shardRegion@host2? Thanks, Yifei On Monday, July 20, 2015 at 3:16:24 PM UTC-4, Jim Hazen wrote: There will be a new actor activated for each unique messageId destination. Messages will arrive at that actor for processing. Actors are grouped into shards, which help manage those actors. If you send 100 messages to unique destinations across your cluster, they'll be handled by 100 unique actors. They'll be managed by 10 shards, across your ?? nodes. So seeing the creation of 100 actors and not 10, is expected. To get the number of actors managed by the shardRegion I assume you'd need to discover the actors representing the shard regions and then size() their children. I believe there is an internal messaging protocol for doing this, but I don't have the docs in front of me. You probably also find actors under the /user/sharding/ namespace using normal actor selection. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to a topic in the Google Groups Akka User List group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/e0fqWMFzZuY/unsubscribe. To unsubscribe from this group and all its topics, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Akka Streams - How to define a Flow depending on the data coming from the Source? (AKA dinamically)
If you already have the solution it shouldn't be hard to put Streams in front of it to help manage back pressure. Call your code from within the sink and apply whatever flow controls you need to the source ahead of it. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Akka Cluster Performance
Akka clustering is built on top of akka remoting. The default serialization used by remoting is terribly slow. Look into swapping it out with a third party serializer. There are some linked/mentioned in the remoting docs. Please post any results if a serializer change helps you. I know in my project a 200k/sec spray http server gets 5k/s out of the default remoting layer. Fortunately 5k is good enough for that service, so I haven't messed with the serializer. Others have had good luck and Kryo seems popular. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Message Broadcasting in an akka cluster
You could implement the mediator yourself and include all your business logic there. That said, why do this? What's the fear of an extra actor dealing with these concerns and providing separation? I don't think it's possible to use an external implementation and not have separation. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] ShardRegion vs EventBus
Tried looking at the DistributedPubSub module? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Best Strategy to Aggregate Multiple Actors Response
Does this work for you? http://doc.akka.io/docs/akka/snapshot/contrib/aggregator.html -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] 500 ISE for Client Side Error Bug Report
Just because the HTTP protocol doesn't define the semantics doesn't mean that RESTful services aren't allowed to take this on themselves, and some do. Kinda like in the US where rights not reserved for the federal government fall to the states. ElasticSearch is one such example: http://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html. They use GET bodies for the REST calls in which they are performing a search given a complex JSON DSL. Commonly this would be performed with a GET with request params. They support this to a certain degree, but they also have an extremely rich JSON DSL for modeling complex queries/filters. This DSL would hard or impossible to express via request params, and since this is still fundamentally a GET operation, they support JSON as the request body. They also support POST as a fallback to work around broken/proper (depends on your point of view I guess) clients. So, in some of my apps I could revert to this POST fallback. However, some services are service wrappers around ES. They provide their own additional functionality to existing ES REST APIs, but attempt to not interfere with existing call semantics. So if their clients are sending GET search requests, they need to be able to accept those calls within their service and proxy them downstream. With the advent of akka-streams and akka-http this proxy becomes even more robust, with better tools to describe data flows and handle backpressure. But only if GETs are allowed to have bodies. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] How does akka actor helps in achieve concurrency without locking/syncronization
On Wednesday, April 15, 2015 at 9:17:44 PM UTC-7, Rohit Jain wrote: On Wednesday, April 15, 2015 at 10:37:36 PM UTC+5:30, Michael Frank wrote: On 04/15/15 04:47, Rohit Jain wrote: How does using akka actor helps in achieving concurrency without use of any external locks? Is it because at a moment only one message can be processed for one actor ? Hence no 2 threads can be acting upon same actor ? that is correct. processing within the receive loop of an actor is single threaded. How does akka make sure that while processing a certain message, no other message thread will be processed or there will be no context switch ? Or it just uses low level constructs of locking on a certain message before processing it, even though context switch for that message thread happens, it wont release the lock till message is processed (This can cause trouble in blocking I/O). Akka uses an ordered mailbox to store messages waiting to be processed. It then uses a single thread to roll through this mailbox, calling your actor's receive method. You don't need locking because there can't be contention on this mailbox. If the thread is context switched to handle another actor, that's fine. The original actor's messages will need to wait until its next time slice to process another message. If you block within your receive, you can/will have problems. Keep your receive non-blocking and delegate heavy lifting (even if it isn't blocking) to another dispatcher to achieve max throughput. If that is the case, then maximum number of threads we require is equal to number of actors in our system. if all your actors perform blocking work in their receive loop, then that would be true. however, you should design your application to be non-blocking (http://www.reactivemanifesto.org/glossary#Non-Blocking). if you need to perform blocking work, then you should use Futures ( http://doc.akka.io/docs/akka/2.3.9/scala/futures.html) to schedule the work on a separate, dedicated dispatcher ( http://doc.akka.io/docs/akka/2.3.9/scala/dispatchers.html). You can then pipe the result of the Future to an actor ( http://doc.akka.io/docs/akka/2.3.9/scala/futures.html#Use_With_Actors) for further processing if necessary. -Michael -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: 500 ISE for Client Side Error Bug Report
I was originally thinking that this would be inconvenient for ES clients. I could work around that. However for anyone attempting to build a transparent proxy in front of ES (where they can't control client calls), this would be an impossible to fix solution if a GET with body was forbidden. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: 500 ISE for Client Side Error Bug Report
Hmm... Spray currently allows this and products like ElasticSearch promote GET bodies. This will be a breaking change for me as well once I migrate to akka-http. How opinionated are the developers here? Can this be more of a guideline than a rule? :) -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Akka EC2 remoting
Try setting JVM system params. For example, an early demo that I hacked up looked like this. It isn't on EC2, but I don't think that EC2 is really your problem. Hack out the stuff you don't care about and omit the -J if not using sbt. Instance 1: sbt -J-Dakka.remote.netty.tcp.hostname=localhost -J-Dakka.cluster.seed-nodes.0=akka.tcp://CloudObjects-core@localhost:2552 -J-Dcom.sun.management.jmxremote.port= -J-Dcom.sun.management.jmxremote.authenticate=false -J-Dcom.sun.management.jmxremote.ssl=false -J-Dcloudobjects.journalLocation=akka.tcp://CloudObjects-core@localhost:2552/user/journal run Instance 2: sbt -J-Dakka.remote.netty.tcp.hostname=localhost -J-Dakka.remote.netty.tcp.port=2553 -J-Dakka.cluster.seed-nodes.0=akka.tcp://CloudObjects-core@localhost:2552 -J-Drole=backend -J-Dcloudobjects.journalLocation=akka.tcp://CloudObjects-core@localhost:2552/user/journal run -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Akka EC2 remoting
Looks like you're trying to connect to a seed that isn't available. Be sure that your remote.netty.tcp host and port settings match your seed node setting. They don't seem to here. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: akka-streams how to chunk ordered stream based on common property efficiently
This should still be easy with akka-streams. You'll probably need to write a custom stage though. Check out the documentation on custom stages along with the cookbook examples. I believe there is an example that implements a batching collector like this. It's not 100% what you're looking for, but should be easy to modify. There is a way to know when the stream runs out, so sending that last batch shouldn't be an issue. The examples demonstrate that functionality as well. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Stream Materialization, a larger discussion.
Hi Roland, Thank you for the reply and sorry for the delay. The way I'd like to look at stream materialization is as a facility for runtime information discovery and action execution. This mechanism sits next to the main information flow and at times has the ability to influence that flow. A management sub-channel is probably not technically accurate, but I hope you get my meaning. The flexibility of this discovery/management facility in its current form is what I'm worried about. There are good examples in the documentation for how to expose both discovery and management operations via materialization. val (promise, cancellable, future) = r11.run() promise.success(0) cancellable.cancel() future.map(_ + 3) This does a pretty good job of getting these monitoring and management concerns across the stream boundary. Issue 16168 will hopefully address access to these values from within a stream. I'm tempted to use this facility for JMX-like functionality on a stream. But if you aren't careful you'll run into a problem with shape. Let's say that version 1 of my lib ships out with a throttler stage. That stage, like in the docs, throttles at 1/s and exposes a cancellable management object. That's great, until folks start asking for the ability to change the throttle rate at runtime as well. This functionality could of course be implemented in v2, however the shape of my stage's materialized object then needs to change. So I replace my Cancellable object with an Operations object that has both cancel() and updateRate(newRate) methods. I push this out in v2, but this is a breaking change for everyone in v1 because the materialized shape is part of the interface and used or not is still there. The folks that make vanilla use of my stage are annoyed because they're forced to recompile against a feature they weren't even using. Even if I have the foresight to build stages with an easier to evolve materialization shape (maybe an object that exposes Operations and Informations in a more flexible way) I don't have the ability to benefit from foundational enhancements by the stream library itself. For example, even when I finish rolling my own JMX concept, I can't ask Akka to help me add to this functionality with their own materialization details (where would such information be placed?). Currently (AFAIK) Akka Streams doesn't expose the ability to discover or manipulate per-stage runtime materialization, as in the materialization provided by the materializer. Can't discover the current default internal buffer size, can't modify this size, update thread pool settings, access stats metrics of message flow through the internal actors, or execution time taken. And the lack of this functionality today is fine. Over time more and more base operations and informations could be exposed by the runtime, and developers could add their own operations and informations on top of this base. But this only works if there is a base and this base is of some flexible shape. I guess what I'm asking for is for the Akka team to look at the materialization value system as more of a per-stage command and control system. Something that would benefit from Akka prescribing a management methodology instead of leaving it completely open. IMHO the current form, while open and type safe, ends up being brittle. There isn't a methodology or framework by which Akka can raise all ships in the future (by providing certain standard ops/infos) and developers without management framework development experience will accidentally find themselves painted into a corner wrt to the shape of their materialization values. I think I'll stop here, but next up would be a discussion on the proper way to interact with materialization values. Even if I get everything that I ask for and the shape turns into some standardized object exposing information and actions, this still ends up being shared mutable state bleeding out of the Stream with a now undefined concurrency model. Wouldn't it be better to expose some sort of action sink and information source instead of a single shared object? Thanks for your time, Jim On Sunday, March 8, 2015 at 11:24:58 AM UTC-7, rkuhn wrote: Hi Jim, thanks for starting the discussion! Replies inline. 5 mar 2015 kl. 21:30 skrev Jim Hazen jimhaz...@gmail.com javascript:: I've been following Akka Streams for bit. I've noticed the changes in M4 and of course read the materialization sections of: http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M4/scala/stream-flows-and-basics.html I have tremendous respect for the Akka team and understand that getting this right is hard. But, are we there yet? Basically yes, but one thing is still missing https://github.com/akka/akka/issues/16168. I hope that we’ll be able to fix the remaining things without further disruptive changes. Originally the concept was Map based
[akka-user] How to use Cluster Sharding, Please do reply it's very urgent
Start here: http://doc.akka.io/docs/akka/2.3.9/contrib/cluster-sharding.html#cluster-sharding -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] How to use Cluster Sharding, Please do reply it's very urgent
Start hers: http://doc.akka.io/docs/akka/2.3.9/contrib/cluster-sharding.html#cluster-sharding -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Split Stream
Have you looked into writing a custom router component? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: akka-http: combine path together
I was confused. I thought you were looking for something like:
path(v2.0 / tokens) {
post {
complete(it works)
}
} ~
path(v2.0 / somethingelse) {
post {
complete(it works)
}
}
Otherwise with
path(v2.0) {
path(tokens) {
You're matching against path(v2.0) and then further trying to also match
against path(tokens). Since a path can't start with both v2.0 and
tokens at the same time, this always fails. There are other path
directives you can use for this type of processing and I think you've
already discovered some of them.
akka-http is based off of spray concepts. Spray's documentation is
currently much better than akka-http. You should probably start
here http://spray.io/documentation/1.2.2/spray-routing/ to learn more about
what routing directives are available and how to compose them together.
All directives may not exist yet in akka-http, but the sections on the
Path DSL should be valid.
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups Akka
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: Problems using a Remote Actor
Have you followed all of the instructions for interacting with remote actors here: http://doc.akka.io/docs/akka/2.3.9/scala/remoting.html ? From the error it appears to me that: 1. You are attempting to create a remote actor from a peer 2. This peer isn't able to properly interact with [akka.tcp:// RemoteActorSystem-Client@127.0.0.1:55003] Are you using a Remote or Clustered ActorRefProviders on both nodes? Is your remote transport properly configured to run on those ports? Is it starting properly? Is your networking configured to allow access to those ports (software firewalls on a node have burned me in the past). Does the RemoteActorSystem-Client AS exist on that port, or is some other AS running there? There's a lot to look at, but it ought to work once everything is wired properly. Maybe look for a working example (from Activator) and then adapt the working thing to your needs? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Stream Materialization, a larger discussion.
I've been following Akka Streams for bit. I've noticed the changes in M4 and of course read the materialization sections of: http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M4/scala/stream-flows-and-basics.html I have tremendous respect for the Akka team and understand that getting this right is hard. But, are we there yet? Originally the concept was Map based, with the ability for stages to set keys within the map and eventually the client had access to this map. Not really type safe, or clear as to what keys were available, or what their types were, etc. So that's not great. But on the flip side (and why I think it was reached for first) it was extensible. Now with M4, we are much more clear and explicit about what materialization each stage offers. This looks a lot better. But I'm worried about what happens when you attempt to evolve these materialization types later, especially for stages provided by the core library. I'm thinking down the road to Monk, where the goal is to provide a nice tracing abstraction and SPI. And if folks are interested in tracing Actors today, with Streams, they'll be interested in tracing materialized flows tomorrow. Given this current mechanism for exposing a per-stage materialization value, how later would we switch all of our Unit materialization values to materialization values that contain standard metrics? It isn't there today, but in the future a lot of runtime metrics could be gathered and exposed. With M4, I don't see a clean way of evolving or exposing new secondary materialization values over time. Secondary is in the eye of the beholder, but we have preferredMerge stages, and things like Try and Option that have success based affinity within flow processing. I can't prescribe a complete solution in this post but I would like to talk this out. Down the road how do we intend to support the exposure of new materialization based information and/or patterns? How would adaptive flows, flows that adapt to runtime realities, be implemented? How would one get access to a Stream of materialization values of a running Stream? I hope this is food for positive thought and a positive discussion. PS: This particular post uses tracing and metrics as an example, but I think there's a larger fundamental problem to be solved. Should each stage need to evolve from a materialization of Unit - tuple/wrapper - new tuple/wrapper. All the while likely breaking backward compatibility in the process. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Problem with dispatcher configuration
Check the nesting. Does test.my-dispatcher work? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: (re-)Chunking Source of ByteStrings, filling the gaps
Wouldn't migrating from a grouped to groupedWithin based chunker be the compromise that ppl are looking for? Perform chunk packing within a given window, but if you're waiting for too long, send what you have? You mention here and in the JIRA that this a conflate problem, but conflate deals with fast producers and slow consumers, and we have a potential problem with a slow producer keeping up with our chunk packer. I don't see conflate helping us here, but I'm willing to learn. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [Streams 1.0-M4] Compatibility with Akka-2.4-SNAPSHOT?
Is there an ETA for akka-http on 2.4? According to Mathias, Spray won't be ported to 2.4: https://groups.google.com/d/msg/spray-user/x0KdMn_7exE/B8Rp2xuSa2sJ and according to you akka-http also isn't yet ready for 2.4. I'd like to develop a REST service that takes advantage of cluster sharding with rememberEntries support and that's somewhat difficult at the moment. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [Streams 1.0-M4] From Source[Future[T], Unit] to Source[T, Unit]?
I think the answer is in the {Scala, Java}Docs of mapAsync:
Transform this stream by applying the given function to each of the
elements as they pass through this processing step. The function returns a
Future and the value of that future will be emitted downstreams. As many
futures as requested elements by downstream may run in parallel and may
complete in any order, but the elements that are emitted downstream are in
the same order as received from upstream.
The section on rate goes into greater
detail:
http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M4/scala/stream-rate.html
You can either go with the default demand as specified by your
Materialization settings, use the default settings (default values in
reference.conf), or manually specify a buffer downstream from your source
to have greater control over buffering and generated demand.
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups Akka
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] [Streams 1.0-M4] Compatibility with Akka-2.4-SNAPSHOT?
Are these 1.0-M4 modules intended to be compatible with akka-2.4-SNAPSHOT?
I'm having trouble passing a custom materializer to an
IncommingConnection.handleWithAsyncHandler(). The default materializer
seems to work fine, but I wanted to play with buffer sizes.
implicit val materializer = ActorFlowMaterializer()
val matSettings = materializer.settings.withInputBuffer(16, 32)
Http().bind(interface = 0.0.0.0, 5001).runForeach { conn =
conn.handleWithAsyncHandler(r =
Future(HttpResponse()))(ActorFlowMaterializer(matSettings))
}.onComplete(_ = actorSystem.terminate())
The fails at runtime with:
java.lang.NoSuchMethodError:
akka.pattern.AskableActorRef$.$qmark$extension(Lakka/actor/ActorRef;Ljava/lang/Object;Lakka/util/Timeout;)Lscala/concurrent/Future;
at
akka.stream.impl.ActorFlowMaterializerImpl.actorOf(ActorFlowMaterializerImpl.scala:185)
at
akka.stream.impl.ActorFlowMaterializerImpl$$anon$2.processorFor(ActorFlowMaterializerImpl.scala:92)
at
akka.stream.impl.ActorFlowMaterializerImpl$$anon$2.materializeAtomic(ActorFlowMaterializerImpl.scala:77)
at
akka.stream.impl.MaterializerSession$$anonfun$materializeModule$1.apply(StreamLayout.scala:336)
at
akka.stream.impl.MaterializerSession$$anonfun$materializeModule$1.apply(StreamLayout.scala:334)
I couldn't find akka-stream or akka-http modules as part of the akka-2.4
distribution. Is there a better way to experiment with akka-streams and
akka-http with akka-2.4? Aside from tweaking the handler's materializer,
is there another way to control the amount of buffering/demand generated by
the handleWith* constructs?
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups Akka
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Akka Cluster (and Cluster Client) on Kubernetes
Might want to look at this thread. https://groups.google.com/forum/#!topic/akka-user/wyA4xNfM5LM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Akka Cluster (and Cluster Client) on Kubernetes
Lol. Or not, since you're the OP. Oh well, need more coffee. On Monday, March 2, 2015 at 10:52:29 AM UTC-8, Jim Hazen wrote: Might want to look at this thread. https://groups.google.com/forum/#!topic/akka-user/wyA4xNfM5LM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: akka system shutdown in sub actor
Seems logical. From an actor within a running system you ask the system if it's dead. It says no, because it still has actors running (there's at least the one asking the question). Then, from outside the system you wait for it to terminate, and then check that it has terminated, which it confirms to be true. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: [Akka-stream] Distributed actor communication via Ask pattern or ActorPublisher/ActorSubscriber
To me, this isn't the correct solution. Passing Source(data) around seems wrong, it would seem dangerous for large files and impossible for remote actors if the source were pointer based. The tl;dr is that IMHO the correct solution, where you process indexing responses in a flow based way, is difficult to get done correctly right now. I'm waiting for this missing component: https://github.com/akka/akka/issues/16416. Read on for what turned into a pretty long winded brain dump... I think of Actors as little (possibly stateful) message processors. They react to messages, and sometimes produce directed response messages. These messages generally ought to be small and immutable. I see Streams and Flows as higher level constructs. As if someone stepped back, looked at all the little discrete message processors that he had and wanted to model and materialize a processing engine out of those parts. In your case I'd probably break things into a few components. Files, IndexingManager, FileToLinesGenerator, any number of LineToX mappers along with XtoY mappers, YtoZ mappers, etc. Finish things off with an Indexer component. You want to get files indexed. That overall responsibility would be owned by the IndexingManager. He'd be responsible for constructing the high level materialized flow, from the building blocks at hand. Maybe he's an actor that gets a NewFile(location) message whenever some file shows up somewhere. He than sets up and executes the indexing flow. I'd probably use a scala.io.Source to get an iterator over that file content. Source(file).getLines() or zipWithIndex() if you want line number information as well (I would). Construct your FlowGraph with whatever intermediate steps you like, and eventually get a small message over to your Indexer. Maybe that message looks like IndexingRequest(fileMetadata, lineNumber, line). It then indexes this bite sized message (also able to be sent across remote boundaries). Eventually the flow completes and your IndexingManager does some final book keeping and declares victory. This is how I'd model an overall indexing system. And it would probably work reasonably well in one direction, from FileData - Indexing. But we want to have the Indexer tell someone about the indexing results. For me, I'm waiting for https://github.com/akka/akka/issues/16416 to come out. It should make custom request/response protocols much easier to model. Until then, I'm probably not going to mess with cyclic flow graphs with Akka Streams. In the mean time, if I needed custom request/response logic I'd build non-Streams logic into my IndexingManager. I'd have the IndexingManager manually buffer requests/responses and simply have some basic ack-based back-pressure (send a few indexing requests blind, and then wait for responses to show up before I sent more requests). This assumes that the IndexingManager is going to do something special with responses. If all it's going to do is log errors, or send them to a queue for followup and possible reprocessing, the Indexer could do that and you can have a unidirectional flow managed by Akka Steams again. On 2nd thought, the blind send of IndexingRequests is probably poor form, knowing a bit about how Streams works I'd probably start the conversation off with a single IndexingSessionRequest that basically says, hey I want to send stuff, how many can you handle. The response can help the IndexingManager set internal buffers as well. The Indexer can respond with IndexingSessionResponse(send me 5) or IndexingSessionResponse(too busy, ask again in 5.seconds). -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: akka-streams - How to define a Source from an arbitrary event stream?
I'd also be interested to know what the best practice patterns for bridging internal/external stream boundaries are. In this case there isn't yet a well understood pattern for external entities to dynamically feed into a stream source. There is likewise no easy way for an external library to tap a sink. How would one model the faucet in my house? It's a well backpressured system, yet I can turn a knob and take(1.cup) off the stream and process it in an external system. In your particular case, I'm thinking that using an ActorPublisher might be the best way to go. Your custom ActorPublisher would act as a Stream source, but also be known externally to the stream as a normal actor. It would provide a generic out of band mechanism for other actors to send messages to it. Newly arriving messages would become available for stream consumption once demand was triggered. You can then subscribe this actor to whatever external event streams you'd like. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: going from using akka in a play application to scaling the akka to multiple nodes, what to plan for?
Take a look at cluster sharding. By far the easiest way to get started with clustered/distributed actors. http://doc.akka.io/docs/akka/2.3.9/contrib/cluster-sharding.html#cluster-sharding On Sunday, February 15, 2015 at 7:58:02 AM UTC-8, gitted wrote: Currently I am a play application that I use akka with. All is fine since my application is running on a single server. If I want to have multiple nodes, how exactly do I go about scaling my akka side of things? Currently it assumes all akka actors are on the same node. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] akka-http long-lived connection backpressure problem
Thanks for creating the issue. Please note that there were actually two issues discovered. The client flow issue was logged, but the server request parsing issue wasn't. The rest of this post is really just going through my thought process, so that the group can benefit from an analysis of this problem and a discussion of its solution as a case study of larger stream processing problems/solutions. The more I dig into this issue, the more I understand streams. Still the more I dig, the more it appears you've got a problem on your hands with this type of flow. My initial thought was to implement this system as a combination of two streams, your outgoing request stream and your incoming response stream, each stream processing their things at full speed. Now I'm still learning the lingo and don't want to unintentionally mix my metaphors. Essentially I'd expect you give something a Source[HttpRequest] and get a Source[HttpResponse] back. Each with the possibility of independent supply/demand rates. This might work in some ways, but presents other challenges. Based on what I *think* the original flow is trying to do, there appears to be a fundamental problem. My (maybe naive) solution treats request and response streams as being independent (from the point of view of the client). Stream ordering is (supposed to be) guaranteed by HTTP 1.1 pipelining. What the original flow attempts to do is remember the HttpMethod of the original request and join it back up with the response. I'm guessing you want to do this for pipeline error detection as well as auto-recovery of recoverable requests. You need the original request's method to know whether or not a response failure is auto-recoverable (GET typically is, POST isn't, etc). Unfortunately, now we're caught between two infinites and the amount of state stored depends on the difference in the two rates. This is a problem for a bounded memory system. We have some tools for dealing with rate differences. My broken solution, introduces a dropping buffer. A better broken solution might be to conflate the httpMethod stream and collapse like methods (recoverable, non-recoverable) together. This somewhat reduces the problem, but doesn't solve it. In order to deterministically match the req/resp we'd need to conflate and still track the overall shape of the conflated httpMethod stream, i.e. (Rs, NRs, Rs, NRs, Rs), where R = recoverable, NR = non-recoverable. Still, we have an issue where the potential rate of change in the conflated httpMethod stream's shape exceeds our buffer. Now we're back to our original problem. The rate difference is significantly better, but not good enough to run in constant space. I'm not sure it's possible to get this into constant space because we can't reduce the problem down to a pure conflation, can't drop and don't want to error. At this point, it's probably acceptable to rate limit the request flow, while the conflated httpMethod (merge or zip) works to drain responses. This means that if we want the client to be able to associate response with request and do nice things for us, we need to accept the possibility that it will limit the rate of accepted requests to keep the memory required to associate request with response bounded. If, on the other hand, we didn't want the client to perform pipeline error detection or request auto-retry, we should be able to use the two independent streams model. Even in this model, we'd have appropriate backpressure. The server will never sink requests faster than it can handle, nor will it produce responses faster than it can handle. The two things are in fact related, however that relationship isn't visible to the client, it simply manifests itself as a rate differential in our two otherwise independent streams. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] akka-http long-lived connection backpressure problem
Now that I think even more on the problem the more it seems that we're trying to solve too much within a single flow. There are concepts that ought to be independent that we're forcing to become dependent. I still like the independent streams model. I'd start with that, but with a realization that this is a low-level protocol. A higher level protocol, ought to deal with more advanced concepts, like auto-retry. There's a *big assumption* that the httpMethod bypass was to support things like that. If not, then I'm going to start getting farther and farther off base. Hmm. Yet, even with a full duplex stream of requests and responses, implementing higher level stateful interactions tends to run into similar problems. remembering what actor ought to receive a given response once it's read off the stream. Easiest way to solve this would be to encapsulate that addressing state into the response being returned. However, we need to work within the confines of the HTTP protocol and I'm not sure the best way to implement that handback in that case (cookies?). -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: Akka Cluster Actor Failover and Rebalancing, is it possible?
There are probably a few ways to do this. Most straight forward might be to start a local actor as part of each node's startup code. Then use Akka clustering to observe node failures. Use some deterministic strategy to reallocate that actor on some other host (let's say failed +1 in your ring). When the old node comes back up it starts its actor normally and then asks node current +1 to remove that actor from its AS. Another way might be to use cluster sharding with a completely balanced distribution and rememberEntries = true. Akka would do more magic for you. This may not totally be what you want. This would guarantee 1 actor per server, however which actor was on which server wouldn't be deterministic. You might be able to provide some additional allocation strategy to Akka sharding that would give you additional control over placement, but that seems like a rabbit hole to me. Even if I could get it to work, I've worked too hard vs. option 1 and it may not be clear to the next developer what's going on. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
